21 KiB

Raw Blame History

TENT C++ API Reference

Overview

This page summarizes the C++ APIs in mooncake-transfer-engine/tent/include/tent/transfer_engine.h. It follows the same structure as the Transfer Engine API documentation.

For conceptual background, see TENT Overview.

Prerequisites and API modes

Build with -DUSE_TENT=ON to enable TENT.
TENT provides two API surfaces:
- TENT-native API (documented below).
- TE-compatible API via the compatibility shim (set MC_USE_TENT=1). For TE-compatible API, see TE C++ API Reference

Core APIs vs Advanced APIs

Core APIs form the minimal path to move data: create the engine, register memory, open segments, submit transfers, and query status.
Advanced APIs are optional; they help with segment export/import, notifications, and engine introspection.

Key Differences from Transfer Engine

TENT redesigns the API surface based on different design goals. The following table summarizes the key differences and the rationale behind them.

Aspect	Transfer Engine (TE)	TENT	Rationale
Initialization	Two-step: constructor + `init(metadata_conn_string, server_name, ...)`	Single-step: constructor with `Config` object or config file path	Simplifies initialization; centralizes configuration in a single place
Transport Management	Manual: `installTransport()`, `uninstallTransport()`, `getTransport()`	Removed from public API	TENT performs dynamic transport selection at runtime; applications should not manage transports directly
Memory Allocation	Not provided; users allocate memory externally	`allocateLocalMemory()` / `freeLocalMemory()`	Provides integrated memory management with automatic registration
Memory Registration	`registerLocalMemory(addr, len, location, remote_accessible, update_metadata)`	`registerLocalMemory(addr, size, Permission)` or `MemoryOptions`	Cleaner semantics; `Permission` replaces boolean flags; metadata updates are internal
Segment Discovery	Via metadata service; manual cache sync with `syncSegmentCache()`	Internal control plane (metadata type = `p2p` or central); no manual sync	Simplifies metadata management; discovery handled inside runtime
Error Handling	Mixed: some APIs return `int`, others return `Status`	Consistent: all APIs return `Status`	Uniform error handling across the API
Topology/Introspection	Exposed: `getLocalTopology()`, `getMetadata()`, `checkOverlap()`, etc.	Internalized; not exposed	TENT handles topology and path selection internally; reduces API complexity

Design Philosophy

Declarative over Imperative: Applications describe what data to move, not how to move it. Transport selection, path optimization, and failure handling are delegated to the TENT runtime.
Single Initialization: Instead of a two-phase constructor + init() pattern, TENT uses configuration objects that fully describe the engine state at construction time.
Internal Metadata Management: TENT does not expose metadata internals (TransferMetadata, Topology). Segment discovery is handled by the control plane (P2P or central), and the cache is managed automatically.
Consistent Error Model: All TENT APIs return Status objects, eliminating the mixed int / Status return types in TE.

API Mapping: TE to TENT

For users migrating from Transfer Engine, the following table shows how TE APIs map to TENT APIs. TENT provides a backward-compatible shim (MC_USE_TENT=1) that allows existing TE code to run on the TENT runtime.

Transfer Engine API	TENT API	Notes
Initialization
`TransferEngine(auto_discover)`	`TransferEngine()`	TENT ignores `auto_discover`; discovery is always automatic
`init(conn_string, server_name, ip, port)`	Constructor with `Config`	Config sets `metadata_type`, `metadata_servers`, `local_segment_name`
`freeEngine()`	Destructor	Resources released automatically on destruction
Transport
`installTransport(proto, args)`	Not available	Transport selection is internal to TENT
`uninstallTransport(proto)`	Not available	—
`getTransport(proto)`	Not available	—
Memory
`registerLocalMemory(addr, len, location, remote_accessible, update_metadata)`	`registerLocalMemory(addr, size, MemoryOptions)`	`MemoryOptions.location` replaces `location`; `MemoryOptions.perm` replaces `remote_accessible`
`unregisterLocalMemory(addr, update_metadata)`	`unregisterLocalMemory(addr)`	Metadata update is internal
`registerLocalMemoryBatch(buffer_list, location)`	`registerLocalMemory(addr_list, size_list, MemoryOptions)`	Batch API uses vectors instead of `BufferEntry`
`unregisterLocalMemoryBatch(addr_list)`	`unregisterLocalMemory(addr_list)`	—
Not available	`allocateLocalMemory(addr, size, location)`	TENT-only: allocates and registers in one call
Not available	`freeLocalMemory(addr)`	TENT-only: frees allocated memory
Segment
`openSegment(segment_name)` → returns handle	`openSegment(handle, segment_name)` → via output param	Return style differs; `Status` indicates success/failure
`closeSegment(handle)`	`closeSegment(handle)`	Same semantics
`removeLocalSegment(segment_name)`	Not available	Segment lifecycle managed internally
`CheckSegmentStatus(sid)`	Not available	Status checking is internal
`syncSegmentCache(segment_name)`	Not available	Cache sync is automatic
Not available	`exportLocalSegment(shared_handle)`	Declared in TENT API but currently not implemented
Not available	`importRemoteSegment(handle, shared_handle)`	Declared in TENT API but currently not implemented
`getSegmentInfo(handle, info)`	`getSegmentInfo(handle, info)`	Same semantics
Batch & Transfer
`allocateBatchID(batch_size)`	`allocateBatch(batch_size)`	Renamed
`freeBatchID(batch_id)`	`freeBatch(batch_id)`	Renamed
`submitTransfer(batch_id, entries)`	`submitTransfer(batch_id, request_list)`	`TransferRequest` → `Request`
`submitTransferWithNotify(batch_id, entries, notify_msg)`	`submitTransfer(batch_id, request_list, notifi)`	Unified API with optional notification
`getTransferStatus(batch_id, task_id, status)`	`getTransferStatus(batch_id, task_id, status)`	Same
`getBatchTransferStatus(batch_id, status)`	`getTransferStatus(batch_id, status)`	Overloaded; single `TransferStatus` output = overall status
Not available	`getTransferStatus(batch_id, status_list)`	TENT-only: get all task statuses at once
Notification
`sendNotifyByID(target_id, notify_msg)`	`sendNotification(target_id, notifi)`	Renamed; uses `Notification` struct
`sendNotifyByName(remote_agent, notify_msg)`	`openSegment()` + `sendNotification()`	TENT requires explicit segment handle
`getNotifies(notifies)`	`receiveNotification(notifi_list)`	Renamed; uses `Notification` struct
Introspection
`getLocalIpAndPort()`	`getRpcServerAddress()` + `getRpcServerPort()`	Split into two methods
`getRpcPort()`	`getRpcServerPort()`	Same
`getMetadata()`	Not available	Metadata internals not exposed
`getLocalTopology()`	Not available	Topology internals not exposed
`checkOverlap(addr, length)`	Not available	—
`setAutoDiscover(auto_discover)`	Not available	Always enabled (ignored under `MC_USE_TENT`)
`setWhitelistFilters(filters)`	Not available	Configure via `Config`
`numContexts()`	Not available	—
Not available	`available()`	TENT-only: check if engine initialized successfully
Not available	`getSegmentName()`	TENT-only: get local segment name

Using the Backward-Compatible Shim

Existing Transfer Engine code can run on TENT by setting the environment variable:

export MC_USE_TENT=1

When this variable is set, the mooncake::TransferEngine class internally delegates to mooncake::tent::TransferEngine. Most TE APIs are translated automatically. APIs that have no TENT equivalent (e.g., installTransport, getMetadata) become no-ops or return placeholder values.

Core APIs

Core Usage Path (C++)

Most integrations follow a short, repeatable path: create the engine, register local memory, open a target segment, submit a batch transfer, poll status, and finally free resources. A minimal example is shown below.

#include "tent/transfer_engine.h"

using mooncake::tent::TransferEngine;
using mooncake::tent::Request;
using mooncake::tent::TransferStatus;
using mooncake::tent::SegmentID;
using mooncake::tent::BatchID;

// Create engine (loads config from default path or environment)
TransferEngine engine;
if (!engine.available()) {
    // handle initialization failure
}

// Allocate and register local memory
void* local_addr = nullptr;
size_t length = 1024 * 1024;  // 1MB
engine.allocateLocalMemory(&local_addr, length, "cuda:0");

// Open remote segment
SegmentID remote_segment;
engine.openSegment(remote_segment, "remote_node");

// Prepare transfer request
Request req{};
req.opcode = Request::WRITE;
req.source = local_addr;
req.target_id = remote_segment;
req.target_offset = 0;
req.length = length;

// Allocate batch and submit transfer
BatchID batch = engine.allocateBatch(1);
engine.submitTransfer(batch, {req});

// Poll for completion
TransferStatus status;
do {
    engine.getTransferStatus(batch, status);
} while (status.s == mooncake::tent::PENDING);

// Cleanup
engine.freeBatch(batch);
engine.closeSegment(remote_segment);
engine.freeLocalMemory(local_addr);

Constructors

TransferEngine();
TransferEngine(const std::string config_path);
TransferEngine(std::shared_ptr<Config> config);

Constructs a TENT Transfer Engine instance.

Default constructor: Loads configuration from the default path or environment variables.
config_path: Path to a JSON configuration file.
config: A pre-constructed Config object for programmatic configuration.

The engine is ready to use after construction if available() returns true.

Types

Request

The core API provided by TENT is submitting a group of asynchronous Request tasks through the submitTransfer interface, and querying their status through the getTransferStatus interface.

struct Request {
    enum OpCode { READ, WRITE };
    OpCode opcode;
    void* source;
    SegmentID target_id;
    uint64_t target_offset;
    size_t length;
};

opcode: READ copies data from <target_id, target_offset> to source; WRITE copies data from source to <target_id, target_offset>.
source: Local buffer address, must be registered via registerLocalMemory or allocated via allocateLocalMemory.
target_id: Segment ID obtained from openSegment.
target_offset: Offset within the target segment.
length: Number of bytes to transfer.

TransferStatus

enum TransferStatusEnum {
    INITIAL,    // Not yet started
    PENDING,    // Transfer in progress
    INVALID,    // Invalid parameters
    CANCELED,   // Transfer canceled
    COMPLETED,  // Transfer completed successfully
    TIMEOUT,    // Transfer timed out
    FAILED      // Transfer failed
};

struct TransferStatus {
    TransferStatusEnum s;
    size_t transferred_bytes;
};

s: Current status of the transfer.
transferred_bytes: Number of bytes successfully transferred (lower bound).

Data Transfer

TransferEngine::allocateBatch

BatchID allocateBatch(size_t batch_size);

Allocates a BatchID that can hold up to batch_size transfer requests.

batch_size: Maximum number of requests that can be submitted under this batch.
Return value: A valid BatchID on success.

TransferEngine::submitTransfer

Status submitTransfer(BatchID batch_id,
                      const std::vector<Request>& request_list);

Submits transfer requests to the specified batch. Requests are executed asynchronously.

batch_id: The batch to submit requests to.
request_list: Vector of Request objects.
Return value: Status::OK() on success; otherwise a non-OK status.

TransferEngine::getTransferStatus

// Get status of a single task
Status getTransferStatus(BatchID batch_id, size_t task_id,
                         TransferStatus& status);

// Get status of all tasks in a batch
Status getTransferStatus(BatchID batch_id,
                         std::vector<TransferStatus>& status_list);

// Get overall batch status
Status getTransferStatus(BatchID batch_id, TransferStatus& overall_status);

Queries the status of transfer requests.

batch_id: The batch to query.
task_id: Index of the specific task (for single-task query).
status / status_list / overall_status: Output parameter(s) for status.
Return value: Status::OK() on success; otherwise a non-OK status.

TransferEngine::freeBatch

Status freeBatch(BatchID batch_id);

Releases a batch. All transfers in the batch must be completed before calling this.

batch_id: The batch to release.
Return value: Status::OK() on success; otherwise a non-OK status.

Memory Management

TransferEngine::allocateLocalMemory

Status allocateLocalMemory(void** addr, size_t size,
                           Location location = kWildcardLocation);

// Advanced version with MemoryOptions
Status allocateLocalMemory(void** addr, size_t size,
                           MemoryOptions& options);

Allocates memory that is automatically registered for transfers.

addr: Output pointer to the allocated memory.
size: Size in bytes to allocate.
location: Device location hint (e.g., "cuda:0", "cpu:0", or "*" for auto-detect).
options: Advanced options including location, permission, and transport type.
Return value: Status::OK() on success; otherwise a non-OK status.

TransferEngine::freeLocalMemory

Status freeLocalMemory(void* addr);

Frees memory previously allocated with allocateLocalMemory.

addr: Pointer to the memory to free.
Return value: Status::OK() on success; otherwise a non-OK status.

TransferEngine::registerLocalMemory

Status registerLocalMemory(void* addr, size_t size,
                           Permission permission = kGlobalReadWrite);

// Batch registration
Status registerLocalMemory(std::vector<void*> addr_list,
                           std::vector<size_t> size_list,
                           Permission permission = kGlobalReadWrite);

// Advanced version with MemoryOptions
Status registerLocalMemory(void* addr, size_t size, MemoryOptions& options);

Registers externally allocated memory for use in transfers.

addr: Starting address of the memory region.
size: Size in bytes.
permission: Access permission (kLocalReadWrite, kGlobalReadOnly, kGlobalReadWrite).
addr_list / size_list: For batch registration of multiple buffers.
options: Advanced options for fine-grained control.
Return value: Status::OK() on success; otherwise a non-OK status.

TransferEngine::unregisterLocalMemory

Status unregisterLocalMemory(void* addr, size_t size = 0);

// Batch unregistration
Status unregisterLocalMemory(std::vector<void*> addr_list,
                             std::vector<size_t> size_list = {});

Unregisters previously registered memory.

addr: Starting address of the memory region.
size: Size in bytes (optional, can be 0 if the engine tracks it).
Return value: Status::OK() on success; otherwise a non-OK status.

Segment Management

TransferEngine::openSegment

Status openSegment(SegmentID& handle, const std::string& segment_name);

Opens a segment by name and returns a handle for use in transfers.

handle: Output parameter for the segment ID.
segment_name: Name of the segment to open (typically the remote node name).
Return value: Status::OK() on success; otherwise a non-OK status.

TransferEngine::closeSegment

Status closeSegment(SegmentID handle);

Closes a previously opened segment.

handle: The segment ID to close.
Return value: Status::OK() on success; otherwise a non-OK status.

TransferEngine::getSegmentInfo

Status getSegmentInfo(SegmentID handle, SegmentInfo& info);

Retrieves information about a segment.

handle: The segment ID.
info: Output parameter containing segment details (type, buffers).
Return value: Status::OK() on success; otherwise a non-OK status.

Advanced APIs

These APIs are optional; use them for segment export/import, notifications, or engine introspection.

Segment Export and Import

Reserved for future use. These APIs are declared but currently return Status::NotImplemented.

TransferEngine::exportLocalSegment

Status exportLocalSegment(std::string& shared_handle);

Exports the local segment as a shareable handle string.

shared_handle: Output string that can be passed to remote nodes.
Return value: Currently returns Status::NotImplemented.
Typical use: Share segment information through an external channel (e.g., gRPC, Redis).

TransferEngine::importRemoteSegment

Status importRemoteSegment(SegmentID& handle,
                           const std::string& shared_handle);

Imports a remote segment from a shared handle string.

handle: Output segment ID.
shared_handle: The handle string obtained from exportLocalSegment on the remote side.
Return value: Currently returns Status::NotImplemented.

Notifications

TENT supports lightweight notifications to coordinate data movement between nodes.

TransferEngine::submitTransfer (with notification)

Status submitTransfer(BatchID batch_id,
                      const std::vector<Request>& request_list,
                      const Notification& notifi);

Submits transfers and sends a notification upon completion.

notifi: A {name, msg} payload delivered to the receiver when the transfer completes.
Typical use: Signal that transferred data is ready for consumption.

TransferEngine::sendNotification

Status sendNotification(SegmentID target_id, const Notification& notifi);

Sends a notification to a specific segment without data transfer.

target_id: The segment to notify.
notifi: The notification payload.
Return value: Status::OK() on success; otherwise a non-OK status.

TransferEngine::receiveNotification

Status receiveNotification(std::vector<Notification>& notifi_list);

Receives pending notifications from peers.

notifi_list: Output vector of received notifications.
Return value: Status::OK() on success; otherwise a non-OK status.
Typical use: Polling loop to trigger follow-up actions on received data.

Engine Introspection

TransferEngine::available

bool available() const;

Returns true if the engine was initialized successfully and is ready for use.

TransferEngine::getSegmentName

const std::string getSegmentName() const;

Returns the local segment name (node identifier).

TransferEngine::getRpcServerAddress

const std::string getRpcServerAddress() const;

Returns the RPC server address for this engine instance.

TransferEngine::getRpcServerPort

uint16_t getRpcServerPort() const;

Returns the RPC server port for this engine instance.

Type Reference

Permission

enum Permission {
    kLocalReadWrite,   // Only local access
    kGlobalReadOnly,   // Remote read access
    kGlobalReadWrite,  // Remote read/write access
};

Location

using Location = std::string;
const static std::string kWildcardLocation = "*";

Location strings identify device affinity: "cpu:0", "cuda:0", "cuda:1", etc. Use "*" for automatic detection.

TransportType

enum TransportType {
    RDMA = 0,
    MNNVL,
    SHM,
    NVLINK,
    GDS,
    IOURING,
    TCP,
    AscendDirect,
    UNSPEC
};

Transport types used internally by TENT. Applications typically do not need to specify these directly.

MemoryOptions

struct MemoryOptions {
    Location location = kWildcardLocation;
    Permission perm = kGlobalReadWrite;
    TransportType type = UNSPEC;
    std::string shm_path = "";
    size_t shm_offset = 0;
    bool internal = false;
};

Advanced options for memory allocation and registration.

SegmentInfo

struct SegmentInfo {
    enum Type { Memory, File };
    struct Buffer {
        uint64_t base, length;
        Location location;
    };
    Type type;
    std::vector<Buffer> buffers;
};

Information about a segment, including its type and registered buffers.

Notification

struct Notification {
    std::string name;
    std::string msg;
};

Lightweight notification payload for coordination between nodes.

Status

The Status class is used for error handling throughout the API. Key methods:

bool ok() const;           // Returns true if operation succeeded
std::string ToString() const;  // Human-readable error description

// Common status factory methods
static Status OK();
static Status InvalidArgument(std::string_view msg);
static Status InternalError(std::string_view msg);
// ... and more

21 KiB Raw Blame History