* Add early mem backend detection method in NVLINK_allocator
Add early detection method for sglang NVLINK_allocator to avoid CuMemCreate
Use enumerate type to indicate mem backend type
format check use pre-commit
* Change enumerate type in allocator.py for different mem backend
* Isolate nvlink intraNode transport from nvlink_transport and modify corresponding transfer_engine_bench
* IntraNode transport isolation to be compatible with SGlang
* Modify the code style using pre-commit check
* isolate intraNode nvlink from previous nvlink_transport
* [TE] Revert to previous nvlink_transport
* Delete extra log and refine the code format
* Discard revert due to new related PR has been submitted
* Change the Code format to align with main branch
* Change to clang-format
* Modify transfer_engine_bench to be compatible with protocol --nvlink_intra
* Delete useless function in intranode_nvlink.cpp
* Used for rerun CI
* Modify the intraNode isolation to be compatible with transfer_bench and transfer_engine_impl
* isolate intraNode header file from mnnvl.h
* Delete specific instaince type for intraNode
---------
Co-authored-by: 百麒 <yaozhong.lyz@alibaba-inc.com>
* [TE]: Add HIP transport for AMD GPUs support
Forked from nvlink_transport and adapted for HIP/AMD GPUs.
* [TE/HIP] Addressed review comments
* [TE] Move NVLINK and HIP common functions to common files
* [TE] Fix incorrect length assignment in relocateSharedMemoryAddress
Use entry.length instead of length parameter when storing OpenedShmEntry.
The length parameter represents the requested transfer length, while
entry.length represents the actual buffer's full length, which is the
correct value to store and is consistent with openShareableHandle usage.
* [TE/Examples] Initialize allocated memory
GPU Driver may optimize transfers for non-initialized memory,
while on real applications the memory is always initialized.
So, need to initialize the memory in the benchmark too.
Added new command line arguments --init_mem to transfer_engine_bench.
Default value is true.
* [TransferEngineValidator] Check data at least once
- transfer_engine_validator checks data on random iterations.
It may happens that it does not check data at all.
So, added checking at least on the first iteration
- Print that data validation passed
* [TE/Examples] Directly call freePinnedLocalMemory for HIP
hipMemRetainAllocationHandle fails if memory was allocated using hipMalloc,
which is possible inside NvlinkTransport::allocatePinnedLocalMemory.
So, call just freePinnedLocalMemory, which will handle how to free.
* [TE/Examples] Directly call freePinnedLocalMemory for CUDA and MUSA too
From gemini:
The logic for CUDA/MUSA has a resource leak. cuMemRetainAllocationHandle increments the reference count, but there's no corresponding cuMemRelease call for the handle obtained here. This will leak resources.
The implementation for USE_HIP is better as it delegates the deallocation logic to freePinnedLocalMemory, which correctly handles different memory types. This approach avoids redundant checks and resource leaks.
* [TE/Examples] Remove unnecessary copyrights
* Improve AMD HIP support with hipify-perl
This commit improves AMD GPU support by migrating to hipify-perl
for automatic CUDA-to-HIP code conversion at build time, and extends HIP
compatibility to the NVLink transport layer.
* [TE] Add HIP support to nvlink-allocator with hipcc compilation
* Add USE_HIP to the documentation
* [TE/NVLINK] Check CU_DEVICE_ATTRIBUTE_HANDLE_TYPE_FABRIC_SUPPORTED only if USE_CUDA
* [TE/EXAMPLE] Fix compiler error if USE_MNNVL and USE_HIP
* Address review comments
* Support AMDGPU (refactor CUDA alike)
Signed-off-by: Xiaodong Ye <yeahdongcn@gmail.com>
* Address review comments
Signed-off-by: Xiaodong Ye <yeahdongcn@gmail.com>
* Fix build
Signed-off-by: Xiaodong Ye <xiaodong.ye@mthreads.com>
* Address review comments
Signed-off-by: Xiaodong Ye <yeahdongcn@gmail.com>
* Minor update
Signed-off-by: Xiaodong Ye <yeahdongcn@gmail.com>
* Minor update
Signed-off-by: Xiaodong Ye <yeahdongcn@gmail.com>
---------
Signed-off-by: Xiaodong Ye <yeahdongcn@gmail.com>
Signed-off-by: Xiaodong Ye <xiaodong.ye@mthreads.com>
Lewis