Powering Next-Gen AI and HPC: k0rdent Validated with AMD Instinct MI300X GPUs

The global demand for high-performance computing (HPC) and artificial intelligence (AI) is skyrocketing, driving the need for robust, scalable, and efficient infrastructure. At the heart of this revolution are high-performance accelerators, and the AMD Instinct GPUs are empowering many enterprises to adopt and deploy AI-powered solutions. But raw hardware power is only part of the equation; seamless orchestration and management are equally critical. This is where open-source k0rdent, originated by Mirantis, comes into play, by efficiently operationalizing the hardware and helping enterprises accelerate time to value for strategic GPU investments.

Today, we're announcing the successful validation of k0rdent with the cutting-edge AMD Instinct MI300X GPUs, helping ensure robust and reliable performance for demanding AI and HPC workloads.

The Power Duo: k0rdent and AMD Instinct GPUs

k0rdent: As a robust Kubernetes-native AI infrastructure platform, k0rdent stands out for its composable architecture, OSS standards, and a large ecosystem of compatible software. Leveraging open-source k0s, a compact, self-contained Kubernetes distribution with zero dependencies, k0rdent is designed for rapid deployment across hybrid clouds, making it an excellent choice for a variety of environments, from hyperscalers to edge deployments to high-capacity data centers. Its focus on a streamlined operational experience aligns perfectly with the agile needs of AI and HPC development.

AMD Instinct GPUs: These accelerators are engineered from the ground up for the most intensive AI training, inference, and HPC simulations. For example, the MI300X GPU is built on AMD's CDNA™ 3 architecture and boasts:

• Exceptional Compute Performance: Designed for leading-edge floating-point and integer operations crucial for complex AI models.

• Massive Memory Bandwidth and Capacity: With up to 192GB of HBM3 memory (MI300X) and high peak theoretical memory bandwidth, they can handle the largest language models and scientific datasets.

• Advanced Features: Support for specialized data formats (like FP8 and TF32), GPU partitioning (via Device Config Manager), and topology-aware scheduling, optimizing resource utilization.

Validation of k0rdent on MI300X GPUs: Assuring Seamless Compatibility

Bringing these two powerful components together requires thorough validation. The goal is to ensure that k0rdent-managed clusters can effectively discover, schedule, monitor, and manage the MI300X GPUs, allowing applications to fully leverage their capabilities. Making k0rdent-managed clusters ready for AI inference and training workloads out of the box. 

How the AMD GPU Operator Enables Validation:

The AMD GPU Operator acts as the bridge between k0rdent-managed clusters and the underlying AMD Instinct hardware. Its critical roles in validation include:

1. Automated Driver & ROCm Stack Management: The operator ensures that the correct ROCm™ (Radeon Open Compute platform) drivers and libraries are enabled and maintained on your k0rdent nodes. This is fundamental for the GPUs to function correctly.

2. Device Plugin Integration: It deploys the necessary Kubernetes device plugin, which registers the Instinct MI300X GPUs as allocatable resources (with node label amd.com/gpu) within the cluster. This allows Kubernetes to see and assign GPU resources to your pods.

3. Node Labeling: The operator automatically labels nodes with detailed GPU information (e.g., amd.com/gpu.product-name: AMD_INSTINCT_MI300X), enabling advanced scheduling based on specific MI300 characteristics.

4. GPU Partitioning (Device Config Manager): A key feature of the MI300X GPUs is its ability to be partitioned into smaller, isolated compute and memory units. The AMD GPU Operator, through its Device Config Manager, enables the configuration and management of these partitions, allowing multiple AI/HPC workloads to share a single GPU efficiently and securely. Validating this ensures fine-grained resource control.

5. Metrics and Monitoring: The operator deploys metrics exporters that expose GPU utilization, memory usage, temperature, and other vital statistics to Prometheus-compatible monitoring systems. Validation involves ensuring these metrics are accurate and provide actionable insights into Instinct MI300X GPU performance within k0rdent.

6. Workload Scheduling and Execution: The ultimate validation involves deploying real-world AI and HPC workloads (e.g., large language models with vLLM, scientific simulations) on k0rdent nodes with Instinct MI300XGPUs and confirming they run as expected, leveraging the GPUs effectively and demonstrating expected performance.

Key Validation Steps Completed for the Instinct MI300X GPUs:

• Basic System Health Checks: Verifying OS configuration, BIOS settings, and host memory.

• GPU Detection and ROCm Verification: Using tools like rocminfo and amd-smi to confirm that the MI300X GPUs are recognized and the ROCm stack is operational.

• Kubernetes Resource Allocation: Confirming that kubectl get nodes -o custom-columns=NAME:.metadata.name,"Total GPUs:.status.capacity.amd\\.com/gpu" shows the correct number of MI300X GPUs.

• Pod Scheduling: Deploying sample GPU-enabled pods and ensuring they are scheduled onto the correct nodes and can access the MI300X GPUs.

• Performance Benchmarking: Running standard benchmarks (e.g., ROCm Validation Suite (RVS), rocm-bandwidth-test, or LLM inference benchmarks using optimized Docker images like the ROCm vLLM image) to ensure the MI300X GPUs deliver their expected performance within the k0rdent environment.

• Scalability Testing: For multi-node setups, validating inter-GPU communication and cluster-level networking performance.

Results:

Apply yaml: “gpupod.yaml”:

apiVersion: v1
kind: Pod
metadata:
 name: amd-smi
spec:
 containers:
 - image: docker.io/rocm/pytorch:latest # 20GB image !
   name: amd-smi
   command: ["/bin/bash"]
   args: ["-c","amd-smi version && amd-smi monitor -ptum"]
   resources:
      limits:
        amd.com/gpu: 1
      requests:
        amd.com/gpu: 1
 restartPolicy: Never

Check results:

kubectl logs amd-smi
# AMDSMI Tool: 25.4.2+aca1101 | AMDSMI Library version: 25.4.0 | ROCm version: 6.4.1 | amdgpu version: 6.10.5 | amd_hsmp version: N/A
# GPU  POWER   GPU_T   MEM_T   GFX_CLK   GFX%   MEM%  MEM_CLOCK
#   0  139 W   48 °C   44 °C   130 MHz    0 %    0 %    900 MHz

The Benefits of a Validated k0rdent + Instinct MI300X GPU Stack

A successfully validated k0rdent and AMD Instinct MI300X GPU environment unlocks significant advantages:

• Optimized Resource Utilization: Efficiently allocate and partition powerful MI300X GPUs, maximizing ROI.

• Accelerated AI/HPC Workloads: Leverage the raw compute power and high bandwidth memory of MI300X GPUs for faster training, inference, and scientific discovery.

• Simplified Operations: The AMD GPU Operator, combined with k0rdent's ease of use, drastically reduces the complexity of managing GPU infrastructure.

• Scalability for Growth: Build scalable AI and HPC clusters ready for future expansion.

• Future-Proofing: Position your infrastructure to take advantage of the ongoing innovations from AMD in GPU technology and ROCm software.

The validation of k0rdent with AMD Instinct MI300X GPUs isn't just a technical exercise; it's about building the reliable, high-performance foundation necessary for the next wave of AI and scientific breakthroughs. As these integrations mature, expect to see even more impressive capabilities emerge from this powerful combination. This validation enables k0rdent’s stack to extend support for AMD Instinct products like the MI325X and MI350 Series GPUs. With continuous integration and validation efforts, expect to see even more impressive capabilities emerge from this powerful combination

To learn how to build and operate a next-generation AI Factory using k0rdent and AMD Instinct MI300X GPUs, please view the Mirantis AI Factory Reference Architecture.