AI Inference Performance Benchmarking for Distributed Graphics Processor Systems

The release of MLPerf Inference v6.0 benchmarks by MLCommons confirms the performance characteristics of high-end workstation and data center configurations utilizing Intel hardware. These results establish a baseline for deploying accessible artificial intelligence workloads across the digital supply chain, ranging from edge devices to multi-GPU enterprise nodes.

Hardware Architecture and Memory Capacity
The technical architecture of the benchmarked systems centers on the Intel Arc Pro B70 and B65 graphics processing units. A four-GPU configuration utilizing the Arc Pro B70 delivers 128GB of video random access memory (VRAM), a threshold required to execute 120-billion parameter models with high concurrency. Data indicates the Arc Pro B70 achieves up to 1.8 times the inference performance of the previous generation Arc Pro B60.

Beyond raw throughput, the hardware supports enterprise-grade reliability and manageability features. These include Error Correction Code (ECC) memory, Single Root I/O Virtualization (SRIOV) for virtualized environments, and telemetry for system monitoring. The hardware design utilizes PCIe peer-to-peer (P2P) data transfers to facilitate communication between multiple GPUs, reducing latency during complex model execution.

Software Optimization and Scaling
Performance gains in the latest benchmark cycle are attributed to an open, containerized software stack designed for Linux environments. This approach allows for the scaling of inference tasks from a single node to multi-GPU deployments. Software refinements have resulted in a 1.18x performance increase on existing Arc Pro B60 hardware compared to the MLPerf v5.1 results, demonstrating the impact of algorithmic and driver-level tuning within the automotive data ecosystem and related industrial applications.

In multi-GPU environments, the Arc Pro B70 demonstrates expanded capacity for Key-Value (KV) caching. Technical measurements show the hardware manages up to 1.6 times more KV cache capacity than comparable competitor solutions when processing larger models and extended context windows. This capacity is critical for maintaining performance in long-form generative tasks and complex data retrieval.

The Role of Centralized Processing in AI Infrastructure
While GPUs handle parallelized acceleration, the Intel Xeon 6 central processing unit (CPU) remains the orchestrator for memory management, task distribution, and cluster efficiency. According to MLPerf 6.0 submission data, over 50% of all submitted systems utilize Xeon as the host CPU.

The Xeon 6 processors with P-cores provide a 1.9x generational performance improvement over previous iterations. Built-in acceleration technologies, specifically Advanced Matrix Extensions (AMX) and AVX-512, enable the execution of classical machine learning and large language model (LLM) fine-tuning directly on the CPU. This capability allows organizations to run specific AI workloads without requiring dedicated accelerator hardware, potentially reducing the total cost of ownership for server infrastructure.

Anil Nanduri, Intel vice president of AI Products and GTM, states that the combination of Xeon 6 and Arc Pro B-Series GPUs offers a validated path for developers to address both LLM and traditional machine learning workloads using a unified hardware and software framework.

Edited by Evgeny Churilov, Induportals Media - Adapted by AI.

www.intel.com