electronics-journal.com
24
'24
Written on Modified on
Intel News
INTEL GAUDI, XEON AND AI PC ACCELERATE META LLAMA 3 GENAI WORKLOADS
Intel’s AI products offer developers the latest optimizations to run Meta Llama 3, its next-generation large language model.
Recently, Meta launched Meta Llama 3, its next-generation large language model (LLM). Effective on launch day, Intel has validated its AI product portfolio for the first Llama 3 8B and 70B models across Intel® Gaudi® accelerators, Intel® Xeon® processors, Intel® Core™ Ultra processors and Intel® Arc™ graphics.
Llama 3 Running on Intel
Intel’s initial testing and performance results for Llama 3 8B and 70B models use open-source software, including PyTorch, DeepSpeed, Intel Optimum Habana library and Intel Extension for PyTorch to provide the latest software optimizations. For more performance details, visit the Intel Developer Blog.
- Intel® Gaudi® 2 accelerators have optimized performance on Llama 2 models – 7B, 13B and 70B parameters – and now have initial performance measurements for the new Llama 3 model. With the maturity of the Intel Gaudi software, Intel easily ran the new Llama 3 model and generated results for inference and fine-tuning. Llama 3 is also supported on the recently announced Intel® Gaudi® 3 accelerator.
- Intel Xeon processors address demanding end-to-end AI workloads, and Intel invests in optimizing LLM results to reduce latency. Intel® Xeon® 6 processors with Performance-cores (code-named Granite Rapids) show a 2x improvement on Llama 3 8B inference latency compared with 4th Gen Intel® Xeon® processors and the ability to run larger language models, like Llama 3 70B, under 100ms per generated token.
- Intel Core Ultra and Intel Arc Graphics deliver impressive performance for Llama 3. In an initial round of testing, Intel Core Ultra processors already generate faster than typical human reading speeds. Further, the Intel® Arc™ A770 GPU has Xe Matrix eXtensions (XMX) AI acceleration and 16GB of dedicated memory to provide exceptional performance for LLM workloads.
In the coming months, Meta expects to introduce new capabilities, additional model sizes and enhanced performance. Intel will continue to optimize performance for its AI products to support this new LLM.
