Surprisingly Useful AI Article Enhancements
Red Hat has completed its acquisition of Neural Magic, a company known for developing software and algorithms that optimise generative Ai workloads. This move is expected to enhance Red Hat’s Ai offerings by integrating performance engineering and model optimisation expertise, making Ai more accessible and efficient for organisations operating across hybrid cloud environments.
Aiming for Accessible Generative Ai
Matt Hicks, President and CEO of Red Hat, highlighted the motivation behind the acquisition. “By adding Neural Magic’s expertise in generative Ai performance engineering and optimisation to Red Hat Ai, we’re furthering our commitment to an Ai that meets customers’ unique needs, from where workloads run to how they are tuned and trained.”
Organisations often find the large language models (LLMs) underpinning generative Ai to be resource-intensive and costly. Red Hat is addressing this challenge by focusing on smaller, optimised, open-source-licensed models that are compatible with diverse compute architectures and deployment environments.
Neural Magic’s contributions to Ai model optimisation align with this approach. The company is a key contributor to vLLM, an open-source project developed by UC Berkeley for open model serving, which provides more flexibility in building and deploying Ai workloads.
Enhancing Hybrid Cloud Generative Ai
Red Hat plans to incorporate Neural Magic’s technology into its hybrid cloud-ready generative Ai platforms. This integration aims to address common challenges in enterprise Ai through several strategies:
- Open-source models ranging from 1 billion to hundreds of billions of parameters, deployable across hybrid cloud environments, including corporate data centres, public clouds, and edge locations.
- Fine-tuning capabilities that allow organisations to customise LLMs using their private data while maintaining security.
- Expertise in inference performance engineering to improve operational and infrastructure efficiencies.
- A partner ecosystem supporting customer choice in tools, server hardware, and chip architectures.
Hicks emphasised the importance of choice in generative Ai. “The right environment, accelerated compute, and inference server are critical for successful generative Ai strategies. Neural Magic’s technology strengthens our ability to provide these choices.”
Neural Magic’s Role in Red Hat Ai
The acquisition will integrate Neural Magic’s expertise into Red Hat’s existing Ai platforms, which include:
- Red Hat Enterprise Linux Ai (RHEL Ai): A platform designed for developing and running enterprise applications using open-source LLMs.
- Red Hat OpenShift Ai: A platform enabling rapid development, training, and monitoring of machine learning models across distributed Kubernetes environments.
- InstructLab: A collaborative open-source project by Red Hat and IBM, allowing developers to improve LLMs using fine-tuning technology.
Neural Magic’s contributions, such as vLLM and pre-optimised models, will become integral to these platforms, bolstering Red Hat’s offerings in generative Ai.
Industry Perspectives
Brian Stevens, CEO of Neural Magic, highlighted the synergy between the two companies. “Neural Magic’s research has significantly reduced the infrastructure needed to deploy advanced LLMs. Red Hat shares our vision that the future of Ai is open, and together we aim to help enterprises capture the value of generative Ai without unnecessary friction.”
Dave McCarthy, Research Vice President at IDC, viewed the acquisition as a strategic enhancement to Red Hat’s Ai capabilities. “This move aligns with Red Hat’s commitment to open-source innovation and positions the company to deliver cost-effective, scalable Ai solutions that reduce reliance on specialised hardware.”
A Step Forward for Generative Ai
For more information about Red Hat and Neural Magic, visit Red Hat’s website.
Last Updated on January 15, 2025 by Nick Ross
