Meta is on track to start manufacturing the latest versions of its custom AI chips in September, according to an internal memo cited by Reuters. The chips, developed under the Meta Training and Inference Accelerator (MTIA) program, are designed to lower the company's dependence on costly GPUs from Nvidia and AMD amid an ongoing global shortage of high-performance semiconductors.
The memo revealed that at least one of the new chips passed its testing phase in about six weeks, a notable achievement for a project of this scale. Meta is working closely with Broadcom on the chip design, while Taiwan Semiconductor Manufacturing Company (TSMC) will fabricate the chips. Additionally, the company is sourcing RAM from Samsung, storage from Sandisk, and fiber-optic equipment from Sumitomo Electric, according to the report.
MTIA Program Details and Modular Design
Meta first detailed four new chips under the MTIA program in March, some of which are currently being deployed or will be integrated this year or next. The company is taking a modular approach to chip design, using chiplets that can be combined in different configurations. This strategy allows Meta to adapt quickly as AI workloads evolve, ensuring that the hardware remains relevant by the time it reaches production.
“Each MTIA generation builds on the last, using modular chiplets, incorporating the latest AI workload insights and hardware technologies, and deploying on a shorter cadence,” the company stated in March. The chips are expected to be used for training models that power Meta's ranking and recommendation algorithms, as well as broader AI workloads and inference tasks within its applications, including Facebook, Instagram, and WhatsApp.
Cost Savings and GPU Dependency
By producing its own chips, Meta aims to reduce its spending on third-party GPUs. However, the company still plans to invest heavily in Nvidia and AMD products. In April, Meta announced that it expects capital expenditures between $125 billion and $145 billion this year, a substantial portion of which is allocated to AI infrastructure. The company has been signing data center and power deals across the globe, spending tens of billions to secure computing capacity to train and deploy its new Muse Spark series of AI models. According to the memo, Meta plans to deploy 7 gigawatts of computing power this year and double that amount next year.
Meta’s pivot to custom silicon is not new; the company has been producing its own AI chips since 2023. The MTIA chips are designed specifically for the types of AI workloads Meta handles at scale, including personalized recommendations and generative AI features. The chips are expected to lower costs per inference and improve energy efficiency, both critical as AI demand surges.
Industry-Wide Trend Toward Custom Chips
Meta is not alone in seeking alternatives to Nvidia. OpenAI recently unveiled an inference processor built with Broadcom, and Anthropic is reportedly considering developing its own chips with Samsung. Amazon and Google both design custom chips for AI training and inference, and a host of startups are emerging to meet the skyrocketing demand for specialized hardware.
Last year, Meta also struck a deal with ARM to secure compute for its recommendation systems. Additionally, it signed a multibillion-dollar agreement with AMD for Instinct GPUs and a multibillion-dollar deal with Amazon to use the cloud provider’s homegrown CPUs for AI-related tasks. These moves illustrate Meta's complex strategy of balancing custom silicon development with third-party purchases to ensure it has enough capacity to meet its ambitious AI goals.
The broader AI hardware landscape is rapidly evolving. While Nvidia remains the dominant player, companies like Meta are investing in alternatives to gain more control over their supply chains and reduce costs. The chip shortage that began during the pandemic has accelerated these efforts, as lead times for GPUs have stretched and prices have soared.
Technical and Strategic Implications for Meta
The new MTIA chips are expected to bring significant performance improvements for Meta's internal workloads. The modular design allows the company to swap out chiplets as new technologies emerge, such as higher-bandwidth memory or more efficient processing units. This flexibility is crucial given the fast pace of AI model development, where new architectures like transformers and MoE (mixture of experts) require different hardware optimizations.
Meta’s investment in custom silicon also has implications for its cloud and enterprise offerings. While the company primarily uses its chips for internal operations, there is potential that these chips could eventually be offered through Meta's cloud services or open-sourced, following the company's history of releasing AI software like PyTorch and LLaMA.
From a financial perspective, the shift to custom AI chips could yield significant long-term savings. According to analysts, a typical GPU-based inference server can cost Meta tens of thousands of dollars per unit, and the company operates thousands of such servers. By designing chips specifically for its needs, Meta can reduce unit costs and improve power efficiency, which translates into lower operational expenses.
However, the development is not without risks. Designing and manufacturing cutting-edge chips is a complex and expensive undertaking. TSMC's advanced processes, such as 3nm and 2nm nodes, require massive investments and have long lead times. Delays in production or design flaws could set Meta back by months. Additionally, Broadcom's role as a design partner may limit Meta's ability to iterate quickly on its own.
Broader Competitive Landscape
The race to develop custom AI chips is heating up across the tech industry. Amazon's Trainium and Inferentia chips have been powering its AWS services for years. Google’s TPU (Tensor Processing Unit) is widely used for both training and inference in Google Cloud. Microsoft has also been developing its own AI chips, though details remain sparse. Startups like Cerebras, Groq, and SambaNova are building specialized hardware designed to compete with Nvidia's dominance.
Meta's entry into this space underscores the importance of vertical integration for large tech companies. By controlling its own hardware, Meta can optimize its AI stack from the data center to the application, potentially gaining a competitive edge in developing next-generation AI models. The MTIA program is a key component of this strategy, and the upcoming production milestone represents a critical step forward.
In the meantime, Meta continues to spend heavily on Nvidia GPUs to meet immediate demand. The company recently placed orders for tens of thousands of H100 and B200 chips, according to industry sources. This dual approach ensures that Meta can keep its AI initiatives moving forward while gradually transitioning to custom solutions.
Overall, the start of production in September marks a significant achievement for Meta's hardware team. The first chips should begin appearing in Meta's data centers later this year, where they will be tested on real-world workloads. If successful, they could reshape Meta's AI infrastructure and provide a template for other large companies looking to reduce their reliance on GPUs.
Source:TechCrunch News