Competition among ASIC service providers is stirring up the tech industry, especially in Google’s realm. The search giant stands on the cusp of potentially saving billions through innovative silicon solutions specifically designed for its artificial intelligence (AI) endeavors.
Unlocking Cost Savings
According to analysts at Bernstein, Google’s custom AI chips, known as Tensor Processing Units (TPUs), are at the forefront of this seismic shift. TPUs are estimated to offer around 4 times the performance per dollar compared to current-generation GPUs. While initial cost savings from chip acquisition alone are impressive, when factoring in total system costs encompassing software development, research and development expenses, and packaging, these savings could range from 1.5 to 4 times.
The Economics Behind TPUs
The economic advantage that TPUs hold over traditional GPUs primarily stems from the profit margins associated with rival graphics processing units. Notably, while there is a marginal difference in bill of materials between TPUs and GPUs covering memory and foundry expenses, NVIDIA’s substantial gross margins exceeding 70% contribute significantly to the remaining cost differential.
Analysts at Bernstein emphasize that as the complexity and cost of training large AI models continue to escalate faster than hardware efficiency improvements dictated by Moore’s Law – which traditionally sees a reduction in cost per operation annually – by only about 25%–40%, it becomes imperative for companies like Google to explore alternatives like ASICs such as TPUs.
Diverse Market Landscape
While some tech behemoths like Amazon and Google itself opt for developing ASICs internally, several other prominent players such as Meta, Microsoft, OpenAI, and xAI rely on external providers for their specialized silicon needs. This trend points towards an evolving landscape where competition among ASIC service providers is set to intensify further.
Bernstein foresees sustained growth in the overall ASIC market despite potential fluctuations in demand due to varying customer requirements and shifts within the industry dynamics overtime.
