From Celestial AI to Untether AI, these startups are seeking to challenge Nvidia’s AI computing dominance or deliver complementary chip technologies that could shake up the tech industry.

While Nvidia commands tens of billions of dollars in AI chip spending every quarter, there are a good deal of companies and investors who believe there is room for other winners in the AI infrastructure market, whether it’s for chips at the edge or in data centers.

Nvidia may face larger threats in the forms of AMD and, to a lesser extent, Intel, as well as its largest cloud partners—Amazon Web Services, Microsoft Azure and Google Cloud—who are building their own AI chips among other homegrown silicon efforts.

[Related: Nvidia Reveals 4-GPU GB200 NVL4 Superchip, Releases H200 NVL Module]

But the AI computing giant is also up against a small army of entrepreneurs and investors who are propping up new chip solutions with novel design techniques and big claims about how they could enable faster and more efficient AI computation.

The semiconductor startups behind these efforts include those focusing on the edge market such as Hailo, SiMa.ai and Untether AI as well as those building chips for AI data centers like d-Matrix, Groq Tenstorrent.

It’s not all about competition, however. Some semiconductor startups, like Enfabrica and Lightmatter, are working on complementary chip technologies that could, for example, speed up the transmission of data between AI servers.

What follows are CRN’s 10 hottest semiconductor startups of 2024, which, in addition to the aforementioned startups, also include Celestial AI and Etched.

Celestial AI

Top Executive:  David Lazovsky, Founder and CEO

Celestial AI says it’s paving the way for advancements in AI computing by overcoming latency and bandwidth bottlenecks with its Photonic Fabric optical interconnect technology.

The Santa Clara, Calif.-based startup in March announced it had raised a “highly oversubscribed” $175 million Series C funding round that was led by the U.S. Innovative Technology Fund and backed by several other investors, including the venture arms of AMD, Samsung as well as Volkswagen Group’s holding company, Porsche SE.

That same month, the startup said hyperscaler firms—the world’s largest consumers of data center infrastructure—and semiconductor companies are “now designing in the Photonic Fabric optical chiplets as an initial phase of technology adoption.”

Then in October, Celestial AI announced it had acquired intellectual property from silicon photonics pioneer Rockley Photonics. The deal included issued and pending patents related to silicon photonics, which gave the startup “one of the strongest IP portfolios in the field of silicon photonics for optical compute interconnect, according to Celestial AI.

D-Matrix

Top Executive: Sid Sheth, Co-Founder and CEO

D-Matrix says it’s smashing the memory bandwidth barrier for generative AI inference workloads with its novel Digital In-Memory Compute (DIMC) architecture.

The Santa Clara, Calif.-based startup—which is backed by the venture arms of Microsoft, Samsung Electronics and Ericsson—announced in November the launch of its first product, the Corsair PCIe card, which uses the DIMC architecture to “accelerate AI inference workloads with industry-leading real-time performance, energy efficiency and cost savings as compared to GPUs and other alternatives.”

The PCIe card is sampling now with customers, and it will be broadly available in the second quarter of next year with support from OEMs and systems integrators. Among the vendors supporting Corsair are Supermicro, GigaIO and Liqid.

Enfabrica

Top Executive: Rochan Sankar, CEO

Enfabrica says it’s delivering the fastest network interface controller chip for GPU servers in the industry with silicon that was built from the ground up for the needs of AI data centers.

The Mountain View, Calif.-based startup announced in November that it had raised a $115 million Series C funding round to help commercialize its Accelerated Compute Fabric SuperNIC chip, which it said will enable a “groundbreaking” 3.2 TB/s of bandwidth when it becomes available in pilot systems in the first quarter of next year.

The ACF SuperNIC supports 800-, 400- and 100-Gigabit Ethernet interfaces as well as a high radix of 32 network ports and 160 PCIe lanes on a single chip, which enables server clusters of more than 500,000 GPUs using a two-tier network design for the “highest scale-out throughput and lowest end-to-end latency across all GPUs in the cluster,” it said.

Etched

Top Executive: Gavin Uberti, Co-Founder and CEO

Etched says it’s betting its whole business model on what it’s calling the world’s first specialized chip for transformer-based AI models.

The San Francisco, Calif.-based startup announced in June that it had raised $120 million in funding from a wide range of investors, including PayPal co-founder Peter Thiel, Fontinalis and Skybox Data Centers.

In a blog post, the company claimed that its Sohu chip will be “an order of magnitude faster and cheaper than even Nvidia’s next-generation Blackwell B200 GPUs” when it comes to transformer-based models like large language models, but it won’t be capable of running any model that isn’t based on the transformer architecture.

Groq

Top Executive: Jonathan Ross, Founder and CEO

Groq says its language processing unit enables blazing-fast AI inference performance through its cloud service and on-premises compute clusters.

The Mountain View, Calif.-based startup announced in August that it had raised a $640 million Series D funding round at a $2.8 billion valuation led by funds and accounts managed by BlackRock Private Equity Partners. Other investors included the venture arms of Cisco Systems and Samsung Electronics.

Back in May, Groq announced that it had entered a distribution agreement with U.S. solution provider powerhouse Carahsoft to target public sector customers. Then in September, the company announced it had signed a memorandum of understanding with the digital and technology subsidiary of Middle Eastern oil giant Aramco to “establish the world’s largest inferencing data center in the Kingdom of Saudi Arabia.”

Hailo

Top Executive:  Orr Danon, Co-Founder and CEO

Hailo is taking on Nvidia by accelerating generative AI workloads at the edge with chips that it says lead the way when it comes to optimizing performance for cost and power.

The Tel Aviv, Israel-based startup announced in April that it raised $120 million from investors as an extension of its Series C funding round on top of launching its new Hailo-10 accelerator chip, which enables “maximum GenAI performance with minimum required power” for devices such as PCs and automotive infotainment systems.

The company has also announced several partnerships, including a deal with Raspberry Pi to provide chips for the Raspberry Pi AI Kit, a deal with Adlink Technology to incorporate its Hailo-8 chip into an edge computing platform, and a deal with SolidRun to incorporate its Hailo-15H into a system-on-module solution for AI vision applications.

Lightmatter

Top Executive: Nick Harris, Co-Founder and CEO

Lightmatter says it’s reinventing AI infrastructure with 3-D stacked photonics chips that can dramatically increase AI cluster bandwidth and performance while lowering energy use.

The Mountain View, Calif.-based startup announced in October that it raised a $400 million Series D funding round at a $4.4 billion valuation. The round was led by new investors advised by T. Rowe Price Associates, with participation from the venture arm of Google and Fidelity Management and Research Company, among other investors.

The company said it plans to use the funding to prepare Passage photonics chip for “mass deployment in partner data centers.” It added that the chip is the “first photonic engine to deliver I/O in 3-D,” which is expected to “free up shoreline” for GPUs and other kinds of processors to “support more memory.”

SiMa.ai

Top Executive:  Krishna Rangasayee, Founder and CEO

SiMa.ai is hoping to displace Nvidia for generative AI at the edge with powerful and efficient chips that it says can handle a wide variety of modalities in one, “software-centric” platform.

The San Jose, Calif.-based startup announced in September that it was expected to begin sampling with customers by the end of the year its new MLSoC Modalix family of chips that support convolutional neural networks, transformers, large language models, large multimodal models and other kinds of generative AI models at the edge. It added that the chips deliver 10 times greater performance per watt than alternatives.

SiMa.ai announced the new edge AI chip product family after saying in April that it had raised a $70 million funding round from investors, including the venture arm of Dell Technologies and Cadence Design Systems Executive Chairman Lip-Bu Tan.

The company has announced several partnerships this year to commercialize its chips, including a deal with Lanner to integrate its chips into edge AI appliances; a distribution deal with Arrow Electronics for Europe, the Middle East and Africa; and a deal with Cvedia and Supermicro to deliver edge appliances with AI video analytics capabilities.

Tenstorrent

Top Executive:  Jim Keller, CEO

Tenstorrent is blazing a new trail in chip design for AI computing with a business model that combines selling specialized processors with licensing chip technologies for others to use and working with other firms to develop computing solutions.

The Toronto, Ontario-based startup announced in November that it has entered a strategic partnership with South Korean AI software company Moreh to challenge Nvidia in the AI data center market. The two companies are working on a solution that combines Tenstorrent’s neural processing units with Moreh’s software to support a wide range of AI applications, including large language model inferencing and training.

In February, the company announced a “multi-tiered partnership deal” with Japan’s Leading-edge Semiconductor Technology Center, which plans to take advantage of Tenstorrent’s RISC-V and chiplet technology for its 2-nanometer edge AI accelerator. The startup will also serve as a co-design partner for the chip.

Tenstorrent has also announced this year a partnership with SingularityNET to develop hardware and software architectures that are optimized for artificial general intelligence as well as the launch of developer kits and workstations based on its Wormhole processors.

Untether AI

Top Executive: Chris Walker, CEO

Untether AI says its new chip delivers the “world’s fastest, most energy efficient AI inference” performance for applications running at the edge and in data centers.

The Toronto, Canada-based startup, which is led by former Intel executive Chris Walker, announced in October the launch of its speedAI240 Slim AI inference accelerator cards, which it said “were recently recognized as achieving the world’s lowest latency and highest throughput on the [peer-reviewed] MLPerf inference benchmark.”

In August, the company announced a “multi-faceted partnership” with Indian AI cloud, model and service provider Ola-Krutrim that will include the “co-development of Untether AI’s next-generation data center solutions, which will be used for the inference deployment and fine-tuning of Krutrim’s generative AI models.”