From Celestial AI to Taalas, these startups are seeking to challenge Nvidia’s AI computing dominance or find other areas ripe for disruption in the semiconductor industry.

Nearly two years ago, the semiconductor industry received a jolt of newfound interest when the world woke up to the game-changing capabilities of generative AI and the powerful chips that make it possible. While the biggest beneficiary of this interest has been Nvidia, a variety of startups have sought to challenge the AI chip giant or find other areas ripe for disruption.

[Related: Analysis: As Nvidia Takes AI Victory Lap, AMD Doubles The Trouble For Intel]

While the risk remains high for semiconductor startups—which typically have much higher costs than early-stage software firms—they could benefit from expectations that the industry could grow as much as 20 percent this year, in part because of strong demand for AI chips.

This leaves rooms for AI chip startups like Cerebras Systems, Hailo and Kneron to capture some of that projected growth in spending, which was forecasted by research firm IDC in December. Other semiconductor startups looking to disrupt the way chips are designed for AI computing include Celestial AI, Eliyan, Rivos and Tenstorrent.

What follows are CRN’s 10 hottest semiconductor startups of 2024 so far, which, in addition to the aforementioned startups, also include MetisX, SiMa.ai and Taalas.

Celestial AI

Top Executive: David Lazovsky, Founder and CEO

Celestial AI says it’s paving the way for advancements in AI computing by overcoming latency and bandwidth bottlenecks with its Photonic Fabric optical interconnect technology.

The Santa Clara, Calif.-based startup in March announced it had raised a “highly oversubscribed” $175 million Series C funding round that was led by the U.S. Innovative Technology Fund and backed by several other investors, including the venture arms of AMD, Samsung as well as Volkswagen Group’s holding company, Porsche SE.

That same month, the silicon photonics startup said hyperscaler firms—the world’s largest consumers of data center infrastructure—and semiconductor companies are “now designing in the Photonic Fabric optical chiplets as an initial phase of technology adoption.” This integration of optical chiplets within multi-chip packages, increasingly becoming the norm in high-performance processors, could enable up to 25 times better off-package bandwidth compared to “other state-of-the-art technologies,” according to Celestial AI.

Cerebras Systems

Top Executive: Andrew Feldman, Co-Founder And CEO

Cerebras Systems is challenging Nvidia’s AI computing dominance with its Wafer Scale Engine chip, which it said enables superior performance-per-watt and “unprecedented scalability.

In March, the Sunnyvale, Calif.-based startup revealed its third-generation chip, the Wafer Scale Engine 3, which it said offers “twice the performance” of the predecessor “at the same power draw and for the same price.” Consisting of 4 trillion transistors and using TSMC’s 5-nanometer process, the WSE-3 packs 900,000 AI cores and 44 GB of on-chip SRAM, making it capable of 125 petaflops of 16-bit floating point performance.

The company’s other milestones this year include a multi-year strategic collaboration with health care giant Mayo Clinic to develop multimodal large language models to improve patient outcomes and diagnoses, a multi-year partnership with AI startup Aleph Alpha to build secure sovereign AI solutions and the groundbreaking of the Condor Galaxy 3 supercomputer for Abu Dhabi-based technology holding group G42.

Eliyan

Top Executive: Ramin Farjadrad, Co-Founder and CEO

Eliyan wants to help chip designers build more powerful chiplet-based processors by breaking down barriers in die-to-die bandwidth with its NuLink PHY interconnect technology.

The Santa Clara, Calif.-based startup in March announced that it closed a $60 million funding round that was co-led by Samsung Catalyst and Tiger Global Management and backed by other investors, including the venture arm of Intel and SK Hynix.

Earlier in the year, Eliyan said it taped out the “highest-performing” solution in the physical layer for connecting multiple dies in a single chip architecture, and it used TSMC’s 3-nanomter manufacturing process, enabling up to 64 Gbps per link.

Hailo

Top Executive: Orr Danon, Co-Founder and CEO

Hailo is taking on Nvidia by accelerating generative AI workloads at the edge with chips that lead the way when it comes to optimizing performance for cost and power.

The Tel Aviv, Israel-based startup announced in April that it raised $120 million from investors as an extension of its Series C funding round on top of launching its new Hailo-10 acceleration, which enables “maximum GenAI performance with minimum required power” for devices such as PCs and automotive infotainment systems.

For example, the company said Hailo-10 can run a 7-billion-parameter Llama 2 model at up to 10 tokens per second while only using 5 watts of power. The chip can also create an image under 5 seconds in the same power envelope for the Stable Diffusion 2.1 model.

Kneron

Top Executive: Albert Liu, Founder and CEO

Kneron seeks to weaken Nvidia’s influence with AI chips designed to reduce server costs for enterprises and lower the price and energy consumption of PCs when it comes to generative AI.

The San Diego, Calif.-based startup announced in June the launch of KNEO 330, its second-generation “edge GPT” server, which it said can reduce AI costs for small enterprises by 30-40 percent. It’s capable of 48 tera operations per second (TOPS) and up to eight concurrent connections, and it supports large language models and retrieval-augmented generation accuracy that is on par with cloud solutions.

Kneron—which has raised $190 million in funding from investors, including Qualcomm and Foxconn—also released its third-generation neural processing unit (NPU), the KL830, which is designed to enable lower-cost AI PCs as well as AI-enabled IoT applications.

MetisX

Top Executive: Jim Kim, Co-Founder and CEO

MetisX aims to make data centers “smarter, faster and more cost-effective” by developing intelligent memory systems based on Compute Express Link (CXL) technology.

The Seoul, South Korea-based startup announced in May that it has raised a $44 million Series A funding round from a variety of investors and said it planned to establish a U.S. presence and introduce a chip next year aimed a hyperscaler customers.

The company said it has already completed prototypes for use cases in large-scale data processing such as vector databases, big data analysis and DNA analysis. It reported finding such prototypes doubling the performance of conventional server CPUs.

Rivos

Top Executive: Puneet Kumar, Co-Founder and CEO

Rivos wants to shake up the data center market with chips that combine high-performance RISC-V CPUs and a data parallel accelerator for data analytics and generative AI workloads.

Founded by former Google, Apple and Intel engineers, the Santa Clara, Calif.-based startup in April announced that it had raised more than $250 million in an oversubscribed Series A-3 funding round from several investors, including the venture arms of Intel and Dell Technologies.

The company emerged with the news of funding, which it said would be used to tape out its first silicon product and expand its team, after reaching a settling a lawsuit with Apple in February. Apple had accused the startup of stealing trade secrets by hiring away dozens of engineers from the tech giant, which Rivos, in turn, countersued for unfair competition.

SiMa.ai

Top Executive: Krishna Rangasayee, Founder and CEO

SiMa.ai is hoping to displace Nvidia for generative AI workloads at the edge with powerful and efficient chips that can handle a wide variety of modalities in one, “software-centric” platform.

The San Jose, Calif.-based startup announced in April that it had raised $70 million from investors, including the venture arm of Dell Technologies and Cadence Design Systems Executive Chairman Lip-Bu Tan.

The company said it will use the funding to continue meeting customer demand for its first-generation Machine Learning System-on-Chip (MLSoC), which specializes in computer vision, while speeding up development of its second-gen MLSoC, which supports multimodal generative AI workloads, including speech, audio, text and images.

Tenstorrent

Top Executive: Jim Keller, CEO

Tenstorrent seeks to blaze a new trail in chip design for AI computing with a business model that combines selling specialized processors with licensing chip technologies for others to use.

The Toronto, Ontario-based startup announced in February that it has entered a “multi-tiered partnership deal” with Japan’s Leading-edge Semiconductor Technology Center, which plans to take advantage of Tenstorrent’s RISC-V and chiplet technology for its 2-nanometer edge AI accelerator. The startup will also serve as a co-design partner for the chip.

The startup, which last year raised a $100 million funding round led by Hyundai Motor Group and Samsung Catalyst Fund, is raising least $300 million in a new round led by Samsung that gives Tenstorrent a $2 billion valuation, The Information reported in June. LG Electronics, another major South Korean firm, is also reportedly in talks for the round.

Taalas

Top Executive: Ljubisa Bajic, Founder and CEO

Taalas seeks to disrupt Nvidia’s general-purpose GPU strategy by designing accelerator chips that directly implement entire AI models, which it said can reduce costs by up to 1,000 times.

Led by Tenstorrent founder Llubisa Bajic, the Toronto, Ontario-based startup in March announced it had raised a $50 million funding round and revealed its plan to create an automated workflow for hardwiring all types of deep learning models into chips.

Thanks to this design approach, Taalas said it was able to design a chip that contains an entire large AI model “without requiring external memory.” As a result, the chip design is projected to provide more performance than a small GPU data center, according to the startup, which opens the path for it to reduce AI computation costs by more than 1,000 times.

The company said it plans to tape out its first large language model chip in the third quarter and make it available to customers in the first quarter of next year.

probably