Masterclass: Sourcing and DD on Our Investment in Groq

Early Access: The Technical Edge Most VCs Missed

For the technically inclined: Groq’s Language Processing Unit (LPU) represented a fundamental rethinking of AI acceleration. When we first invested in 2021, their architecture demonstrated 20x lower latency than comparable GPUs for inference workloads — a technical achievement that went largely unnoticed by the mainstream. Back in 2021, few knew what an “inference workload” even was. Early customers then included a quant hedge fund and an automotive company working on autonomous travel.

An AI inference workload refers to the computational tasks involved in running a trained machine learning model to generate predictions or decisions based on new input data. This is the stage after the model has been trained and deployed, where it is actively being used in real-world applications.

In hindsight, this is obvious — at the time, a lot less so.

NVIDIA’s CEO, Jensen Huang, has highlighted the immense scale of AI inference compared to training. In a discussion with Brad Gerstner of Altimeter, when Gerstner noted that inference accounts for about 40% of NVIDIA’s revenue and asked about future expectations, Huang responded, “It’s about to go up a billion times.” 

Key Aspects of AI Inference Workloads

Examples of AI Inference Workloads

The Groq Difference

Groq was founded by Jonathan Ross, who previously designed Google’s Tensor Processing Unit (TPU). Ross’ team eliminated the traditional bottlenecks in inference by using a deterministic single-core design rather than the probabilistic multi-core architecture that dominates the market. Read the whitepaper for the deep dive on the technology.

Translated into a performance comparison (mid 2024 metrics – details here), here’s the bottom line:

This isn’t just incremental improvement—it’s a fundamentally different approach that’s now proving crucial for real-time AI applications.

Being Ahead of the Curve

We got to know Groq in early 2019 and were tracking the company since their 2018 Series B round. We mobilized our alumni network to make a strong connection and were able to get allocation in the 2021 Series C.

For Strategic Asset Allocators: The Portfolio Perspective

AI infrastructure represents a crucial allocation within the broader technology ecosystem. Unlike software plays that can face instant competition, hardware innovation creates defensible moats through IP and manufacturing expertise.

Current market context:

Our $3M 2024 follow-on investment in Series D was a calculated portfolio move, balancing earlier stage investments with a proven technology that’s entering its scaling phase.

For Those Who Seek Exclusive Opportunities: The Inner Circle

What separates truly exceptional investors from the crowd? Access and timing.

Groq’s Series C in 2021 wasn’t broadly marketed. It was a carefully constructed syndicate of investors with deep technical understanding and strategic vision. The participants weren’t just writing checks; they were making a statement about the future of AI infrastructure.

Today, these early backers are regularly featured at invite-only events like:

This wasn’t luck — it was calculated vision paired with exclusive access. The same networks that provided our entry into Groq continue to surface opportunities in adjacent spaces before they reach more mainstream awareness.

As Groq goes to its next version LPU on 4nm silicon, their inference advantages will continue to increase. Groq announced this partnership with Samsung in 2023 for their next generation hardware.

Our investment thesis centered on Groq’s ability to deliver on both performance and adaptability — a combination that established players couldn’t match and most startups couldn’t execute on. Groq also has an energy advantage over competitors. At an architectural level, Groq LPUs are up to 10X more energy efficient than GPUs. Since energy consumption is one of the biggest cost factors in running AI models, better energy efficiency generally translates to lower costs of operation.

The Path Forward: From Technical Advantage to Market Dominance

Groq’s journey from technical innovation to market adoption follows a familiar pattern we’ve seen in other successful hardware investments:

Each of these transitions has expanded Groq’s valuation, with our investors benefiting from positioning at the Series C and D stages — the ideal balance of derisked technology with tremendous growth ahead.

Limited Access: Join The Next Wave Before It Crests

The window for Groq may have narrowed, but our investment thesis — identifying infrastructure plays that enable transformative technologies — continues to surface opportunities.

Three key insights drive our current outlook:

This communication is from Alumni Ventures, a for-profit venture capital company that is not affiliated with or endorsed by any school. It is not personalized advice, and AV only provides advice to its client funds. This communication is neither an offer to sell, nor a solicitation of an offer to purchase, any security. Such offers are made only pursuant to the formal offering documents for the fund(s) concerned, and describe significant risks and other material information that should be carefully considered before investing. For additional information, please see here. Example portfolio companies are provided for illustrative purposes only and are not necessarily indicative of any AV fund or the outcomes experienced by any investor. Example portfolio companies shown are not available to future investors, except potentially in the case of follow-on investments. Venture capital investing involves substantial risk, including risk of loss of all capital invested. This communication includes forward-looking statements, generally consisting of any statement pertaining to any issue other than historical fact, including without limitation predictions, financial projections, the anticipated results of the execution of any plan or strategy, the expectation or belief of the speaker, or other events or circumstances to exist in the future. Forward-looking statements are not representations of actual fact, depend on certain assumptions that may not be realized, and are not guaranteed to occur. Any forward-looking statements included in this communication speak only as of the date of the communication. AV and its affiliates disclaim any obligation to update, amend, or alter such forward-looking statements, whether due to subsequent events, new information, or otherwise.