Jonathan Ross, founder of Groq (LPU inference chips), recounts the company’s near-death experiences, the $20B NVIDIA partnership that closed in three weeks, and the leadership and hiring lessons from a decade building a capital-intensive hardware startup.
The $20B NVIDIA Deal Closed In Three Weeks
The partnership originated from a call where Groq asked to buy ~100,000 GPUs to deploy alongside their LPUs; Jensen Huang saw the integration and proposed making it available to all NVIDIA customers instead.
From first discussion to money in the bank took roughly three weeks; the technical groundwork (integrating GPU and LPU workloads) had been in progress for three to four months prior.
Groq’s COO Sunny first suggested combining the chips; the engineering team then mapped decoder-layer operations — attention on GPU, weight application on LPU — and proved the combined system beat bottlenecks across the whole performance curve.
Why GPUs And LPUs Are Better Together
LLMs spend cycles on both compute-bound (attention) and memory-throughput-bound (weight application) matrix multiplies; no single architecture wins at both.
Splitting prefill vs. generation across chips misses the point: generation (the “thinking” step) is the hard part and benefits most from LPU’s memory bandwidth.
The combined system delivers “broadband instantly” on existing models: token generation drops from ~1 minute to ~10 seconds, compounding speed and quality gains.
When AI Talks To AI, Speed Wins
Human latency tolerance is seconds; agents produce and consume tokens orders of magnitude faster, so every millisecond of inference latency compounds exponentially in multi-agent workflows.
Agents will spawn research sub-agents, make micro-payments for tools/data, and coordinate in real time; current payment rails (KYC, human-in-the-loop) are a bottleneck.
Ross’s hobby project (getting phone numbers for Signal/WhatsApp agents via Twilio) illustrated the friction: an agent with a budget could have completed it autonomously.
Always Start With A Hobby Project
Ross builds personal tools on GCP/AWS (travel optimizers, daily briefs, math utilities) before bringing patterns to work; keeps risk off the production codebase.
His “daily brief” evolved from a long AI-researched email to an interactive headline feed where he asks follow-up questions — like a personalized Presidential Daily Brief.
Interactivity beats static podcasts/summaries: 20-questions-style dialogue lets him drill to what he actually cares about.
Ask The Right Questions, Not Answer Them
Information-age success rewarded answering questions (school, exams); AI-age success rewards asking the right questions because the model already knows the answers.
The shift mirrors IC → leader: good leaders don’t do the work, they frame the problem so the team (or AI) can solve it.
Ross’s challenge coin — “25 million tokens per second” with an upward graph — gave the whole company a single, brutally clear priority that left room for creative solutions.
There Are Infinite Ways To Be A Leader
Leadership = having followers; there are as many valid styles as there are investment strategies (venture, PE, debt, public markets, etc.).
Ross delegates heavily, hires autonomous people, and avoids command-and-control; he doesn’t even have a driver’s license because he’d rather think than drive.
Founders must pick the style that fits their personality; early-career hires should join cultures that teach the lessons they can actually use.
I Was One Of The World’s Worst Leaders
Early on, Ross gave autonomy to people who needed direction; work stalled, he’d intervene awkwardly, and the team didn’t trust his sudden directives.
The fix: distill the goal to a single, under-constrained objective (the challenge coin) so creative people can surprise him with solutions.
Kelly Johnson’s Skunk Works principle — “extreme performance often comes from one brutally clear priority” — validates this: innovation requires the freedom to surprise you in a good way.
At NVIDIA There Is No Politics
Jensen Huang runs the least political large organization Ross has seen: no 1-on-1s with conflicting messages, no side emails — everything is said in open forums with everyone copied.
Ross adopted this at Groq: stop 1-on-1s for directional communication; hold all-hands, copy everyone on emails, force transparency to kill side conversations.
Jensen’s customer obsession: “What does the customer need? Build that. Don’t play 3D chess with deal structures.” Trust compounds.
East Coast VCs Think, West Coast VCs Follow
Groq struggled to raise because early backers fell out of favor; West Coast VCs acted like lemmings (one lead → all follow; one pass → all pass).
East Coast crossover funds ran independent analyses and invested; West Coast missed the largest NVIDIA partnership ever (~3× prior record).
The Keynesian beauty contest explains the herding: VCs bet on who others will bet on, not intrinsic merit — but startups now have plenty of capital, so the game has changed.
Groq Bonds: Trading Salary For Survival
Three weeks from zero cash, leadership drafted layoff lists; Ross realized cutting the compiler team (writing a novel kernel-free compiler) would kill the product before it worked.
Instead, he proposed “Groq Bonds”: voluntary salary-for-equity swaps down to statutory minimums (~$50–60k). 80% participated, ~50% took the minimum.
Runway extended from 3 weeks to ~2 months; attrition dropped below 5%. “Put everyone’s hands on the steering wheel” — owners tolerate risk better than passengers.
Hire For Negatives, Grow For Positives
Ross’s “people spec” (versioned like a product spec) listed positives: return on luck, poetic design (semantic density), loss aversion.
Each positive has a negative inverse (squanders luck, maximalist design, books wins late). One negative hire poisons the team.
Hiring = vetting (look for negatives); growing = coaching (show positives). He split the mental modes after watching a Head of HR excel at firing by spotting negatives.
Loss Aversion And Booking The Win Early
Humans feel losses ~6× more than equivalent gains. Ross “books the win” the moment he hears something is possible: “If we don’t do this, the chip is half as fast.”
He hires people with the same reflex — they hear an architectural improvement and immediately treat not doing it as a loss.
Michael Jordan weaponized humiliation: taunt opponents, bet on everything, force himself to perform or suffer the shame. Manufactured pressure → superhuman output.
Manufactured Discontent Drives Everything
In a room of successful people, entrepreneurs were the least happy with their wealth despite having the most; their discontent fueled the next company.
Others were discontent with past work product (writing, code). Everyone at the top runs on some engineered dissatisfaction — divine discontent (Ogilvy), “what’s next?” (Jobs).
Ross’s current discontent: global compute shortage. Every day without enough compute delays medical breakthroughs, aging research, cancer cures — real lives lost.
Code Was Rationed, Now It’s Nearly Free
Software engineering has been “code rationing”: features are expensive, so engineers say no. Agile helped, but the default is still restraint.
