They’re Building the AI That OpenAI Never Wants You to Own | AGI Inc.

EO • • 25min → 4 min • #43
They’re Building the AI That OpenAI Never Wants You to Own | AGI Inc.
Watch on YouTube

Summary

  • AGI Inc. is a startup building truly personal AI agents that run entirely on-device — on phones, smart glasses, watches, and custom wearables — so users own their model and data, unlike cloud-dependent offerings from OpenAI or Anthropic; the company was founded by Div Garg, who dropped out of a Stanford PhD and turned down a billion-dollar OpenAI offer to pursue this vision of a Jarvis-like assistant that equalizes access to an “army of agents” for everyone.

Origin and founding vision

  • Div Garg studied physics and CS at Cornell (International Physics Olympiad gold medalist), then joined Stanford for graduate studies where he co-created the university’s first Transformers course (CS 25), recruiting lecturers like Geoffrey Hinton, Anthropic co-founders, and OpenAI researchers by cold-emailing them.
  • He dropped out of his PhD and declined a billion-dollar offer from OpenAI because he believed building on-device personal AI is “the most impactful thing we can do for the whole humanity right now” — removing the gap between billionaires who have human assistants and everyone else by giving each person an army of AI agents.
  • The core analogy is Jarvis from Iron Man: a personal AI butler that knows you, takes ambiguous requests, understands underlying intent, and eventually acts proactively — all while keeping data on the device so there is “no cloud to trust.”

Technology stack and on-device differentiation

  • The stack has three layers: application layer, agent harness, and inference engine; the agent receives a task and a screenshot, predicts the next action (click, type, scroll), executes it, and loops until the task is complete.
  • Current bottleneck: getting the full inference pipeline on-device — right now it runs in the cloud, and shrinking the model reduces capability; the research challenge is maintaining accuracy and speed on edge hardware.
  • Memory and context are the new rate limiters: the system must understand the user well enough to handle ambiguity, recognize intent, and eventually act proactively; trust enables users to share more context, which increases capability.
  • Compounding error problem: a 90% single-turn accuracy drops to ~40% over 20 tasks; AGI Inc. focuses on solving this reliability gap for production use, which is why partners like Qualcomm come to them.
  • Benchmarks: currently number one on Android World for on-device agent performance.

Product capabilities and demos

  • Hands-free voice control of any app: demonstrated ordering a customized Chipotle bowl via DoorDash — the agent opened the app, selected steak, brown rice, lettuce, corn, guac, salsa, sour cream, cheese, and confirmed the $91 order with a guardrail prompt.
  • Accessibility focus: enabling grandparents to order food or FaceTime grandchildren by voice alone, without navigating complex UIs.
  • The goal is a new operating system for devices that lets people look at screens less and spend more time in real life — the phone stays in your pocket while the AI acts in the background.

Hardware and wearables strategy

  • Will leads growth, wearables, and hardware; the team is experimenting with form factors including pendants you speak into (modern-day pager), “mini phones” (chunky iPod-like devices), and partnerships with chip companies like Qualcomm.
  • Qualcomm and other device makers partnered after seeing tech demos: “Wow, I’ve never seen a phone do that before” — and it was running on their chip.
  • The unusual path: instead of competing in crowded markets like spreadsheet automation, AGI targets edge-device inference where device makers have a need no one else was filling.

Business model and competitive positioning

  • Fundamental difference from frontier labs: AGI sells/runs models on the user’s device — you own the model, you own the data; OpenAI and Anthropic’s business models depend on cloud inference, so they “don’t want” to ship on-device models.
  • Not renting a model on someone else’s cloud; the vision is billions of personalized, self-learning models that improve over time and are fully trusted because they never leave the device.
  • Partners and customers are device manufacturers who need on-device agent capabilities; AGI remains “the best in that category” per partner feedback.

Technical challenges: demo to production

  • The gap between a demo (works 10% of the time) and production (billions of users, robust edge-case handling) is like self-driving cars: Waymo and Tesla took over a decade to go from concept to road-ready.
  • Real-world chaos: users ask unpredictable things (e.g., “file my taxes on my phone”), apps change, screens vary; the agent must handle this reliably across 10–20 step tasks without compounding failures.
  • Current focus: improving per-turn accuracy and memory/context systems so multi-step tasks stay reliable; this is the core value proposition for partners.

Company culture and team

  • Work style: “996 type of company” — 9:30 AM to 9:30 PM most days; founders and team say “if you love what you’re doing, it doesn’t matter” and describe it as a meaningful sacrifice for something valuable.
  • Night owl culture: many work late (2 AM+) for distraction-free focus; autonomy is high — people own decisions and unleash creativity as long as they perform.
  • Hiring: “hire really slow, fire very fast” — culture fit is paramount; every person adds to the culture, which “cannot be enforced” but forms organically.
  • Trials before full-time: new hires (like Shariq on day one) get a year’s worth of feature implementation as a test, evaluated on handling ambiguity, product intuition, and iteration speed.
  • CK described as “big brother of the company” and central to culture; team events (co-working sessions, trips releasing mini phones) build serendipity and recruit researchers.
  • Creative freedom: early on, people identify strengths and get ownership; e.g., one member’s first contribution was the mini phone project, which became a conversation-starter at events.

Future vision

  • Billions of personalized models — each person buys and customizes their own model that learns preferences over time, like modifying your own car rather than sharing one generic car for humanity.
  • The AI relationship may become “more influential than any human relationship or at least as influential”; choosing the design of that AI is critical.
  • Ultimate goal: contribute to making the world better by equalizing access to AI agency, reducing screen time, and letting people be more present in real life.
Back to EO