AI Platform
How DoorDash Built a GenAI Platform That Works for Non-Engineers
DoorDash shared how it evolved from an LLM gateway to a vendor-agnostic platform that now serves over 5,000 internal users—including legal, sales, and operations teams—while balancing speed, reliability, and cost.
InfoQ
After ChatGPT's breakout moment in late 2022, DoorDash moved quickly to harness generative AI across the company. What started as a contract signing in April 2023 evolved into a full-fledged platform serving thousands of users and reshaping how the delivery giant thinks about AI adoption, cost control, and model portability.
From ML Engineers to Everyone#
The team's initial assumptions crumbled fast. Early on, DoorDash's GenAI Platform operated as part of the broader ML platform organization, naturally oriented toward machine learning specialists. But when product engineers asked basic questions—"what's a notebook?"—it became clear the real customers weren't ML researchers. They were general software engineers, and soon after, non-technical teams.
This shift forced a hard pivot: the platform moved from notebooks and GPU compute to APIs and SDKs. By 2024, DoorDash had onboarded over 5,000 internal users daily—including staff from legal, sales, operations, and strategy. Forty percent of the platform's user base comprises non-engineers, a reality the team had not anticipated.
The LLM Gateway: One Door to Many Models#
To unblock this explosion in demand, DoorDash built the LLM Gateway—a central abstraction layer that solved velocity and accountability at once. Rather than every team maintaining custom integrations with OpenAI, Anthropic, Gemini, and open-source models, the gateway provides one API and SDK.
The benefits compounded:
- Model experimentation: Teams can switch between providers and models with a single config change, crucial since new models launch every few months.
- Reliability and fallbacks: When one provider hits capacity limits, requests automatically route to secondary endpoints (e.g., Azure fallback for OpenAI, Bedrock for Claude).
- Cost tracking and attribution: Finance gained visibility; departments could track spend and optimize their usage by model, team, and workload.
- Centralized logging: Engineers can debug production performance without building custom observability systems.
The gateway's design—built for the entire company, not one team—also enabled seamless onboarding after DoorDash acquired Wolt, Deliveroo, and SevenRooms. All subsidiaries got the same velocity, reliability, and cost control instantly.
