OpenAI Tests Outcome-Based Pricing: Payment Only When AI Succeeds

Traditionally, AI vendors have billed users based on computational tokens consumed, regardless of whether the model delivered useful results. OpenAI is now testing a shift toward outcome-based pricing with some enterprise customers, where payment depends on whether an AI agent actually completes its assigned task successfully.

The model sounds appealing in theory—customers pay only for work that gets done. But in practice, defining and verifying success introduces technical and economic challenges that could reshape how AI vendors operate.

The Problem With Measuring Success#

Unlike token counting, which is straightforward, success evaluation becomes murky with nuance. Straightforward cases exist: a support ticket closes without escalation, a unit test passes, an API returns the expected response. But many real-world tasks leave room for interpretation.

Consider a coding agent tasked with fixing an authentication bug. It rewrites the code, passes all tests—but the patch breaks something once deployed to production. Technically, the agent completed its assigned task, yet the customer wouldn't call it successful. Similarly, a multi-step agent might complete 90% of a job before failing on the final step. Under token-based pricing, all that effort gets billed. Under outcome pricing, a partial success may yield zero revenue.

Turning Evaluations Into Billing Infrastructure#

Developers already use evals—automated tests that grade model performance—to catch issues before deployment. Some of these are objective: did the unit test pass? Did the database query return the right record? Others are subjective and require semantic judgment, using an LLM as a judge to compare outcomes.

The problem emerges when those evals become the billing mechanism. An LLM-as-a-judge could incorrectly approve failed work (costing the vendor revenue) or reject successful work (making the customer pay for nothing). OpenAI would essentially be grading its own homework and charging based on that self-assessment—a potential conflict of interest, especially on subjective tasks like evaluating report quality or attribution of business outcomes.

Who Bears the Cost of Failure?#

Outcome-based pricing inverts the economic risk. Under today's model, a failed agent that burns through multiple retries still generates revenue from token consumption. Under outcome pricing, that failed run becomes the vendor's loss.