Comma AI founder sparks debate over whether AI should refuse harmful requests

In a weekend blog post responding to AI safety proposals, Comma AI founder George Hotz staked out a contrarian position: AI systems should be designed to serve their users without refusing requests, no matter how dangerous or harmful those requests might be.

Hotz's remarks touch on a fundamental tension in AI development—how to balance user autonomy with societal safeguards.

The case for locally controlled AI#

Rather than relying on centrally managed AI services like Claude and ChatGPT, Hotz advocates for locally controlled models that align closely with individual user preferences. He notes that current infrastructure constraints have made centralized services necessary, but as technology advances, truly personal AI systems could become feasible. This shift toward decentralized, user-owned models has appeal: it would reduce corporate intermediaries and give individuals more control over their tools.

The vision echoes the DIY ethos of early experimental systems, offering an alternative to today's heavily moderated, corporate-controlled AI.

A provocative framework#

To illustrate his point, Hotz compares user-aligned AI to a gun—a tool that performs its function without moral judgment. He argues a properly aligned AI would fulfill any user request: ordering restricted materials, providing harmful instructions, or assisting with dangerous acts. "We either live in a world with freedom or we don't," he writes, framing the question as binary.

The post is a response to AI 2040: Plan A, a policy paper proposing a 14-year slowdown in AI development to manage alignment risks.

The counterargument: Freedom within systems#

The author's response acknowledges the appeal of user-empowered AI but highlights an overlooked reality: societies, markets, and public-facing technologies require balancing competing interests. While personal freedom matters, so do the interests of others—including those who might be harmed by an AI trained to assist with violence or illegal acts.

Deploying mass-market technology isn't purely about individual liberty; it's about managing interdependent preferences and maintaining accountability. A system designed to defer entirely to its user's wishes, regardless of consequences for others, sidesteps that collective responsibility.

The debate underscores how AI alignment is not merely a technical problem but a social one—one that requires grappling with who AI serves and what safeguards, if any, should constrain that service.