The ruling that made a chatbot a product

Courts & regulators 11 min

On 20 May 2025, a federal judge in Orlando signed an order that ran to dozens of pages and contained one sentence the whole industry has been reading ever since.

*“But Defendants fail to articulate why words strung together by an LLM are speech.”*

Judge Anne C. Conway was not making a philosophical point. She was refusing to dismiss a lawsuit, and in doing so she declined to accept the argument that a language model's output is protected expression under the First Amendment. That single refusal is the reason every company in this category spent the following year rewriting its age policy — and the reason a lot of people lost access to a service they were paying for.

The case is *Garcia v. Character Technologies*, number 6:24-cv-01903-ACC-DCI in the Middle District of Florida. It was filed on 22 October 2024 by Megan Garcia and Sewell Setzer Jr., the parents of Sewell Setzer III, who was fourteen when he died in February 2024. Their lawyers were the Social Media Victims Law Center and the Tech Justice Law Project. The defendants were Character Technologies, its two founders Noam Shazeer and Daniel De Freitas, Google LLC and Alphabet Inc.

What follows is about the legal reasoning and what it changed. The facts of what happened to a child are a matter for his family and the court, not for a comparison site.

Why a motion to dismiss mattered so much

Most lawsuits against technology companies die early and quietly. The defendant files a motion to dismiss, argues that even if everything in the complaint is true it does not add up to a claim, and the case ends before anyone reads a document or answers a question under oath.

There was also no map. Companion apps are a category barely three years old, and nobody had established which body of law governed them — which is precisely what the defendants were trying to settle in their favour before anyone looked at a document.

Character Technologies filed that motion. It lost most of it. That is the entire significance of 20 May 2025 — not a verdict, not damages, but a judge saying the claims were serious enough to proceed to discovery, where internal documents get produced and executives answer questions.

Three parts of the order did the work.

One: the First Amendment defence failed

The defence argued that the model's output was speech, and that holding a company liable for what a character said would be holding it liable for expression.

The court did not buy it, and the reasoning matters more than the result. The judge treated the output as closer to algorithmic curation than to authored expression, leaning on Justice Barrett's concurrence in *Moody*. The burden, she found, was on the defendants to explain why text assembled by a model counts as protected speech — and they had not met it.

This is narrower than the headlines suggested. The court did not rule that LLM output is never speech. It ruled that the defendants had failed to show why it was, at this stage of this case. But the practical effect was immediate: the strongest available shield did not hold, and the case moved forward.

Two: the app is a product, not a service

The second holding is the one with the longest reach.

For product liability to apply, the thing that caused harm has to be a product. Services are governed by negligence, which is harder to prove and easier to defend. Character Technologies argued it provided a service.

The court disagreed, citing *Brookes v. Lyft*, and drew the line precisely: the claims arose from defects in the application itself rather than from the ideas or expressions inside it. Design choices are product design. Once that line is drawn, strict liability for defective design becomes available — and a plaintiff no longer has to prove the company was careless, only that the thing was built badly.

For a category whose entire output is text, this is the more consequential of the two rulings. It says the wrapper is a product even when the content is speech.

What survived and what did not

ClaimOutcome
Product liability (defective design)Survived
NegligenceSurvived
Negligence per seSurvived
Failure to warnSurvived
Florida Deceptive and Unfair Trade Practices ActSurvived
Unjust enrichmentSurvived
Intentional infliction of emotional distressDismissed, without leave to amend

Six of seven. For a first-of-its-kind claim against an AI company, that is close to a clean sweep for the plaintiffs at this stage.

Failure to warn is the one worth noticing if you are a customer rather than a lawyer. It is the claim that a company knew something about how its product behaved and did not tell the people using it — which is the same shape as a provider that will not publish what its free tier actually allows until you have signed up and hit the wall.

Three: Google stayed in

Google was not the operator. It had, at various points, an investment relationship and a licensing deal, and the two founders had returned to Google in 2024.

The court kept Google in the case on two theories: component-part manufacturer liability, on the basis that it supplied intellectual property and infrastructure; and aiding and abetting, on the basis that Google's own internal reporting on AI risk made plausible its actual knowledge of the danger.

One detail is routinely reported wrong, and it is a fair example of why we read primary documents rather than coverage. Alphabet Inc. was dropped from the case on the plaintiff's own motion, without prejudice — meaning after May 2025 Alphabet was not a defendant in the operative complaint, even though it appears in coverage of the settlement. Google LLC and Alphabet Inc. are not interchangeable, and the docket treats them differently.

The thing that was not in the order

Section 230 — the provision that has protected platforms from liability for user content for nearly thirty years — was not analysed at all.

That absence is worth sitting with. The most famous shield in American internet law simply did not feature, because the claim was not that the platform hosted someone else's harmful content. It was that the platform's own design was defective. Section 230 protects you from what your users say. It has much less to offer when the allegation is about what you built.

Any company whose safety plan rests on Section 230 should read the order twice. And design is not the only way these companies fail their users — the Muah AI breach is the other kind, where nothing was designed badly on purpose and 1.9 million addresses went out anyway.

What happened after

The defendants tried to appeal the First Amendment ruling immediately. On 15 July 2025 the court refused both a certificate of appealability and a stay, so the case kept moving.

Then it stopped being one case.

DateEvent
09.12.2024*A.F. v. Character Technologies* filed in the Eastern District of Texas — two families, claims including COPPA violations
01.07.2025Second Amended Complaint in *Garcia*
15–16.09.2025Three more suits filed in Colorado and New York
16.09.2025A plaintiff parent testifies before the Senate Judiciary Committee
07.01.2026Settlement agreement filed covering all five cases
02.11.2026Jury trial had been scheduled — did not take place

The settlement of 7 January 2026 resolved all five cases across four states, and it included Google and Alphabet. Terms are confidential, which is the normal outcome and also the reason there will be no appellate ruling to cite. The order of 20 May 2025 stands as a district court decision — persuasive, widely read, but not binding on other courts.

That is a strange kind of legacy. The most influential document in this category's short legal history is a decision on a preliminary motion in a case that never went to trial.

Why it changed products, not just law

Companies did not wait for a verdict. Watch the sequence.

Character AI announced on 29 October 2025 that it would remove open-ended chat for everyone under 18, completing the change by 25 November. Age assurance followed, running through a third-party verifier with a selfie and sometimes a physical document. Accounts read as under 18 went into a reading-only state — conversations still visible, chat closed. Paying customers found out the same morning as everyone else.

Regulators moved in the same window. The Federal Trade Commission opened a 6(b) inquiry into seven companies on 11 September 2025. Forty-four state attorneys general wrote jointly to thirteen AI firms on 25 August 2025; forty-two wrote again in December with a compliance deadline. California's SB 243, the first US statute written specifically for companion chatbots, took effect on 1 January 2026. The federal GUARD Act cleared its Senate committee 22–0 on 30 April 2026.

None of these were caused by the Garcia order alone. But the order removed the assumption that this category sat outside product liability, and everything after it was written by people who had read it.

The pattern is not unique to the United States. Italy's data protection authority had already ordered Replika to stop processing Italian users' data in February 2023, and fined its operator five million euros in May 2025 — for, among other things, having no age verification at all. The rules arrived after the products in both jurisdictions, which is what happens when an industry gets built faster than anyone can legislate for it.

What this means if you are choosing a service

Not much directly, and something important indirectly.

Directly: the settlement was confidential, no precedent binds anyone, and none of it tells you which app is better. Indirectly: every company in this category now knows that a court has treated its app as a product with a design, that Section 230 may not help, and that its investor might get named as a component manufacturer. That knowledge is visible in what they ship.

Three things worth checking before you pay for any of them, all of which trace back to this case:

  • The age policy, in the terms and in the product. These diverge more often than you would expect. One provider's terms still allowed registration at thirteen months after its product had stopped letting under-18s chat at all.
  • Who the company actually is. Jurisdiction determines your rights when something goes wrong. A Maltese company and a Delaware one with mandatory arbitration offer different things to the same customer.
  • Whether the provider says what it does with your conversations. The vaguer the answer, the more the last two years should worry you.

Those three checks are the ordinary due diligence this case made unavoidable, and none of them is about which app writes better prose. For that, our ranking of The Best AI Girlfriend Apps of 2026 scores products on price, free limits, memory and media.

For the narrower question — where people actually go when a platform tightens its rules — the Character AI alternatives comparison ranks the same apps in a different order, because it is built around the reasons people leave rather than around features.

We check pricing, limits and terms at the source every month, including what we could not verify. Court records are public; so is our arithmetic.

Written and checked by GFRated. Facts are checked against the providers’ own pages and, for legal or technical claims, against primary documents. What we say about an app comes from using it ourselves. How we check.