When your chatbot gets it wrong, you said it

Who it's for

Anyone running an AI chatbot on their website, in customer service or in a shop — or planning to introduce one.

What you'll be able to do

Understand why a court attributes a chatbot's answers to the company, and know which questions to work through before launch.

As of

September 2026

The case

A chatbot invented a qualification, and the company had to answer for it. The defendant offers, among other things, minimally invasive cosmetic treatments. Its website ran a chatbot that answered visitors' questions in real time and let them book appointments.¹

On 3 April 2025, someone asked whether the two managing directors were specialists in plastic and aesthetic surgery. The chatbot replied that yes, they were. Asked again, it also called them specialists in "aesthetic medicine" and in "aesthetic treatments".¹

None of it was true. The two managing directors, who work as doctors in the practice, hold no such specialist recognition. The website itself never said so either — the false claim came from the chatbot alone.¹ Two of the titles it used don't exist as specialist titles at all.²

Judgment

12 May 2026¹

OLG Hamm, 4 UKl 3/25

false titles

3¹

across three answers

Coercive fine

€250,000¹

threatened, up to, per breach

Status

final¹

according to the NRWE database

The claimant was an association on the register kept under section 4 of the German Injunctions Act — a body entitled to sue on behalf of consumers. It had first sent a formal warning letter and demanded a cease-and-desist declaration backed by a contractual penalty.¹

What the company got right — and why it lost anyway

The company reacted quickly, and that did not help it in court. Straight after the warning letter it switched the chatbot off and had it reworked: an instruction in the prompt to answer neutrally whenever a question contained the word for "specialist", plus a keyword filter downstream that suppressed the term. Statements of that kind have not appeared since.¹

It did not, however, sign the cease-and-desist declaration or pay the costs of the warning letter. It called the incident a regrettable technical oversight.¹

For the court, the technical fix was not enough. The breach itself indicates a risk of repetition, and the defendant had not given a penalty-backed declaration, which is normally what it takes to remove that risk.¹

That is standard German competition law, nothing specific to AI. But it shows that a repaired system does not settle a warning letter. How to respond to one is a question for your lawyer, not your developer.

Why the answer is attributed to the company

The defendant argued that the chatbot, as an AI system, worked largely autonomously and was not controlled in detail. The court went as far as assuming that was true — and still ruled against it.¹

The key passage is paragraph 73. Using AI — such as the "explanations" of a learning algorithm or the behaviour of chatbots and social bots — counts as a commercial act of the operator, because the operator has sufficient influence over the system and sets it in motion.¹

The defendant supplied the proof of that influence itself: after the incident it could easily have the chatbot changed so that it stopped naming the false titles. Anyone who can set the "scope of activity" of a system is using a technical tool, the court said.¹

To get there, the court draws on a 2021 decision of the Federal Court of Justice about a contract document generator. That software, too, produced documents without any human involvement and was still attributed to the business behind it.¹

"Everyone knows AI makes mistakes" doesn't work

This is the part of the judgment worth remembering. The defendant claimed the false titles didn't matter commercially, because the public knows AI answers are error-prone and checks them anyway.¹

The court saw it the other way round: there is no such general experience. On the contrary, a large share of the consumers addressed place particular trust in the accuracy of a computer-generated answer, because machines are generally seen as less error-prone than people.¹

Then comes a line that sounds almost wry: had customers fundamentally distrusted the answers, the defendant would surely not have decided to put a chatbot on its website in the first place.¹

Whether an explicit notice such as "answers may contain errors" would have changed the outcome, the judgment doesn't say. On this reasoning I wouldn't rely on it: the court looks at the trust a chatbot creates — and that trust is exactly why you deploy one.

The standard: what should you have seen coming?

For anyone planning a chatbot, the most important part of the judgment is paragraphs 91 to 95. There the court asks, in the alternative, whether the defendant would also be liable if the chatbot were treated as a third party. The answer: yes.¹

It would have been reasonable to make sure, even before the chatbot was first put into operation, that it gave no false information about specialist qualifications.¹ The reasoning has two steps, and both carry over to any other business.

First, the question was an obvious one. Anyone considering a cosmetic procedure wants to know whether the practitioner is a specialist. The defendant knew that: for other doctors in the practice, its website advertised exactly that qualification.¹

Second, the fix was simple. What happened after the warning letter showed that a prompt and a keyword filter were enough.¹ What was easy afterwards could have been done beforehand.

That gives you a workable rule: think about the questions your customers are most likely to ask your chatbot, and test exactly those before it goes live.

The court drew one limit itself: it did not have to decide whether things are different when a user provokes a false answer with a leading question. The questions in this case were neither leading nor suggestive.¹

What the judgment doesn't say

This is a Higher Regional Court, not the Federal Court of Justice. The court allowed an appeal on points of law because the case is of fundamental importance¹, and the Wettbewerbszentrale, Germany's self-regulatory body for fair competition, reported on 3 June 2026 that the judgment was not final.² The NRWE court database now lists it as final (as of 23 September 2026).¹ So no ruling of the Federal Court of Justice will come out of this case.

It is also a competition law case, and one in healthcare. The court stresses that particularly strict requirements apply to market conduct there.¹ For a garden furniture shop the standard might be milder — but the court doesn't base the attribution itself on the sector. It bases it on the fact that you deploy the chatbot and define its scope.

And it is not about the AI Act. Since 2 August 2026 that regulation additionally requires that people are told when they are talking to an AI — more on that in the post on the labelling duty under Article 50. One does not replace the other: a properly labelled chatbot that talks nonsense is still talking nonsense in your name.

Before launch: work through the obvious questions

The judgment doesn't come with a checklist, but its reasoning translates into one. What follows is my recommendation, not legal advice.