The Right to Be Forgotten by a System That Never Remembered You

In April 2024, a public figure in Austria asked ChatGPT for their date of birth.

ChatGPT got it wrong.

This is not, in itself, remarkable. Large language models generate incorrect information routinely; the industry calls it hallucination, a term that implies the system is perceiving something that isn't there, when what it is actually doing is producing a statistically plausible answer that happens to be false. The answer looks like a memory. However, it is a guess wearing a memory's clothes.

The public figure, through noyb, asked OpenAI to correct the date. OpenAI responded that this was technically impossible. Not that they would not do it. That they could not. The model does not store biographical facts as retrievable records. It stores statistical relationships between tokens - patterns derived from billions of documents, compressed into parameters that cannot be individually located, identified, or modified. To "correct" the date of birth would require identifying which of the model's billions of parameters contribute to the incorrect output, changing them without disrupting everything else those parameters do, and verifying that the correction holds across all possible prompts that might elicit the answer.

OpenAI offered, instead, to filter the output - to block certain prompts from returning information about the complainant. This is not correction. It is suppression. The incorrect data remains in the model's weights; the model is simply prevented from saying it out loud under specific conditions. The data subject's personal data has not been rectified. It has been gagged.

noyb filed a complaint with the Austrian Data Protection Authority. As of June 2026, the complaint is unresolved.

What Erasure Means When Nothing Was Stored

GDPR Article 17 grants data subjects the right to obtain erasure of personal data "without undue delay" where the data is no longer necessary for its original purpose, where consent has been withdrawn, where the data has been unlawfully processed, or where the data subject objects to processing. The controller must erase the data and, where it has been made public, take reasonable steps to inform other controllers processing it.

The provision was designed for databases. Records in systems. Rows that can be located and deleted. The mental model is a filing cabinet: the document exists, it can be found, and it can be removed. After removal, it is gone. The absence is verifiable.

In a large language model, personal data does not exist as records. It exists - if it exists at all - as distributed patterns across the model's parameters. The person's name, biographical details, and professional history are not stored in an identifiable location. They are encoded as statistical relationships between tokens, derived from training data that may no longer exist in its original form, compressed into a representation that is, by design, not reversible to its inputs.

To erase personal data from a trained model, the controller would need to do one of three things. Retrain the model from scratch without the data - a process that costs, by current estimates, seven figures or more. Apply machine unlearning techniques - a field that is experimental, with no established standard for verifying that the unlearning was effective. Or filter the output - which is what OpenAI offered, and which is not erasure but suppression.

The right to erasure meets a system where the data was never stored in a form that permits erasure, where the model generates the data fresh on each request from patterns that cannot be individually addressed, and where the only available response is to prevent the model from saying what it still, in some statistical sense, knows.

The Clause - the one that gives every data subject the right to demand erasure of personal data - is entirely clear on the obligation. It is less clear on what erasure means when the data was never stored, is generated fresh each time, and cannot be proven absent.

But, then again, that is another man's problem...

The Judge Who Said: Remember

On May 13, 2025, Magistrate Judge Ona Wang of the Southern District of New York issued a preservation order in the New York Times copyright litigation against OpenAI. The order required OpenAI to retain all ChatGPT output logs - indefinitely. Including logs that users had requested to be deleted. Including logs from users who had exercised, or would exercise, their right to erasure under applicable privacy law.

The order was not about privacy. It was about evidence. The New York Times alleged that ChatGPT reproduced copyrighted content. The logs were evidence of reproduction. Judge Wang found that the potential evidentiary value outweighed OpenAI's privacy commitments to its users. OpenAI called the ruling "a privacy nightmare." The court affirmed it in December 2025, ordering production of over twenty million anonymized logs.

For European data subjects, the conflict is not theoretical. GDPR Article 17 creates a right to erasure. A US federal court order creates a duty to preserve. The obligations point at the same data. They point in opposite directions. OpenAI operates in both jurisdictions. It cannot comply with both simultaneously.

This is not a novel conflict between legal systems. International businesses routinely navigate conflicting regulatory obligations. What is novel is the object of the conflict: conversation logs that the user believed were ephemeral, that the company promised could be deleted, that European law requires to be deletable, and that an American court now requires to be permanent.

As for all the specs of irony throughout the history of the international law, this one hit pretty spot on - the data subject who asked to be forgotten has been ordered - by a court in a different jurisdiction, in proceedings to which they are not a party - to be remembered.

The Fine That Did Not Survive

In December 2024, the Italian Garante per la Protezione dei Dati Personali fined OpenAI €15 million for multiple GDPR violations related to ChatGPT. The violations included inadequate legal basis for processing, failure to provide adequate notice to data subjects, and failure to implement age verification. It was the only significant GDPR fine imposed on a generative AI company in Europe.

On March 18, 2026, the Court of Rome annulled the fine entirely.

The annulment was not on the merits. The court did not rule that OpenAI's practices were lawful. It ruled that the Garante lacked jurisdiction. OpenAI had established an Irish subsidiary in 2024, making the Irish Data Protection Commission its lead supervisory authority under GDPR's one-stop-shop mechanism. The Garante issued its final decision after the Irish DPC became the lead authority. The jurisdictional timing was fatal to the enforcement action.

The practical consequence: the only enforcement action that tested whether GDPR's provisions apply meaningfully to large language models was resolved on a procedural technicality. The substantive questions - whether training constitutes lawful processing, whether the accuracy principle applies to generated content, whether erasure is possible in the technical sense Article 17 requires - remain unanswered.

The Garante tested the wall. The wall did not answer. It redirected the inquiry to Dublin.

Thirty Authorities, One Question

The EDPB launched its 2025 Coordinated Enforcement Framework action on the right to erasure - the largest coordinated GDPR enforcement action focused on a single article. Thirty-two data protection authorities across Europe participated. Nine initiated formal investigations. Twenty-three conducted fact-finding exercises. The report, adopted in February 2026, describes what they found.

Seventeen DPAs - more than half of those participating - raised concerns that controllers lack documented procedures for handling erasure requests, or have procedures that are incomplete and reviewed only in response to incidents. Half of responding DPAs found that many controllers have no specific procedures for erasure from backup systems, with some not deleting personal data from backups at all. Multiple DPAs identified weak anonymisation techniques that amount to mere pseudonymisation, leaving re-identification risks unaddressed.

These findings describe the conventional erasure problem: data in databases, files in systems, records in backups. The procedures do not work well even for data that can be located and deleted. The EDPB's coordinated action does not specifically address AI training data or model weights - the erasure problem where the data cannot be located at all.

The action was necessary and useful. Its scope was the erasure problem of 2018, not the erasure problem of 2026. The filing cabinet has been audited. The neural network has not.

The EDPB has since selected transparency and information obligations as its 2026 coordinated enforcement topic - a move in the regulator's gift tradition: addressing the obligation to tell people what is happening to their data, rather than the obligation to stop it from happening. Transparency is the lighter instrument. It is also, in the AI training context, the instrument that requires the least confrontation with the technical impossibility of the alternative.

The Erasure That Cannot Be Verified

The right to erasure rests on two assumptions that large language models break simultaneously.

First: that personal data exists as discrete, locatable records that can be identified and removed. In a trained model, personal data exists - to the extent it exists at all - as distributed statistical patterns that are not individually addressable. You cannot delete a person from a neural network the way you delete a row from a database. You can delete the training data that mentioned them. You cannot verify that the model has forgotten them.

Second: that erasure is binary — the data either exists or it doesn't. In a language model, the boundary between "contains personal data" and "does not contain personal data" is probabilistic, not categorical. The model may generate accurate personal information about a person not because it "remembers" that person but because the statistical patterns derived from millions of documents produce output that happens to be correct. Is the person's data "in" the model? The question does not have a binary answer. It has a probability distribution.

The right to be forgotten was designed for a world of records. It was designed for a world where a data controller could receive a request, locate the data, delete it, confirm its deletion, and move on. In the world of AI models, the controller receives the request and faces a choice: retrain the entire model at extraordinary cost, apply experimental unlearning techniques of unverified efficacy, filter the output while the data remains in the weights, or explain that compliance in the sense Article 17 contemplates is not technically achievable with current technology.

None of these responses satisfy the right as written. All of them are being offered as compliance. The gap between the legal obligation and the technical response is not a temporary problem awaiting a solution. It is a structural feature of the technology, and it will persist as long as the regulatory framework assumes that data can be found and deleted, and the technology distributes data across parameters that cannot be searched.

An Austrian public figure asked ChatGPT for their birthday. The model got it wrong. The data subject asked for correction. The controller said it was impossible. A US court ordered the conversation preserved. An Italian regulator tried to enforce the law and lost on jurisdiction. Thirty-two European authorities audited erasure procedures and found that most controllers cannot manage erasure from backup systems, let alone from neural networks.

The right to be forgotten is the clearest right GDPR provides. It is also, in the AI context, the right whose exercise is most dependent on a technical architecture that does not support it.

The model does not remember you. It cannot prove it has forgotten you. The right exists. The erasure does not. Somewhere between these two facts, a compliance officer is writing a response that says: we have taken appropriate measures.

In the meantime - the measures are appropriate. The forgetting is not verifiable. The filing cabinet has been replaced by something that has no drawers, and the regulation is still looking for the key.