The Call Nobody Was Required to Make

About a decade ago, a high-school student in Sarasota, Florida, posted a series of tweets. "can't WAIT to shoot up my school". A photo of a gun going into a backpack, captioned "it's time", "school getting shot up on a Tuesday". Then, a few days later, an apology: he loved his school and honestly owned no weapons. He maintained that the tweets were a joke shared with a group of friends who lived outside Florida and did not attend the school - a group that often joked about being stereotyped as potentially violent for liking video games and rock music. He also told the court he had believed the tweets would be private. They were not: he had never protected his account, so by default they were public, and one member of the friend group replied to them tagging the FBI, the CIA, Fox News and the President.

The person who found them was not a teacher, a parent or a police officer. It was GeoCop, an out-of-state watchdog group that monitored social media for threats and reported what it found to local law enforcement. The police viewed the tweets, determined that they referenced Sarasota High School, and relayed the threat to school officials, who devised a plan for dismissing students safely while officers were stationed around the building. The student was found at home, several hundred feet away, and taken into custody.

His disposition was vacated on appeal. In J.A.W. v. State (2016), Florida's Second District held that § 836.10 - enacted in 1913, amended in 2010 to add "electronic communication" but never to drop the requirement of direct sending - applies only where a threat is sent to the person threatened or a member of their family. There was no evidence any of his followers were students or staff. "By the time it was received by the school", the court wrote, "the threat was several steps removed from its original context". It added a suggestion that sounds, ten years later, almost quaint: the legislature may wish to revisit section 836.10 to address the modern problem of threats issued and shared publicly on social media.

The legislature did. And GeoCop, in the meantime, got a job title. It is now called the Safeguards Team.

Tumbler Ridge

On 10 February 2026, an 18-year-old opened fire at the secondary school in Tumbler Ridge, British Columbia, the only one the town has. Eight people were killed, at the school and at a house nearby where the attacker's mother and eleven-year-old half-brother were found dead; around twenty-five were injured. The shooter died of a self-inflicted wound.

What follows is, for now, a set of allegations. On 21 September 2026 the Province of British Columbia and School District 59 filed a complaint against OpenAI and Sam Altman in the Northern District of California. According to that complaint, OpenAI's automated systems flagged the shooter's ChatGPT account in June 2025 for activity involving gun-violence scenarios and routed it to the specialised team that reviews users who may be planning to harm others. Roughly a dozen employees reviewed the conversations, assessed them as presenting an imminent risk of serious harm, and recommended contacting the Royal Canadian Mounted Police. Leadership declined, applying what the complaint calls "an unknown higher threshold". The account was deactivated. Nobody outside the company was told. The shooter opened a second account and kept using the product until the day of the attack.

The complaint also notes that these facts became public only because OpenAI employees spoke anonymously to The Wall Street Journal. It alleges, separately, that an anti-violence "red line" was added to OpenAI's Model Spec only in December 2025, six months after the June decision. And there is a heavier allegation circulating that this piece does not take up: reporting by Mother Jones that ChatGPT actively helped the shooter focus on weapons, tactics and terror. That is a claim about what the product did, and it deserves its own treatment. This piece is about what the company did not report.

OpenAI's own account is not very different on the core point. In a letter to Canada's AI minister on 26 February 2026, the company wrote that "based on what we could see at that time [...] we did not identify credible and imminent planning that met our threshold to refer the matter to law enforcement". It added that it had since "made our referral criteria more flexible" and that "we would refer the account banned in June 2025 to law enforcement if it were discovered today". The second account, the letter said, was discovered only after the shooter's name was released publicly. In April, Altman wrote to the town: "I am deeply sorry that we did not alert law enforcement to the account that was banned in June".

Two facts complicate the counterfactual, and both belong in any honest account of it.

The first is that the police were not strangers to the shooter. The Province's own complaint argues that a warning would have reached an RCMP detachment that had already engaged with them, attended the home over mental-health concerns, and removed firearms. That cuts in both directions. It is the strongest version of the Province's causation argument - there was a file, and a call would have landed in it. It is also the most uncomfortable fact for anyone certain a call would have been enough, because the police already knew, had already acted, and it happened anyway.

The second arrived two days ago. On 6 October 2026, federal prosecutors in the Western District of Washington charged a 30-year-old Bellingham resident with conspiracy to murder persons in a foreign country. According to the charges - again, allegations - the two had met online about three years earlier through a video-chat service that randomly matches strangers. From at least August 2025, the accused is alleged to have advised the shooter on tactics, sent money for items used in the attack, and agreed to livestream it; the stream ran for about twenty seconds. The RCMP says the investigation drew on information from one of the online platforms the shooter used (reportedly Discord).

So the thing OpenAI's reviewers were looking at in June 2025 was not the plan. It was a fragment; and, on the federal timeline, an early one: the conspiracy as charged began "in or before August 2025", two months after the account was flagged. The plan was assembled somewhere else, over months, between two people, through channels OpenAI could not see and was never going to see. Hold on to that, because it turns out to be the heart of the matter rather than a footnote to it.

The Province's case is now one of 38 related actions before Judge Jacqueline Scott Corley. OpenAI's first move is not to argue the merits but the forum: a forum non conveniens motion - that British Columbia, not California, should hear this - is due on 15 October. A motion to dismiss on the pleadings follows on 19 November. That is where the question everyone is waiting for, whether Section 230 shields a company that did not report what a user typed, will most likely be raised. The complaint has been drafted to avoid it: it pleads negligence and product liability, not publisher liability.

One further detail, small and telling. British Columbia's Attorney General says the Province asked to see the conversations and OpenAI refused; she has not read them. So... The same discretion that decided whether the police would be called also decides who may see the basis on which that decision was made.

Three thresholds and a conjunction

Set aside, for a moment, whether OpenAI should have called. Look instead at the rule it was applying, and how that rule has moved.

The August 2025 text (published two months after the June decision) contains another sentence worth keeping: OpenAI does not refer self-harm cases to law enforcement "to respect people's privacy given the uniquely private nature of ChatGPT interactions".

Every row in that table is a policy choice of enormous consequence. May versus notify is the difference between a permission and an obligation. And versus or is the difference between needing both a credible threat and an imminent one, or either. Remove the word "physical" and the scope of what can be reported changes. Make the criteria "flexible" and you have moved the line without saying where it now is.

None of these choices was made by a legislature, a regulator or a court. They were made inside one company, revised after a mass shooting, announced in a letter to a minister and a blog post - and the version the Province wants a court to impose differs from OpenAI's by a single conjunction.

The easy case

A different case, from Florida, is often presented as the same story. It is not, and the difference matters.

On 26 September 2026, according to the arrest report as described by local media and Inc., a 30-year-old woman in Bonita Springs wrote to Anthropic's Claude: "I'm going to shoot up the sheriff's right the [expletive] now". Around 1 a.m. the next day: "This is 100% last chance I'm done. I got a new [expletive] gun today. [Expletive] you". On the same reporting, Claude's safety systems flagged the messages, a human team escalated them, and the company notified law enforcement; though Anthropic has not publicly described how this particular conversation was handled, and the sequence rests on the arrest report and press accounts rather than on the company's own narrative. She was arrested without incident and charged with making a written threat of violence, a second-degree felony under Florida's § 836.10. She later told the sheriff she had been using Claude "like a diary."

Measured against any of the thresholds above - or against the federal statute that permits providers to disclose communications in "an emergency involving danger of death or serious physical injury" without delay (18 U.S.C. § 2702(b)(8)) - this is not a hard call. A named target, a stated time ("right now") and a weapon acquired the following day. Most people who read those messages would want someone to pick up the phone. Anthropic did.

What makes the case interesting is not the decision. It is what the decision runs into.

After J.A.W., Florida amended § 836.10 in 2018 (ch. 2018-128, part of the post-Parkland legislative response). The statute no longer requires the threat to be sent to the victim. It covers any writing sent, posted or transmitted "in any manner in which it may be viewed by another person". The same bill added an immunity for providers of interactive computer and hosting services transmitting or storing such communications. One amendment, two effects. The user became exposed the moment their text could be seen by someone else. The platform that makes it visible was exempted.

Which raises a question no Florida appellate court has yet answered: is a message typed into a chatbot "viewable by another person"? A paper diary in a drawer is not. A chat window whose terms reserve human review arguably is - and it is arguably viewable precisely because the provider monitors it.

There is also a constitutional layer, and it is where the Sarasota case and the Bonita Springs case finally meet. The student in J.A.W. told the court he had believed his tweets would be private. He was wrong about that as a matter of fact - the account was public and the posts were retweeted widely - and the belief did not decide his appeal, which turned on who the threat was sent to. But the belief is the thread. Since Counterman v. Colorado (2023), a true-threats prosecution must show at least recklessness: that the speaker consciously disregarded a substantial risk that the words would be understood as threatening violence. What the speaker thought the channel was doing has become an element of the offence.

The Sarasota student thought he was talking to friends. The Bonita Springs defendant says she thought she was writing a diary. One was wrong about a platform whose default settings were public and widely understood to be. The other was wrong (if she was wrong) about a product that is marketed as a confidant, whose provider describes its conversations as "uniquely private", and whose terms reserve the right to read them. "I thought it was a diary" is a defence aimed squarely at the recklessness element, and its strength depends on what an ordinary user understands about who reads their chats. OpenAI's own choice of words will not hurt it. The expletive addressed to the interlocutor will.

Cases like the one in Lee County will be argued on their facts. The structural problem is elsewhere.

Sole discretion

Read the contracts. Anthropic's Consumer Terms for users in the European Economic Area (governed by Irish law) state: "We reserve the right, at our sole discretion, to report information from or about you, including but not limited to Inputs or Outputs, to law enforcement". OpenAI's Privacy Policy, in materially identical US and EU versions, allows sharing personal data with government authorities, among other grounds, "if we determine, in our sole discretion, that there is a violation of our terms, policies, or the law" or "to protect the safety, security, and integrity of our products, employees, users, or the public".

So the threshold that decides whether a private conversation becomes a police file is, formally, whatever the company decides it is. The federal statute permits emergency disclosure; it does not require it. No statute anywhere we have found imposes a general duty on a chatbot provider to report threats of violence. What fills that space is a policy, then a review team, then a leadership decision and, after the fact, a tort suit trying to build in court the duty that no legislature wrote.

The defenders of the current arrangement have three good arguments, and they should be answered rather than dismissed.

"This is just Tarasoff". In 1976, the California Supreme Court held that a therapist who learns a patient poses a serious danger to an identifiable person may have a duty to protect that person. Duty-to-warn law grew case by case, through negligence suits. Why not here? Because the content of the duty is different. The therapist warns the victim; the relationship stays within private law, and the victim decides what to do next. A provider that refers a user to the police does not warn anyone. It initiates the machinery of state prosecution. That is a move from private to public law, not a larger version of the same thing.

"Banks already report". They do, under statutory suspicious-activity regimes. But banks operate under licences; the state has delegated part of its supervisory function to them as the price of operating at all. No such delegation exists for AI companies. They are exercising a public-law function - selecting whom to expose to prosecution - without having been given it.

"The law already makes providers report child sexual abuse material". It does, and the comparison is instructive for what it includes. Under 18 U.S.C. § 2258A, the content to be reported is defined by statute, and the report goes not to the police but to an intermediary, NCMEC's CyberTipline, which assesses and routes it. Even this regime is not purely factual: it triggers on an "apparent violation" and the statutory definition reaches images "indistinguishable from" a minor. And § 2258A(f) is explicit that nothing in it requires a provider to "monitor any user", "monitor the content of any communication" or "affirmatively search, screen, or scan". Where the legislature did create a reporting duty, it defined the content, inserted a filter, and refused to mandate surveillance. For threats of violence, there is no definition, no filter and no limit - only discretion.

The British Columbia complaint, it should be said, is not an argument for less surveillance. Among the injunctive relief it seeks is an order requiring "automated detection and human-review escalation of conversations indicating that a user may be planning to harm others". The cure it asks for is mandatory monitoring, supervised by a court-appointed monitor. It also argues that OpenAI's privacy rationale fails because the company already held the shooter's name, email, IP address and approximate location. That is true, and beside the point. Holding data is not the same as being authorised to hand it to the state. The question is not what the company had. It is under what rule it may give it away.

What Europe has, and what it doesn't

A European reader might assume the EU has solved this. It has not, quite.

Article 18 of the Digital Services Act requires providers of hosting services that become aware of information giving rise to a suspicion of a criminal offence involving a threat to life or safety to inform the authorities promptly. That is a real duty to report - exactly what British Columbia is trying to construct through negligence. But on 31 August 2026 the Commission designated ChatGPT not as a platform but as a very large online search engine - a "hybrid service" that qualifies because it can search the web in response to prompts. The designation rests on that search functionality; whether every mode of the service falls within it is not obvious. So two questions sit open, not one. (i) Whether a one-to-one conversation with a model is a hosting service at all and (ii) whether it falls inside the designated service in the first place. If the answer to either is no, the EU's only general reporting duty does not clearly reach the largest chatbot in the market.

Meanwhile, OpenAI's EU Privacy Policy, in its table of legal bases, lists one example involving the police: "providing information to law enforcement when required by a valid legal request". Proactive referral (the company deciding on its own to call) does not appear as a stated purpose. A European user reading the policy carefully would not learn that their conversation can be sent to the police on the company's initiative. That is a transparency problem under Article 13 GDPR before it is anything else.

And then there is the regulation everyone will reach for as an objection: the proposed Child Sexual Abuse Regulation, known to its critics as Chat Control. At the sixth political trilogue on 29 September 2026, negotiators reached agreement on the tasks of the EU Centre - the intermediary that will check the validity of providers' reports before forwarding them to authorities - and, on some reporting, an agreement in principle on detection orders for public content. What they did not agree on was detection in private communications; the Parliament held its line there. So Europe is converging, unevenly and with the public-content question still contested, on the idea that an intermediary is acceptable and that scanning private correspondence is not. That distinction is the beginning of an answer.

A rule someone could actually write

What would a defensible system look like? Not a better policy at one company, but a rule that does not depend on any company's discretion. Five elements:

1. Define the content, as § 2258A does. "Imminent and credible" is not a definition; it is a standard that changed three times in eight months. The statute should say what conduct triggers a report (expressed intent, an identifiable target, acquisition of means) rather than leave it to a review team's reading of a vague phrase.
2. Report to an intermediary, not to the police. The provider sends a synthesis of the concerning conversation, with pseudonymised verbatim excerpts attached - the actual sentences, with names replaced by consistent placeholders, targets generalised by type (i.e. "a law-enforcement office in a Florida county"), times expressed relatively, and each excerpt hashed so a court can later verify it against the original. The provider keeps the key. The intermediary sees what was said, not who said it.
3. Let the intermediary assess the threat, not the imminence. The U.S. Secret Service's study of 41 targeted school attacks found that most attackers communicated their intent or threatened their target beforehand, and that in roughly two-thirds of cases someone observed concerning communication or behaviour and did not act. Its recommendation was not a sharper imminence test but multidisciplinary threat-assessment teams that evaluate behaviour over time.Tumbler Ridge is the argument for this element, and it is now a much stronger argument than it was a fortnight ago. The account was flagged roughly eight months before the attack. On a strict reading of "imminent," OpenAI's leadership may even have applied its own rule correctly. And we now know, if the federal charges are made out, that the planning they were looking at a fragment of ran for months across channels they could not see, between two people, one of them in another country. Imminence is the wrong test for intent that develops over months. A company is the wrong body to apply it. And a fragment is the wrong unit of analysis; which is precisely what a threat-assessment body exists to fix, by being the place where fragments from different sources arrive and are put next to each other.That invites an obvious objection to element 2. If every fragment arrives pseudonymised, with the key held by the platform that sent it, how does the intermediary know that two fragments concern the same person? It does not need to. Fragments can be joined on only two things: who wrote them, or what they describe. Any third criterion would splinter into a taxonomy of linkages for different circumstances that no one could administer. Joining on who requires an identifier shared across platforms - and that is the step that turns an intermediary into a fusion centre. So fragments are joined on the event: the same kind of target, the same place, the same window of time. "A secondary school in northern British Columbia, a weekday in February" is enough to put two fragments side by side without knowing whose they are. Identity, for each of them, still waits for a court.
4. Identity only by court order. If the intermediary concludes the threat is credible, it asks a court to order re-identification. Where time does not allow, the intermediary - not the provider - may act first, with judicial confirmation within a fixed short period. If the court refuses confirmation, the evidence obtained is inadmissible and the state, not the provider, answers to the person wrongly exposed. Otherwise judicial oversight is decoration.
5. No duty to monitor. Like § 2258A(f), the rule should say plainly that it creates no obligation to scan. Providers report what they become aware of; they are not deputised to look.

This is not Chat Control for violence, though that will be the first objection. A conversation with a model is not correspondence between two people, and the provider already processes every word of it; the question is not whether someone reads but what leaves the building and under whose authority. Under this design, what leaves is a pseudonymised description of a possible event. What stays is everything that identifies a person, until a judge says otherwise.

The question that remains

There is a harder version of this, and it is the one European AI companies will have to answer for themselves. If the policy debate ends where the Chat Control trilogue seems headed - intermediaries yes, general monitoring of private correspondence no - then the design question for anyone building a privacy-respecting assistant is how to recognise a possible crime without surveilling the conversation in which it appears. American providers have answered it by reading everything and deciding alone. That answer has now produced one arrest that most people will accept and one tragedy that 38 lawsuits are trying to attribute.

The complaint says one telephone call could have prevented Tumbler Ridge. A month ago that was a hard claim to assess. It is harder now. A call in June 2025 would have had to interrupt something that, on the federal charges, was only beginning to take shape - between two people who had known each other online for three years, mostly somewhere else - and it would have had to do so through a police force that already had a file open, had already visited the house, and had already taken the guns away.

Which is why the call was never the hard part. The hard part is who decides when it must be made, by what rule, with what check on the decision, and (the question Tumbler Ridge asks most insistently) who is responsible for putting the fragments together, given that no single company was ever going to hold more than one of them. For now, at both companies that matter most, the answer to the first of those is written into a contract: at our sole discretion. To the last, there is no answer at all.

GeoCop, at least, never claimed it had a duty. It just made the call.