A Quick Disclaimer: Backstory & Strategy is a personal, independent publication. The views, analysis, and commentary expressed here are strictly my own and do not represent the official position, strategy, or endorsement of the American Press Institute, its leadership, or its board. This is my personal space for analyzing the media landscape, testing new frameworks, and thinking out loud.
I left a legacy newsroom once before. Decades before Substack existed, I walked away from what was then the Gannett-owned The Courier-Post in Cherry Hill, New Jersey, to start my own print publication. My byline. My presses. My rules. Except it wasn’t, not entirely. I still answered to a printer, a distributor, an ad rep. Independence just meant trading one set of landlords for another.
At least those landlords were visible. I could see the printer. I could call the distributor.
Substack has an AI-training toggle that defaults to off, meaning on. Write there and you’re opted in until you go find the switch yourself.
Flipping it does something real. It adds a disallow instruction to your robots.txt file, the signal crawlers are supposed to check before scraping. Supposed to is doing a lot of work in that sentence, because the signal isn’t self-enforcing and some AI crawlers ignore it outright.
Nobody knows how many writers have found that switch, or what the same question looks like on the other platforms journalists write on now. So this summer I ran an AI-readiness audit across Substack, Ghost, and Beehiiv, checking robots.txt, llms.txt, public AI policies, and the TDM reservation standard across more than 700 independent publications.
Across 616 independent Substack newsletters, 95.3 percent carry zero AI-crawler protection. That’s not most. That’s nearly all.
Zero of the 616 had adopted the TDM reservation standard. Not a handful. Zero.
Independence is the entire pitch of these platforms. Substack, Ghost, and Beehiiv all sell the same promise. Leave the corporate newsroom. Own your relationship with your readers. Answer to no one.
Most of the writers in this data made that trade to get more control, not less. But on one question, who decides what an AI company can do with your work, most of them ended up with less control than the reporter they used to sit next to in a newsroom, whose employer at least had lawyers making that call on purpose.
Each of the three platforms gets you there a different way.
Substack: the checkbox almost nobody gets handed
Split the writers by domain type and the picture sharpens. Five hundred and sixty-six of them run on Substack’s own subdomain, and among that group, llms.txt adoption isn’t low, it’s zero, because Substack doesn’t let a native subdomain host a file at its own root. That’s a real technical wall. Nobody’s choosing not to do this. They can’t.
A public AI policy page is a different story. Nothing stops a writer on a native subdomain from publishing a plain page explaining how AI training is handled and by whom. It’s just content. No file hosting, no custom domain, no technical lift at all.
And yet the adoption rate among those same 566 writers is also zero. The platform isn’t blocking this one. As far as the data shows, nobody’s written one.
The forty-nine writers running custom domains are the only place either shows up at all, and they reach full protection at roughly five times the rate of everyone else, 16.3 percent versus 3.5 percent. Names you’d recognize sit in that minority. Nate Silver’s site. Dropsite News. This newsletter is one of eight, out of forty-nine scored, sitting in that fully-protected group.
But a custom domain is necessary, not sufficient. Emily Atkin’s HEATED also runs on its own domain and still shows zero protection. Sit with that for a second. Paying for the option doesn’t protect you. Knowing the second, unmarked step exists does, the part where you go find Cloudflare’s separate AI-bot toggle and turn it on yourself.
When I left the Courier-Post, the infrastructure standing between me and full control was visible. A printer. A distributor. An ad rep. People I could call.
The infrastructure standing between a newsletter writer and their own AI exposure is invisible by design. It’s buried in a settings panel nobody thinks to open, because nothing on the writing side of the job points you there.
I didn’t think of my own setup as a decision when I made it. I thought of it as a checkbox. It’s only next to six hundred other writers that it looks like what it actually was, one of the only real choices available to anyone in this dataset, and one that required money for a domain, a vendor most writers have never heard of, and the accident of already knowing to go looking.
Ghost: protected on paper, open in practice
Substack’s failure is absence. Ghost’s failure is stranger. Arguably worse.
I checked forty-one Ghost-hosted independent newsrooms. Small, hyperlocal outlets. The kind with a masthead and a handful of reporters, not a corporate parent.
Every one of them has a robots.txt file. Every one has an llms.txt file. Every one has a page describing its AI policy. On paper, that’s a newsroom that did the work Substack’s writers never got the chance to do.
None of it blocks anything. Protection score is zero. Same number as Substack’s worst cases, reached from the opposite direction.
This isn’t nothing offered. It’s something offered that doesn’t work, generated automatically, identically, on every site, by the platform itself. A newsroom that pulls up its own AI policy page sees a real document, and would reasonably conclude the question was handled. Nothing on that page says otherwise.
One site in the batch broke the pattern. swvacountry.com had a real, configured block, and a composite score of 74 that nothing else in the cluster comes close to. That’s the exception that proves the rest is a platform default, not a policy. Someone can override it. Almost nobody has.
That’s a different version of the same root problem, and in some ways a more troubling one. The writer isn’t making a decision either way. They’re just further from noticing it.
Beehiiv: the pro-creator pitch, checked against what’s actually live
Beehiiv has spent the past couple of months positioning itself as the newsletter platform that takes this more seriously than Substack does. In June, Cloudflare and Beehiiv announced a partnership giving creators one-click controls to manage AI bots. Trade coverage at the time framed it explicitly as a contrast with Substack’s opt-out-by-default model. It’s a real feature, and a fair pitch. I wanted to know how many creators had actually turned it on.
I checked fifty-eight Beehiiv-hosted newsletters. Zero of them reached full protection. Not one.
Protection scores in this batch only ever land on one of two numbers, zero or seven out of a hundred. That seven isn’t partial credit for blocking anything. It’s the baseline score for simply having a robots.txt file present, and every one of the thirty-six sites at seven has one, while almost none of the twenty-two at zero do. Whether that file actually names or disallows a single AI crawler is a separate question. Mostly, it doesn’t.
The files themselves are more common here than on either other platform. A robots.txt on 69 percent of sites, an llms.txt on 60 percent, both well ahead of Substack’s numbers. None of it moves the needle. Seven out of a hundred is still wide open.
This is what separates Beehiiv from Substack. A custom domain was the one lever that worked there, unlocking real protection for a motivated minority. On Beehiiv, it isn’t. Twenty-seven of these fifty-eight writers run their newsletter on its own domain, the exact move that helped on Substack, and not one broke past that same seven-point ceiling.
The announced capability may well be real, and worth using. What this batch shows is that, as of this month, almost nobody has found it.
The shape that doesn’t change
Line the three platforms up and each one fails a different way. Substack withholds the option entirely, except for a narrow, hard-won escape hatch a small minority find their way to. Ghost hands out the appearance of protection to everyone, real or not, with no way for a newsroom to tell the difference from the inside. Beehiiv built the real tool and shipped it. Almost nobody has found it yet.
Across all three platforms, that’s over 700 independent creators checked in one summer, more than any published study of newsletter platforms I could find, and enough to say this pattern isn’t a fluke of any one platform’s userbase.
Three platforms. Three different failure modes. The same outcome underneath all of them. A writer’s actual say over their own AI exposure has nothing to do with how much they care about the issue. It has almost everything to do with which platform they happened to pick, and whether they’ve found the one setting that matters.
A few places to actually look
None of this requires a developer. It requires knowing which setting to look for. None of these platforms make that obvious.
On Substack: the AI-training toggle lives in Settings → Privacy, and it’s off by default. Go turn it on. You can also add a plain-language AI policy page today, regardless of your domain, since that’s just a page of content and nothing about a native subdomain stops you from writing one. Almost nobody has. llms.txt is different. It’s a root file, and Substack’s native subdomains genuinely can’t host one. Reaching real crawler blocking means moving to a custom domain and layering Cloudflare’s AI-bot controls on top.
On Ghost: don’t take your own AI policy page at its word. Pull up yoursite.com/robots.txt directly and check whether it actually names crawlers like GPTBot, ClaudeBot, or CCBot, and disallows them. A generated policy page and a real block are two different things. On this platform, having one doesn’t mean you have the other.
On Beehiiv: the Cloudflare AI-bot controls the company announced this summer appear to be real. Almost nobody in this batch had turned them on. If you’re on Beehiiv, this is probably the single highest-leverage five minutes available to you.
None of this is a guarantee. robots.txt is a request, not a lock, and not every crawler honors it. But a request costs you nothing, and right now, most independent writers aren’t even making one.
When I left the Courier-Post, the landlord I traded up to was visible. I could see the printer. I could call the distributor.
The landlord an independent writer answers to today doesn’t return calls. There’s no one to call. Just a default, set once, upstream, before the writer ever showed up.
Independence used to mean choosing your own infrastructure. For most people making that choice today, it means something closer to the opposite. They leave to get free of someone else’s decisions. They land inside a new set they never knew they’d agreed to.
Related: The First AI Election and the Infrastructure Nobody Built
💬 If you write on one of these platforms, what did you find when you checked your own settings? Tell me in the comments — I’ll read every one.
📬 If this was useful, the best way to get the next one is to subscribe. It’s free, and I’d rather you hear it from me than find it reshared out of context.
🔁 Know a newsletter writer, editor, or platform person who should see this? Send it their way.
I'm on beehiiv (www.courtwatch.news) Do you know if turning on the controls would affect search results visibility, AI search result visibility and ability to be picked up on Google News? That was my hesitation on turning it on.