The Tool I've Been Sitting On (Sorry About That)

AI Readiness Check tests what crawlers, citation bots, and AI search tools can actually do with your site. Nearly 5,000 sites in, and it's free.

Most newsrooms haven't locked the door on AI crawlers. Most don't know there's a door. (AI-generated image).

A Quick Disclaimer: Backstory & Strategy is a personal, independent publication. The views, analysis, and commentary expressed here are strictly my own and do not represent the official position, strategy, or endorsement of the American Press Institute, its leadership, or its board. This is my personal space for analyzing the media landscape, testing new frameworks, and thinking out loud.

I’ll be honest, I’ve been holding out on you. While I’ve hinted that I created a tool that assesses AI readiness, the truth is, it’s been done for quite some time, but I have been having so much fun, I just plain forgot to launch it.

I think it’s truly one of a kind. As FT Strategies showed last week in their robots.txt analysis of 70 major publishers, you can piece parts of this together by hand — but that’s 70 sites, one file each, one snapshot in time. Nothing does this automatically, continuously, across your whole site, checking crawler access, citation rights, and your own protections all at once. It’s been tested on nearly 5,000 sites, so it’s solid. Ok, without any further ado, let me introduce AI Readiness Check. Put simply, it tests whether training crawlers, citation bots, and AI search tools can access, cite, and profit from your site — and whether your own policies protect you if they do. Best of all, it’s built for publishers, editors, and press associations, and this is a big one: no developer or advanced degree is required to read the report.

---

If this is the kind of thing you want in your inbox before your competitors have it, subscribe. Backstory & Strategy is free, no catch, same as the tool.

---

Here’s the thing most newsrooms get wrong: they think “AI and my site” is one question. It’s actually four, and they don’t have the same answer.

• Can training crawlers reach you? That’s the bots scraping your archive to build the next model.

• Can citation bots see you clearly enough to cite you accurately when someone asks an AI chatbot a question your reporting answered first?

• Can AI search tools summarize you in a way that still sends a reader to your site, or does it just strip-mine the answer and let them leave satisfied?

• Do your policies back you up if any of that happens? Does your robots.txt, your licensing terms, your AI policy actually protect you, or are you standing there with the door wide open and no sign on it either way?

AI Readiness Check answers all four, for any site, in a report a non-technical newsroom leader can actually read.

I’ve run this across nearly 5,000 sites at this point, and a few things keep showing up no matter who I’m looking at. The most striking one: formal TDM (text and data mining) opt-out, the mechanism recognized under EU copyright law and reinforced by the AI Act for telling AI companies “don’t train on this,” sits at effectively zero adoption. Not rare. Zero. Most newsrooms aren’t choosing to leave the door open. They don’t know there’s a door.

And it’s not a resource problem. I’ve seen small weeklies with tighter AI protection than national chains, because the fix is often a text file, not a redesign. What separates the protected sites from the exposed ones usually isn’t budget or a developer. It’s whether anyone ever asked the question.

This isn’t a new toy I’m testing on you. I’ve already been running it quietly behind the scenes. It’s what let me write about public media’s cloud infrastructure with actual numbers instead of guesses, and what powered the piece on who really controls an independent journalist’s AI exposure once they’ve built their audience on Substack, Ghost, or Beehiiv instead of their own domain. The tool came first. The pieces just happened to be where I showed my work.

Here’s why it’s free, and I want to be straight with you about the reasoning instead of just saying “because I felt like it.”

I think AI is a real threat to journalism. Not the kind where newsrooms wake up one day and the staff is gone, replaced by a script. I don’t believe that’s where this goes. And I’m not in the camp that thinks this ends civilization as we know it either. But there’s a real, specific harm sitting in the space between those two extremes, and it’s the harm that happens when an industry just sits back and lets it happen. Waits for someone else to figure it out. Assumes it’ll sort itself out the way these things sometimes do.

I don’t think it sorts itself out. I think it gets baked in, quietly, one uninformed robots.txt at a time, and five years from now nobody remembers there was ever a choice to make.

So for now, AI Readiness Check is free. Quick registration, full report, no card required.

Now, I’ll be honest about the other side of that too: free isn’t a business model, and running these scans costs real money. I know that. I’m comfortable delaying that question. This is a moment where I’d rather get the data into as many newsroom leaders’ hands as possible than optimize for revenue on day one. If that changes down the road, you’ll hear about it here first, same as everything else.

So what does running your own site through this actually look like? Enter your URL, wait a minute or two while it works, and you get back something built to make sense on a first read, not something you need a developer to translate for you.

At the top: two scores, not one. That split matters more than it sounds like it should. Visibility measures how findable and citable you are, whether the AI tools people are increasingly asking questions through can actually see and credit your work. Protection measures the opposite instinct, how well your policies and technical setup guard against being scraped and repackaged without consent or compensation. Most sites are lopsided one way or the other. A handful are strong on both. A lot are weak on both and don’t know it.

Below that, the report breaks down exactly who’s knocking. Fifteen separate training crawlers, tracked individually, not lumped into one generic “AI bot” line. ClaudeBot and the older anthropic-ai user agent get counted separately, because they’re genuinely different crawlers and treating them as one undercounts what’s actually happening on your site. Same with the citation and search crawlers, the ones behind tools people are using right now to ask a question and get an answer without ever clicking through to a newsroom.

Then it goes granular in a way most sites have never had spelled out for them: robots.txt, present or not. llms.txt, the newer file some sites use to set AI-specific terms. A formal AI policy, publicly stated or missing. And TDM opt-out, the actual mechanism for saying “don’t train on this,” which across almost 5,000 sites I’ve now run, is sitting at effectively zero adoption industry-wide. If you have it, you’re already ahead of nearly everyone.

One more thing worth knowing before you run yours: the report also catches newer signals, like Cloudflare’s AI-permission settings, that some sites have without even realizing it, and it’s honest about the difference between “we found a blanket site-wide setting” and “we found this crawler named explicitly.” That distinction sounds small. It isn’t. A lot of sites think they’re protected because of one general setting when the reality underneath is messier.

None of it requires you to know what a crawler is going in. That’s the point.

A quick note on the numbers themselves, because a 30 and a 70 mean genuinely different things, and I don’t want you squinting at your report wondering if you should panic.

Composite scores across almost 5,000 sites average in the mid-30s. That’s not a typo and it’s not your site being uniquely bad. Most of the industry is sitting somewhere in “hasn’t really addressed this yet” territory, which is exactly the point of running this in the first place. A low score isn’t an indictment. It’s information you didn’t have yesterday.

What’s more useful than the composite number is which half is dragging it down. High Protection, low Visibility means you’ve locked things down tight, maybe too tight, and you might be invisible to legitimate citation tools you’d actually want crediting you. Low Protection, high Visibility is the opposite problem: wide open, easy to find, easy to scrape, with nothing in place saying otherwise. And if both numbers are low, that’s usually not a strategic choice. That’s just nobody having looked yet, which describes most of the sites I’ve run.

The report tells you which one you are and, in plain language, what the actual gap is. Not “improve your AI governance.” The specific missing file, the specific policy gap, the specific crawler that’s currently walking right in.

I built AI Readiness Check because I kept writing pieces about AI and journalism and getting tired of guessing. I wanted the data, not the vibes. Now I’ve got it, on almost 5,000 sites and counting, and starting today, you can get it on yours too.

It’s free. Enter your domain, give it sixty seconds, and get the breakdown. Whatever your score comes back, you’ll know something concrete about your site that you didn’t know this morning, which beats the alternative most newsrooms are currently running on: hoping it’s fine.

Once your report generates, hit reply and tell me what surprised you. I’m tracking patterns across this next wave of scans, and the most compelling findings might end up in next week’s dispatch.