Why We Started With Our Own Sites
We are building a service around AI visibility. Before taking it to any client, we wanted to run the method on ourselves — both to see whether it works and to know our own score.
We measured four sites we run: three education sites and this site itself. The results were worse than we expected, and most of them are our own fault. We are publishing them anyway, because the numbers show the method works.
How We Measured
For each site we defined and froze 8 industry-relevant questions. Each question ran three times, because AI answers vary between runs. Measurement covered two modes: what the model knows without searching (“memory”) and what it answers while reading the live web (“search”).
192 observations in total. The raw text of every answer and the sources it cited were recorded. Zero errors.
Two disclosures up front, because you need them to read the numbers correctly:
- All 192 observations were run on a single AI engine. Differences between engines are a separate matter; we come back to it below.
- One of the 8 questions is a brand question — we literally ask “what do you know about this site?”. Since the brand name is inside the question, it is almost guaranteed to appear in the answer. So the table below gives each score twice: including the brand question, and excluding it. The second column is the one that matters.
Result: All Four Are Invisible
| Site | Visibility (search mode) | Excluding the brand question | Cited as a source | Own domain’s rank |
|---|---|---|---|---|
| Exam-focused education site | 21% | 10% | 42% | 4th of 52 sources |
| Middle-school education site | 17% | 5% | 33% | 7th of 70 sources |
| Online tutoring site | 12% | 0% | 12% | 15th of 44 sources |
| This site (software) | 12% | 0% | 12% | 24th of 114 sources |
The second column is the painful one. Once we stop asking about ourselves by name, two of the four sites appear in no answer at all — zero. The other two hover around one in ten. So it is not that the AI knows us; it is that it repeats our name when we put our name in the question.
Our exam-focused site ranks in Google’s top three for six of the 28 keywords we track, and first for one of them. It is still absent from 79% of the AI answers. That was the clearest evidence we found that Google rankings do not translate into AI visibility on their own.
In memory mode all four scored in the 8–12% band — but all of that score comes from the brand question. Remove it and all four sites drop to zero in memory mode. The models do not recognise these brands from training data; whatever visibility exists comes entirely from live search.
Marketplaces Eat the Answers
Across the questions for our three education sites, the brands named in the answers broke down like this:
- A services marketplace: 41%
- An international tutoring platform: 22%
- Us (all three sites combined): 11%
This looks almost identical to the Google picture, with one difference: on Google you can still appear second or third below the marketplace. In an AI answer there is no such room. Three names are given, and that is the end of it.
Two Findings We Did Not Expect
1. Our sites cannibalise each other. In the measurement for our online tutoring site, one of the most-cited sources was another site of ours, with 14 citations. The site being measured appeared 15th, with 3. The AI picks one of our two sites to answer that question — and not the one we expected.
2. The real competitor list is not the one we knew. On the software side, the sites the AI reads included names that were not on our competitor list at all. We did not know who was actually being read in our own sector.
Engines Do Not Agree
All 192 observations above ran on one engine, so this measurement says nothing about differences between engines. But in a separate, far smaller test — our free scan tool, 3 questions, no repeats — we asked three engines about the same business: two returned 100%, one returned 67%. The engines also recommended different companies; one put a business at the top of its list that another never mentioned.
That is a signal, not a measurement — three questions with no repeats do not count as measurement, by our own rule. But it is enough to raise the question: “am I visible in AI” has no single answer, and you have to ask which AI. Measuring the gap properly means running the same frozen panel, with repeats, on every engine. That is our next job.
One More Thing We Saw
In one run, the model added this about a business it had just recommended:
“Patient reviews: for example, there are some complaints about [business name] — people have reported issues such as inattentiveness and appointment scheduling.”
(Quoted verbatim from the raw record, translated; we removed the business name.)
The AI is volunteering that complaint to a potential customer — and it did so when asked about the sector, not about that company. We doubt the business knows. This showed us that visibility measurement is not only about whether you were mentioned.
What We Learned
- A single query is not measurement. The same question gives different answers on the same day.
- Do not measure by asking about yourself. The brand question inflates the score. The real question is whether you appear when a customer asks about your sector without knowing you. For two of our four sites, the answer was a flat no.
- Google ranking is no guarantee. Our top-three-ranking site was absent from 79% of answers.
- The source list is the to-do list. Seeing which sites the AI reads tells you what to work on.
- Fix your own house first. Our next job is repairing our own sites and measuring again in two weeks. We will publish that result too — good or bad.
If You Want to Measure Your Own Site
We opened a small version of the same method to everyone: a free AI visibility scan. Enter your domain, let the questions run, and see the actual answer along with whether your brand appears. No signup required.
The method behind the measurement above, step by step: How to Measure AI Visibility.
If you want the method and the GEO/AEO concepts behind it: our AI visibility service.
