How to run AI visibility tracking

AI visibility tracking, also called LLM tracking, means repeatedly asking AI tools such as ChatGPT, Gemini, and Perplexity the questions your customers would ask, then tallying how often they name your business. We call that tally visibility, and we measure it across repeated runs and several phrasings of one question, always reporting it with a band that shows how far it swings on its own. Starting in June 2026, Complete SEO's tracker asked each question up to five times per engine in a single scan, and our own repeat study in August found those runs captured one sitting while the figure you care about shifts from one sitting to the next. In September, we rebuilt the tracker around one run of each question per engine per day, phrasing clusters, and a noise band.

Why one check can't tell you much

One check says little about whether ChatGPT will name your business, since asking the same question again seldom returns an identical list. In August 2026, our three-day repeat study collected 12,120 ChatGPT answers to 46 questions through DataForSEO, a commercial scraper of the ChatGPT app, and found that across 188,520 pairs of runs minutes apart, the same local question returned an identical set of businesses only 3.6% of the time. Across the 40 local questions out of 46, about 28 businesses per question appeared in at least 5% of runs (the median appeared in 30%), while only 36 of the 1,115 business-question pairs at that 5%-plus level appeared in at least 95% of runs. For businesses named roughly half the time, we grouped their runs into 6,624 blocks of five consecutive runs and found that a business appeared one to four times in 89% of blocks, zero times in 6%, and all five times in 5%.

Between sittings 12 or 24 hours apart, a business's uncorrected rate over 40 runs moved at least 10 points in 40% of 5,575 consecutive-batch changes across the same 1,115 business-question pairs. For two scans of the same untouched questions to exceed that between-sitting movement, the gap needed to be roughly 28 points, or 28.3 after correcting for extraction errors, and more runs per scan didn't lower that threshold. Part of that movement came from ChatGPT switching the response format it served the scraper on the study's first afternoon, and across batches after the switch, the threshold for two scans was still about 14 points. The study covered 72 midweek hours, so each figure is a floor for how far a month can move.

A 2026 paper found the same instability in the sources three AI search engines cited for consumer-product questions. Its conclusion was that "single-run visibility metrics provide a misleadingly precise picture of domain performance in generative search." Our guide explains why the answers change at all.

What our old tracker got wrong

Our old tracker ran each question up to five times per engine in one scan during a single sitting, and those runs mostly captured that sitting. The repeat study found that runs within a sitting acted as independent samples around that sitting's rate, so repeats sharpened that reading but said little about the next sitting. Businesses appeared in 5% to 20% of runs most often, and a five-run rate for them couldn't detect a change under about 40 points. Because the answer itself kept moving, more repeats stopped helping after about 65 runs for those businesses and after about 10 runs for a business named 50% to 80% of the time.

Adding phrasings to the same scan doesn't help against drift that moves every phrasing together, as during the format switch on the study's opening afternoon. If drift is shared that way, the study's main reading found that no number of phrasings in one sitting could detect a 10-point change, while its pre-registered reading found that detecting the change took a cluster of at least 21 phrasings. Our dashboard headline also called the whole-chart change up or down with no band around it. On Sept. 13, 2026, we switched three pilot clients to daily single runs, one per phrasing, per engine, per day, and most of our other clients followed on Sept. 22.

What to track

Track clusters of phrasings, with one cluster for each service and place pairing that produces real jobs. A cluster captures one thing a customer wants, such as a plumber for a slow drain, in several phrasings that you score as a single number. Its location is the town named in those phrasings.

One phrasing measures only one door into the answer, and our census of 44,988 local answers found that changing only the wording produced averages of 3.4 businesses named per answer for "list three," 5.8 for the plain question, and 10.6 for a paragraph-length request seeking a locally owned company, based on 3,915 answers per phrasing. In a ChatGPT pilot from Sept. 13 to 27, 2026, the phrasings in one client's cluster were asked once daily, and the client was named in 12 of 15 answers to the locally owned phrasing and 2 or 3 answers to each of the other five. Across that cluster, it was named in 23 of 89 answers, while another cluster for that client named it in all 87 answers over the same 15 days. Follow any one of the six phrasings, and you could read that same client's visibility for the same service as anywhere from 13% to 80%.

In the pilot, we use four to six phrasings per cluster, and the right count will come from the bands we're measuring. The phrasings are spread across the doors that change who ChatGPT names:

• Terse search, such as "[trade] [town] [state]"
• Plain question, "Who's the best [trade] in [town]?"
• A few first-person sentences describing a real job and asking who to call, with "best" left out
• A request for locally owned companies
• "List three [trade] companies in [town]."
• The specific service, if the cluster covers one

Keep one town in each phrasing by splitting any question that covers two towns into two phrasings. Leave out your business name and competitors' names, since a business named in the question starts ahead in the answer. Once a phrasing has run, keep its wording fixed because a reworded phrasing starts a new series. Log the date whenever you change the set so the break is visible on the chart.

Put the town in the words because trackers carry no location. DataForSEO sends every question from the United States as a whole, while an engine's API receives only the question's text. A real customer asking for a plumber "near me" may get an answer tailored to their location. OpenAI says ChatGPT "may use an approximate location based on your IP address to provide relevant local results."

Build the list from the outside in. In one client test, the owner's list, a read of the website, and a look at Search Console each turned up real questions the other two missed. Keep informational questions, such as repair cost, in their own cluster. They usually get zero businesses named, so that cluster measures citation of your page.

Which engines to track, and through what

Track the engines your customers use on separate lines, and record whether each number comes from the engine's app, a scraper of the app, or the engine's API. We track ChatGPT and Gemini through DataForSEO's scrapes of their consumer apps. We track Perplexity through its API because DataForSEO doesn't scrape it, and Claude through its API weekly on a separate line. The engines disagree about local businesses, as our comparison of how each engine searches shows, so one blended score hides which engine you're absent from.

The route changes the number, as we found in two weekly sweeps in late July and early August 2026 when we sent identical questions through OpenAI's API using a smaller model and through a ChatGPT-app scraper. Across 309 matched pairs, the app side named the business we were tracking in 143, while the API named it in 55. OpenAI's docs describe API web search as a developer tool with configurable settings, including whether it "fetches live content or uses only cached/indexed results" when it searches. We don't treat the API as a proxy for the app.

How accurate are AI visibility tracking tools?

Accuracy depends on how a tool reaches an engine, and on ChatGPT a scraper mostly identifies which businesses appear in the card, ChatGPT's map-pinned shortlist, while missing much of what surrounds it. A signed-in browser session receives all the data a customer's browser gets, even data the screen doesn't show. A commercial ChatGPT-app scraper such as DataForSEO returns a structured copy of the answer, and much of the industry measures with it, as does our tracker. The API offers a third route to the model's developer-facing version, with settings of its own, and we handle it as a distinct product from the app.

We compared the browser and scraper routes. On Sept. 8, 2026, we asked 60 identical local questions through a signed-in ChatGPT session and DataForSEO, with three scraper runs per question.

The field count comes from 173 business records captured on Sept. 23. The ads count comes from 16 questions asked that same day, with the scraper pass roughly five and a half hours after the browser pass. Among the fields the scraper strips from the record the app receives are the star-rating source, claimed-listing flag, link to request a quote, and ranking score. The scraper also puts the businesses shown on the card and those behind Expand into one list of about ten, while a customer sees only the first handful before opening Expand, averaging slightly under four per card across 57 cards from the Sept. 8 logged-in answers that carried a card.

Deciding whether a business was named adds another source of error. Against a 150-answer reference set drawn from our census, DataForSEO's built-in business-name field caught 43.1% of businesses, while our extractor caught 92.6% with 94.1% precision. For one pilot client tracked over 30 days through Sept. 18, the business-name field registered 128 of the 241 ChatGPT answers naming the client, and all 113 misses named the client in the answer text. Our tracker counts a business as named through a text mention, an alias match, or a citation of its website, and reports card presence as a separate number.

Before you trust any tracker's numbers, including ours at Complete SEO, ask:

• Does each engine's number come from the app, a scraper, or an API?
• How many separate days stand behind a number, and how many answers?
• Does it show a band, and do changes within that band read as no change?
• What's its rule for counting a business as named, and what's its miss rate?
• Does "in the card" mean the few visible on screen or the full pool behind Expand?
• Is rank or average position among its reported numbers?

How often to run it

Run every phrasing on every engine once a day, then read the results across a rolling 30-day window. Each day is a fresh sitting, so daily runs give you 30 sittings in that window, while weekly five-run scans give you four. Only more sittings shrink the drift's share of the band.

If you need to ask fewer questions each day, you can rotate a cluster's phrasings and run one per day. Over 30 days, that gives you roughly 30 answers per engine for the cluster, compared with 150 to 180 when you run every phrasing. We estimate the resulting band is roughly 2 to 2.5 times wider, and the pilot is testing that estimate.

What a report should show

Your report should show visibility for the window and its band, with a change displayed only when it exceeds that band. We calculate window visibility by dividing the answers that named you by all answers in the window, since averaging daily percentages would give a light day a full day's weight. We compare it with the previous window of equal length and draw an in-band change in gray, since the data can't separate it from answers shifting on their own. When fewer than half the prior window's days have data, we suppress the change.

The band adds together two kinds of noise, sampling error from the window's answers and between-day drift divided by the number of days.¹ For one pilot client in the 30 days to Sept. 18, visibility was 56.2% of 949 answers from 9 days with data, not 30, and a change against the prior 30 days would have had to clear about 10 points (9.92) to count. Dividing drift by days assumes each day's drift is independent, and a platform change lasting weeks breaks that assumption, which is part of why we're measuring our band from our own daily numbers.

For a local business, also report how often you appear in ChatGPT's card of local listings, plus your listing's star rating and review total against the median of 4.9 stars and 28 reviews across the 276,628 card listings in our census. Share of voice is a separate number measuring your cut of all business names appearing in answers, with every business the engines mention in the denominator, and a report should never present it as visibility.

Some numbers shouldn't be goals at all. Rank or average position, which an AI rank tracker leads with, has no stable measure when the same business set repeats only 3.6% of the time, a one-day change reflects a single sitting, and total share of voice or one composite score erases the per-question, per-engine gaps you'd act on.

How to read the gaps

Read a low number by identifying which kind of absence it is, since each kind points to a different fix. These are the questions we ask of every client's data when an AI search project with us begins:

• Does the engine know you? Start a new chat and ask about your business by name. If it can describe you but never names you for your trade, you're known and passed over. If it can't describe you, you're unknown to it. This is a sorting step, since no published source confirms the cause.
• Which service, door, or town has the gap? Totals can hide a hole. In June 2026, one early client appeared in 0 of 12 answers in one town and 6 of 12 in another. For a service, missing means nobody owns it yet, while displaced means a competitor owns it. A missing service is easier to win than a displaced one.
• Is another business getting credit for something you do? Ask "who around here [does the thing you're known for]?" many times, then tally which businesses get the credit.
• Are you named while another site gets cited? Your standing then rests on pages outside your control. The sites cited in your category give you a to-do list of where to get listed and what page to build.
• Is it a memory gap or a live-search gap? Run the question with search off, then with it on, to help tell them apart. In 95.5% of 3,303 ChatGPT local answers, the card still appeared with the scraper's web search setting off, so "off" doesn't mean pure memory there.

How our dashboard does it

Our dashboard stores daily counts by phrasing and engine, adding rows to build any 7-, 30-, or 90-day window. The tracking view starts with visibility pooled across the three daily engines, then shows one tile per engine and a Claude tile marked weekly. Each tile's change against the previous window goes gray inside the band. The chart has a trailing 7-day line, a shaded band, and event markers, including Aug. 5, 2026, when we started scraping ChatGPT, and Aug. 24, when ChatGPT changed its format.

Each cluster row opens to show its phrasings as counts, with 18 of 30 as an example of the format. A cell shows a change only when it clears the band. Local clients also get the card view, and we're rebuilding the monthly client report to compare each calendar month with the preceding month using the band. Clients of our AI search work see this view.

Methods

The numbers on this page rest on our own data. Each note says where one set of them comes from.

• Repeat study. 12,120 ChatGPT answers to 46 questions, 40 of them local, collected through DataForSEO's ChatGPT scraper in seven bursts 12 hours apart from Aug. 24 to 27, 2026. Pre-registered. A business enters a question's pool when it appears in at least 5% of that question's answers. Every across-day figure is a floor on month-to-month movement.
• Census. The pre-registered analysis of 44,988 answers collected Sept. 3 to 7, 2026 through DataForSEO's ChatGPT scraper. The web-search-off arm is 3,303 answers to the plain question with the scraper's web search setting off, and each phrasing comparison uses 3,915 answers on matched trade and metro pairs. Named-business counts use our extractor, with 92.6% recall and 94.1% precision on 150 answers coded blind by two AI coders and read a third time.
• Scraper comparison. The same 60 local questions asked on Sept. 8, 2026 in a logged-in free ChatGPT account with memory off and through DataForSEO, three scraper runs per question. Audited Sept. 8 to 10, 2026.
• Logged-in captures. The 57 fields per business come from 173 business records in 16 conversations captured from a logged-in free ChatGPT account with memory off on Sept. 23, 2026. The ad count compares those 16 questions with the scraper's answers to them.
• Our tracker. Pilot and client figures come from our own tracker and were rebuilt from our database on Sept. 25, 27, and 28, 2026. The 30-day windows to Sept. 18 include weekly runs from before daily tracking began on Sept. 13, and the cluster comparison uses daily runs from Sept. 13 to 27. The app and API comparison comes from our late July and early August sweeps, and the town comparison from one client's June 2026 scan.
• Derived figures. The 13% to 80% range is our arithmetic on the pilot counts. The 15 to 18 point and 10-point thresholds use the change formula in the note below, with the repeat study's figure for movement between sittings until we've measured our own.
• OpenAI's documentation. Checked against its help center and its developer documentation on Sept. 25, 2026.