Anatomy of a ChatGPT local answer

Someone asks ChatGPT which HVAC company in Austin is best. On screen, they get a map with a handful of businesses, paragraphs about them and, recently, an ad up top. It arrives as one answer, though six separate steps build what they see.

We spent the past month taking that answer apart. We first collected 44,988 answers through a scraper, then captured the raw payload delivered to a logged-in browser. We used both sources because their contents differ. The figure tags each part of the answer with the step that produced it.

Step 1

It rewrites your question

ChatGPT searches with a query it writes itself, using different words from the question you typed. The payload carries that query in a field called map_search_model_queries. We captured three on September 23, each beside the question that produced it.

"Who is the best roofer in Phoenix, AZ?" went out as "best roofers Phoenix AZ reviews roofing contractors." "Who is the best HVAC company in Austin, TX?" went out as "best HVAC companies Austin TX reviews licensed AC repair." "Who is the best plumber in Raleigh, NC?" went out as "best plumbers Raleigh NC reviews licensed plumber."

The word "reviews" appeared in all three queries, though nobody on our side typed it. "Licensed" appeared in two. ChatGPT read "best" as reviewed and licensed, then searched on those words. For your business, the query you have to match is the one ChatGPT wrote, and the person asking never sees it.

For the same prompts on the same day, the logged-in browser got the rewritten query on 95% of responses. The DataForSEO scraper returned nothing for that field on 95% of them.

Step 2

It looks up listings and searches the web

The map card comes from a places lookup. ChatGPT queries an index of business listings and gets records back, much as a maps app does. That's a separate task from reading web pages and summarizing them.

The places lookup and a normal web search travel in the same call. In all 100 captures from September 8 and 10 where we could see the call itself, ChatGPT sent the lookup and web search in a single request. One call from September 10 is shown as it was sent.

The lookup line contains one to four category phrases ChatGPT wrote itself, and some include words that weren't in the question. "Top rated" appeared in 12 of the 100 lookups, though the questions behind those lookups didn't contain it. The web search line also picks up added words. "review" appeared in 70 of the 111 web queries, while none of the 100 questions contained it.

In our census of 27,405 local answers, the card appeared in 95.4%. In a same-day check using a real browser, 57 of 60 prompts showed a card.

We can't name the supplier of the listings index. The business details in those rows closely track Google's listing data, but nothing we can capture identifies the provider.

Step 3

Each business comes back as 57 fields

Every business on the card comes back as a record with 57 fields. We checked 173 business records across 16 captures taken September 23, and all 173 had the same 57 fields.

Only eight of those fields are drawn on the card. For the same business, the DataForSEO scraper, which much of the industry measures with, returns nine fields.

The other 49 fields include whether the listing is claimed, its hours, review highlights, a quote-request link, and a ranking score. One field, provider_logo_url_short, holds a small image URL that records the source of the star rating. Among card rows with a Yelp logo, 15 of 16 were exact matches to Yelp's own rating and review count. No rows without that logo matched Yelp's own rating and review count exactly.

That field lets us identify each business's rating source. Yelp is the top rating source among businesses on the card, ahead of every other source, though its exact share changes with the list being counted.

Step 4

The records split into three lists

What looks like one card holds three distinct lists of businesses. The lists differ so much that a pooled figure describes none of them.

The scraper merges the map card with the pool behind Expand into one set, so all the figures we published about "the card" in our census describe the Expand pool, a separate list from the businesses on screen. As far as we can tell, the same is true of all figures others have published from that source. For our only direct check, we hand-counted one rendered page for a single prompt and saw five businesses before Expand. When we sorted the data from that capture, five businesses were in its map list.

Step 5

It writes the answer from other sources too

ChatGPT's prose paragraphs often name businesses that don't appear on the card at all. Among answers with a card, 70.4% name at least one business in the prose that isn't on that answer's own card.

Directories account for a large share of the sources behind those answers, and in our census, the most common domains behind local answers were angi.com, expertise.com, bbb.org, and reddit.com. Among answers with a card, directories made up 29.9% of cited sources, while business websites made up 56.5%. In answers where both retrieved and cited URLs were visible, directories accounted for 39.8% of retrieved URLs and 29.9% of cited URLs, so ChatGPT fetched directory pages it didn't use.

In the logged-in captures, search results and answer citations appear as separate lists that let us compare what ChatGPT retrieved with what it cited. Across 116 logged-in captures, about a fifth of retrieved pages were cited, and the median answer retrieved 12 distinct visible pages and cited 2.

Directories vary widely in how often they get cited. Thumbtack and Expertise.com were each retrieved in 45 answers. Expertise.com was cited in 24 of those answers and Thumbtack in 2.

Step 6

An ad can sit on top

On September 8, we captured 63 logged-in ChatGPT payloads and found no ads in any of them. On September 23, we used the same account, free tier, and method and saw an ad on 10 of 16 local answers.

The ChatGPT payload records an ad through a single field on the user's own message, the ad_delivered flag. The ad itself never appears in the captured data. The rest of the message metadata matches across both dates, so the September 8 zero wasn't a recording gap.

Hours later, DataForSEO's scraper ran the same 16 questions. It reported no ads on any of them.

If you track AI visibility through that scraper, ChatGPT ads on local answers appear to vanish in late August and stay near zero. In a logged-in browser, ads appeared on over half of the answers to those same questions.

Lead-generation marketplaces bought six of the 10 ads, three from Thumbtack and three from Angi. The other four came from businesses and brands, including an Austin HVAC company and a Raleigh plumber.

Only one of the 10 advertisers also appeared in the organic card beneath its own ad. Directories account for 29.9% of cited sources on these answers, and they're now buying the ad slot above them too.

Thumbtack bought three of the 10 ads. Across our three samples, totaling 116 logged-in captures, ChatGPT retrieved Thumbtack in 45 answers and cited it in two.

What we still can't tell you

Every row on the card has a ranking_score, but the card's display order doesn't follow it. The popularity_score field sits next to it and has been blank on every row we've captured so far. Neither field has documentation.

We can tell you the card exists, how often it appears, what arrives for each business, which of the three lists a number refers to, and where each star rating came from. We can't yet tell you what puts a business on the card or higher up it. Fields that might explain that are in the payload, and modeling them is our next project.