Back in September, I wrote about why some of your reports were running slower than usual, and what was driving it.: Google’s growing effort to block automated tools, including rank trackers like ours, from collecting search data at scale.
That problem hasn’t gone away. In the past few weeks, Google has added a new tactic, and it’s a trickier one to identify and handle. I want to explain what’s happening, what it means for your rank tracking reports, and what we’re doing about it.
Google’s latest anti-scraping tactic is “poisoned search results“
Until recently, Google’s main defense against scraping was to block requests outright. This meant more captchas, more hard blocks, and the removal of the ability to pull 100 results per page. These changes made collecting full result sets slower and more expensive for every rank tracking tool on the market.
The newest measure works differently. Instead of blocking the request, Google sometimes serves a complete, normal-ish looking results page where many of the results have been swapped out for irrelevant results. These pages often return social media profiles and forum threads in place of the actual ranking websites. The pages look legitimate, but they’re not accurate.
From our observations, this doesn’t affect page 1. It shows up on page 2 and beyond, which is why you’re most likely to notice it below position 10.
Additional research and coverage of this issue
Nozzle.io published some research on this on October 10, analyzing 27.5 million search result pages across 49 countries. It found that:
- 8.0% of pages were affected since mid-September, up from a baseline of 0.17% before the change,
- The worst days saw it reaching between 25% and 27% of pages betweenon the worst days so far, September 27 to 29.
Search Engine Journal covered the same report’s findings, confirming the pattern and the scale of this change by Google and its impact.
What this means for your BrightLocal reports
If you’re seeing rankings that look different from one report to the next, particularly for positions 10 and below, this is almost certainly what you’re seeing. Here’s how it breaks down by result type:
| Result type | Impact |
|---|---|
| Google Desktop results | Significantly affected |
| Google Mobile results | Significantly affected |
| Google Map Pack and Local Finder results | Less affected |
| Google Maps results | Not affected |
Here’s how that breaks down by BrightLocal report:
| Report | Impact |
|---|---|
| Local Rank Tracker | Affected |
| Local AI Visibility | Partially affected |
| Local Search Audit | Partially affected |
| Local Search Grid | Not affected |
| Citation Tracker | Not affected |
| Google Business Profile Audit | Not affected |
| Reputation Manager | Not affected |
This is an industry-wide problem affecting all rank trackers and SERP scrapers
This is a problem affecting the whole SERP scraping and rank tracking industry, including large players like SEMrush and Ahrefs, as well as the data providers much of the industry relies on behind the scenes, like DataForSEO and SerpApi. If you use more than one rank tracking tool, you may see the same inconsistency across all of them.
What we’re doing about it
This is our top priority to resolve, and we’re putting significant resources into solving this by improving our data collection methods and working directly with our data providers so that fixes land faster. We have increased the number of data providers we use to make sure we have failsafes if there is an issue with one.
There’s no single fix that solves it completely, not for us or for anyone else in the industry, but we’re not stepping back from it. We’ll keep investing until we’ve improved the situation as much as we can.
We want to answer your questions.
If you have questions about what you’re seeing in your reports, contact our support team, or your Customer Success Manager if you have one. They can give you the latest update on where things stand.
