Why cookie scanning is broken (and how Iris fixes it)

Page count isn't the metric that matters. Tracker coverage is. Here's why most scanning tools aren't built to maximize it, and what Iris does differently.

If you manage privacy for a website of any real size, you've probably run into one or more of these cookie/tracker scanning issues: 

• Cookies keep changing. New ones appear, old ones disappear, and nothing about your site has actually changed. Every scan report looks a little different from the last one, with no clear cause.
• The scan isn't finding everything. You know a specific tracker exists on your site – a checkout flow, a regional campaign – and the scan never surfaces it. Cycle after cycle, it's just not there.
• The scan finds things you're confident aren't on your site, and then they vanish. A tracker shows up once, you go looking for it and can't find any trace of it in your own code, and the next scan doesn't show it either.

If any of that sounds familiar, it's not bad luck, and it's not your site being unusually messy. It's a predictable symptom of how most scanning tools decide which pages to look at, and what they do with what they find.

This post covers:

• Why "how many pages can you scan" measures the wrong thing entirely
• How a scanner actually works under the hood, and why every page costs real time and money
• Why the long tail of trackers is structural, not random, using a real streaming-site example
• Why page-embedded monitoring can't see the cookies that matter most
• What Iris, our patent-pending, cohort-based scanning algorithm, does differently

‍

‍

“How many pages can you scan?” is the wrong question 

Most evaluations of a scanning tool start with the same question: “how many pages can you scan?” 

It's an understandable question for a few reasons: 

• It's quantifiable. 
• It's easy to compare across vendors.
• It maps to the (natural!) instinct that more pages scanned = more confident results. 

Unfortunately, it’s the wrong question. Page count measures scanning effort, but doesn’t measure actual tracker coverage. You’d think those numbers would correlate, but in practice they rarely do.

The right question: what percentage of the trackers actually on your site does this tool find? 

If a vendor can't answer that with a real number and a real methodology, you're being sold effort, not coverage.

How does scanning work, actually?

We’re going to talk about why “coverage” is a hard problem to solve. But first, let’s unpack what a scan does. The mechanics behind it are exactly why coverage is a genuinely difficult problem to solve.

A scanner is essentially a headless browser – a real browser engine, like Chromium – running without a visible window. Here’s what happens in a scan: 

1. The headless browser is pointed at a URL. 
2. Instructions tell it to load it the way an actual visitor's browser would. 
3. The browser fetches the HTML, runs the JavaScript on the page, and renders it.

Now: loading a page isn’t enough to surface and catalog all of the page’s trackers. Many trackers fire later, well after the page loads, according to visitor behavior: 

• Scrolling past a certain point
• Hovering over an element
• Clicking through a multi-step flow
• Interacting with a widget, like a chatbot or "add to cart" button

So: a REAL scan doesn’t just load the page, catalog trackers, and walk away. It actually simulates visitor behavior – scrolling down, clicking through, even filling in test data on a form – to coax the page into doing everything it would do for a real visitor.

While all of this is happening, the scanner records everything observable: 

• Every cookie that gets set.
• Every local storage and session storage write.
• Every network request fired, including which domain it went to and what triggered it.

That last part matters more than it sounds like it should, because it lets you reconstruct the sequence of who called whom. Imagine a tag manager loads a vendor's script, and that script calls another vendor, and that one calls a third. A good scanner can capture that whole chain, not just the end result.

That's the only way to tell what a site operator actually put there on purpose, apart from what showed up several steps downstream through no direct action of their own.

One more constraint worth calling out: We’re talking about scanning real, in-production websites here. So a scanner has to be “polite” about all of this! It rate-limits itself, respects robots.txt, and throttles how many requests it sends to a single customer's servers at once. Because hammering a production website with automated traffic is its own kind of problem. 🫠

None of this is a simple fetch. Each of these steps – loading, interacting, watching, recording – takes time per page. Now consider the size of a typical website. If you’re going for page volume, scanning is a time-consuming and expensive process.

Why 80% of pages isn't the same as 80% of trackers

As I mentioned, most people (reasonably) equate page scans with tracker coverage. “If I scan 80% of pages, I’ll probably find 80% of the trackers.” Unfortunately this isn’t the case. 

Let’s use a fictitious example. SandStream is a streaming media company with three million URLs. Those three million URLs? They’re not created equal. They break down into a handful of rational segments, each running on different infrastructure with a different tracking footprint:

• sandstream.com/video/*, the content pages: probably two and a half million of the three million URLs, built from a handful of templates and carrying mostly the same analytics, ad, and personalization tags.
• sandstream.com/knowledge/*, the help center: likely running on Zendesk or a similar third-party platform, with a tracking stack that has nothing to do with the video player.
• sandstream.com/shop/*, a merchandise storefront: running on Shopify, with payment processors, fraud detection, and checkout-specific trackers that exist nowhere else on the property.

Like any rule, there are exceptions; but this is how a typical website and its trackers are structured. Scan one product page on Shopify, and you've effectively scanned the tracking profile of every product page on Shopify. Rare trackers aren't randomly scattered throughout the three million pages. They are concentrated at the seams, where one segment ends and a structurally different one begins. 

A scanner that understands this can find nearly everything by sampling intelligently within each segment. A scanner that treats all three million URLs as one undifferentiated pile has to get lucky.

Adtech vendors make it more complicated behind the scenes

There’s another behind the scenes problem that exacerbates things: ad tech platforms constantly exchange visitor IDs with each other behind the scenes. A single tag can trigger a cascade of calls to other vendors, who call others in turn, resolving differently depending on which real-time auction happens to run at that moment. This is called “cookie syncing.” 

(It's also why the same scan can produce different results run to run: which parties get synced depends on which real-time ad auction happens to fire at that exact moment, so the chain isn't fixed.)

‍

‍

A scanner that faithfully records every single hop in these cascades ends up with a long, constantly shifting list of trackers, most of which you never directly integrated and have no way of knowing exist. Chase that all the way down and you're not producing a disclosure, you're attempting to disclose every tracker on earth. 🤪

That's a big topic for a separate time. But the short version that matters here: a competent scanning system should classify what it finds by how many hops removed it is from something you actually placed. 

1. First-party tags and what they deliberately trigger through a known platform are direct.
2. Everything several auctions downstream gets tracked too, but labeled as such. 
3. Categorizing and disclosing are choices you get to make, rather than a default buried in the vendor's methodology.

Why do vendors have page scanning caps? 

Let’s set the page variety and adtech cookie syncing aside for a moment. There's a third reason why trying to scan ALL the pages simply isn’t the smart solution to the scanning problem: the economics. 

Scanning every page is expensive, and someone has to pay

Imagine a company with 200,000 webpages wants a full scan every week. Remember: each page-load isn't a simple fetch. The scanning will load, simulate interaction, observe, record. 

A reasonable estimate for that whole sequence is 5 to 8 seconds; documented benchmarks for headless browser rendering alone already run 5 to 10 seconds per page on production infrastructure. Let’s call it 6 seconds. That's 1.2 million seconds, or about 333 hours of browser time, for a complete scan for one company. 

Now, scale that across a real book of business for a privacy software company with 500 customers averaging that page count. 

• 100 million page-loads a week
• Roughly 166,667 hours of browser time
• A week has 168 hours
• Keeping up requires 992 headless browser instances running continuously, all week, every week, just for this one workload.

That capacity isn't free. Headless Chrome caps useful concurrency at roughly 20 to 25 instances per server regardless of spare memory, because CPU contention is the bottleneck. A compute-optimized AWS instance well-suited to this, 16 vCPUs and 32 GB of memory, runs about $0.68 an hour on demand.

At 25 concurrent instances per box, sustaining that fleet takes on the order of 40 servers running around the clock, roughly $4,570 a week, or about $238,000 a year, in raw compute alone, for one weekly pass across that customer base, before proxy infrastructure, anti-bot tooling, retries, storage, and orchestration. 

Do some companies want a daily scan? Multiply the whole fleet by seven. 

None of this is impossible, but it’s expensive and someone has to pay for it. How much do you want to pay for your cookie scanning product? This is the actual reason page caps exist across this market. Vendors could build bigger scanning fleets if they wanted to, but scanning every page on a fixed schedule is a genuinely inefficient way to spend a compute budget, and a rational business puts a ceiling on it. The cap itself is a reasonable response to real economics.

Most vendors have chosen random sampling over smart sampling

Confronted with this ceiling, the dominant response in this market hasn't been a smarter way to choose what to scan within it. It's been a fixed, small, often literally random sample.

Several widely used scanning tools publish their actual limits in their own documentation. They include restrictions like: 

• Caps in the range of tens of thousands per scan, run on a monthly cadence.
• Hard limits in the dozens to low hundreds of URLs per account, with guidance to manually hand-pick a handful of representative template pages rather than attempt full coverage.
• Audit tools that explicitly select a small number of pages at random on every run, with the stated goal of building "a general view" of compliance over time rather than a complete one.

I agree with setting a cap – it makes good, economic sense for the vendor and buyer. But the whole problem is rooted in scanning a random sample of pages without an intelligence approach. Random sampling colliding with the cookie-syncing cascade means a different small slice of pages, and a different slice of that day's ad auctions, every single time the scan runs.

New cookies appear, known cookies go missing, and none of this is anyone's imagination. It's the arithmetic of picking a tiny, unrelated sample out of a massive, structured site, over and over, with no memory of what was already learned.

If you’re a site owner who has experienced these infuriating day-to-day admin moments: I see you.

A quick debunk: the “on page code snippet” approach 

There's a second model popping up in the market: instead of crawling pages on a schedule, a JavaScript snippet sits on the live site and reports back on whatever real visitors trigger. 

But this approach has a real blind spot. A script on your page can only read cookies that belong to your own domain. It has no way to see a cookie that a different company sets under its own domain, even if that company's tag is running right there on your page. This holds true no matter how that company got onto your page, whether you added them directly or they showed up through another vendor's redirect chain.

The script can often tell that your page contacted that company at all. But knowing you "talked to" a vendor isn't the same as knowing what cookie they set, or who they directed to. That second piece of information, the actual disclosure-worthy detail, is exactly what stays hidden.

On top of that, a growing share of this activity now happens directly between two companies' servers, with no code on your page involved at all, so there's nothing for the snippet to even observe.

A scanner that drives a headless browser from the outside doesn't have this limit. It isn't a script trapped inside your page; it's an external party that can see every cookie in the browser's storage, from every company, not just your own.

So the fair claim isn't that this approach finds nothing. It's this: if your scanning approach runs inside the site, you are necessarily missing the third-party cookies that end up on a visitor's device, the ones that matter most for a disclosure. That's not a coverage percentage you can improve with more traffic or more time. It's a wall.

Meet Iris, the patent-pending scanning algorithm from Ketch 

Random samples and on-page code will never solve the scanning problem. These tactics are focused on volume, not intelligence. We need to bring this problem back to what privacy and tech teams actually need: a real, current understanding of the risk sitting on their own website.

This is the problem Iris is built to solve: the patent-pending, cohort-based scanning algorithm from Ketch. Powering tracker scanning within Ketch Data Sentry and Ketch Consent Management, Iris gives privacy and tech teams a complete, current view of the risk on their digital properties, not just a page count.

‍

‍

Start from the signature, not the URL. Every page scan produces a tracker signature: the specific set of cookies, storage entries, and network calls observed on that page. Cohorting groups pages by that signature, not by what their URL looks like.

‍

‍

Cohorts are discovered, not assumed. The three segments in the streaming example above are the obvious top-level pattern, but the real cohorts are finer-grained and emerge from the data itself. A video page served to a paying subscriber with ads suppressed has a different signature than the same URL served to a free user, so they land in different cohorts even though the URLs are identical.

This is what makes the approach hold up against personalization, A/B tests, and regional consent differences.

‍

‍

Pick the minimum representative set per cohort. Once cohorts exist, you need just enough pages from each one to account for every tracker known to exist in it, sometimes just one or two pages for a cohort of 50,000. A three-million-page site might resolve into a few hundred to a few thousand distinct cohorts, not millions.

‍

‍

Every cycle splits between confirming and discovering. Part of the budget re-checks the representative set to confirm known trackers are still behaving as expected. The rest explores unscanned territory, looking for cohorts or trackers that don't exist in the model yet. Coverage compounds cycle over cycle instead of resetting to zero.

‍

‍

Re-cohorting keeps the model honest. When discovery turns up a page that doesn't match any existing cohort's signature, the structure updates: a new cohort forms, or an existing one splits.

This is also what makes hop-distance classification deliverable at scale. Because that classification runs on the representative pages, and every member of a cohort shares that signature by definition, the classification applies to the whole cohort at once.

That's what makes "show me only what I control" versus "show me everything, with hop-distance labeled" a practical choice, not a theoretical one.

‍

‍

Quick recap‍

We covered a lot of ground here, so let's summarize.

Every problem in scanning today traces back to the same root cause: treating scanning as a volume problem instead of an intelligence problem. Random samples, page caps, on-page snippets, they're all just different flavors of "scan harder." Here's what actually holds up:

• Page count measures effort. Tracker coverage measures completeness. Ask for the second number, not the first.
• Every page costs real time and money to scan properly. That's why every vendor caps something; the question is how they choose within the cap.
• The long tail is structural. Rare trackers cluster at the seams between site segments, not randomly across every page.
• Page-embedded monitoring has a real wall. It can't read third-party-domain cookies, only detect that a request happened, and that gap doesn't close with more traffic or more time.
• Cohort-based sampling gets you most of the way there on a fixed budget, by grouping pages that already look identical instead of treating every page as a fresh mystery.

That's what Iris is built to deliver: an expansive, expanding understanding of the risk on your own website. It's live today, powering tracker scanning within both Ketch Data Sentry and Ketch Consent Management. If you want to hear more, request a demo with our team.