Methodology

How RankingDiff analyzes a Google Search Console comparison

RankingDiff compares two periods of your own Google Search Console export — queries and pages, clicks, impressions, CTR, and position — and reports what changed between them. This page documents exactly how that comparison works: the processing pipeline, the nine diagnostic patterns the engine checks for, the thresholds behind each one, and how a confidence score is built. It's the same engine that runs when you upload your own data on the Analyze page — nothing here is simplified or approximate.

The comparison pipeline

Every analysis — yours or the worked example below — runs through the same seven stages, in this order:

01
Validation. Input is checked against a strict schema before anything else runs. Invalid or malformed rows are rejected with a specific error; the engine never guesses at what a bad row was supposed to contain.
02
Normalization. Clicks and impressions are converted to daily rates, so a 7-day period and a 28-day period can be compared without one distorting the other. CTR and position are already period-independent ratios and are left as reported.
03
Entity matching. Every query and page is classified as retained (present in both periods), before-only, or after-only. The retained cohort is analyzed separately from new and lost traffic, so genuinely new queries or pages can never mask a decline among the ones that were already there.
04
Metric calculations. CTR is always total clicks ÷ total impressions for the period — never an average of individual rows' CTR values. Average position is always impression-weighted (SUM(position × impressions) ÷ SUM(impressions)) — never a simple mean across queries or pages.
05
Diagnostic rules. Nine independent pattern detectors run against the normalized data — each a fixed, documented rule, not a model making a judgment call. One is selected as the primary pattern; any others that also matched are reported as secondary patterns.
06
Confidence scoring. A 0–100 score is built from data sufficiency and the strength of the detected pattern — confidence in the observed pattern, not in any external cause.
07
Structured output. Every pattern is reported in three parts: what was observed in the data, the measurable pattern that represents, and what to check next — never a jump straight from data to a causal conclusion.

How queries and pages are matched between periods

Entities are matched by exact string equality only. "Buy Shoes" and "buy shoes" are treated as two distinct queries, and "/shoes" and "/shoes/" are treated as two distinct pages — neither case nor trailing slashes are normalized away.

This is deliberate: silently folding entities that differ in case or trailing slash risks merging data that isn't actually equivalent, since a trailing slash can point to a different resource and case-folding can hide a genuine rename. RankingDiff would rather under-merge — showing a case or slash change as one lost entity and one gained entity — than over-merge and hide a real change behind a false "retained" match. If your export needs cleanup like this, that's a decision to make before uploading, not one the engine makes for you.

The nine diagnostic patterns

Every comparison runs through all nine detectors below. At most one becomes the primary pattern; any others that also matched are reported as secondary patterns. Each is a fixed rule over the normalized data — never a guess at an external cause.

AGGREGATE_METRIC_CONTRADICTIONHeadline metrics contradict entity-level data

What it means: The site-wide aggregate numbers look neutral or positive — position improved, or impressions rose — but clicks still fell, or a large share of retained queries actually got worse underneath a flat aggregate.
Evidence the engine checks: Aggregate position and impression changes, aggregate click change, and the share of retained-query impressions sitting behind queries whose position worsened materially.
What it does not mean: It doesn't mean Search Console's own numbers are wrong or inconsistent with each other — both the aggregate and entity-level figures come from the same export. It doesn't identify what caused the divergence.
Why it matters: This is checked first, ahead of every other pattern, because it changes how every other number on the page should be read — an improving headline can hide a real, concentrated decline.

SEARCH_APPEARANCE_DECLINESearch appearance declined broadly

What it means: The site was shown in search results materially less often, and that impression decline is spread across a meaningful share of the retained query cohort — not just one or two queries.
Evidence the engine checks: The aggregate impression and click percent changes, and the share of retained-query impressions behind queries whose own impressions fell materially.
What it does not mean: It doesn't say whether the drop is from lower search volume for these terms, a ranking change, an indexing issue, or a SERP layout change — all of these can produce the same measurable pattern.
Why it matters: A broad appearance decline calls for a different investigation than a click-only one: the site is being surfaced less, before CTR or position on what is surfaced even comes into play.

POSITION_ASSOCIATED_DECLINEClicks declined alongside worse rankings

What it means: Clicks fell while impressions stayed broadly stable, and the impression-weighted average position got measurably worse across a meaningful share of retained queries.
Evidence the engine checks: Click and impression percent changes, the absolute position change in places, and the share of retained-query impressions behind queries whose position worsened by at least half a place.
What it does not mean: It doesn't identify which ranking factor moved, or claim any manual or algorithmic ranking action occurred — it only reports that clicks and worse observed position moved together.
Why it matters: It narrows the next investigation to ranking-position questions — content, what else appears on the results page, technical ranking signals — rather than appearance or CTR.

CTR_ASSOCIATED_DECLINEClicks declined while click-through rate fell

What it means: Clicks fell even though impressions and average position both stayed broadly stable — the measurable difference is a material drop in click-through rate at the same visibility.
Evidence the engine checks: Click and impression percent changes, CTR before and after and its percent change, and confirmation that the aggregate position change stayed below the material threshold.
What it does not mean: It doesn't identify a SERP feature, AI Overview, snippet change, or title rewrite as the cause — those are hypotheses to check manually against the current result, not conclusions the engine draws.
Why it matters: A CTR-only decline points attention at how the result appears in search, rather than at rankings or overall visibility.

HIGHLY_CONCENTRATED_PAGE_LOSSClick loss concentrated in a few pages

What it means: Most of the lost clicks trace back to a small handful of pages, rather than being spread evenly across the site.
Evidence the engine checks: The top 1/3/5/10 share of total gross page click loss, the number of pages that lost clicks, and the specific pages carrying the largest losses.
What it does not mean: It doesn't say why those specific pages lost clicks — only that the loss concentrates there.
Why it matters: Concentrated loss means a targeted look at a few pages will explain most of the change, rather than needing to audit the whole site.

HIGHLY_CONCENTRATED_QUERY_LOSSClick loss concentrated in a few queries

What it means: Most of the lost clicks trace back to a small handful of queries, rather than being spread evenly across the retained query set.
Evidence the engine checks: The top 1/3/5/10 share of total gross query click loss, the number of queries that lost clicks, and the specific queries carrying the largest losses.
What it does not mean: It doesn't say why those specific queries lost clicks — only that the loss concentrates there.
Why it matters: It tells you whether to investigate a handful of specific queries or look for a broader, site-wide explanation.

COMPOSITION_CHANGELarge gains and losses offsetting each other

What it means: The query set has large gains and large losses happening at the same time, largely cancelling out — so the net click change looks small even though a lot moved underneath it.
Evidence the engine checks: Gross losses per day, gross gains per day, and the net change per day.
What it does not mean: It doesn't mean nothing changed, and it doesn't say whether the gaining and losing queries are related to each other or entirely unrelated.
Why it matters: A small net change can otherwise be mistaken for a stable period; this flags that the gains and the losses need to be analyzed as two separate groups.

LIMITED_DATANot enough data for a confident diagnosis

What it means: The compared data itself — retained entity counts, or raw click/impression volume — is too small for the retained-cohort shares and thresholds above to be fully reliable.
Evidence the engine checks: The retained query and page counts, and which specific minimum (entity count, click volume, or impression volume) wasn't met.
What it does not mean: It isn't a statement about the site's overall traffic being low — only that this specific comparison has too little retained data to trust fully.
Why it matters: Confidence is reduced whenever this pattern is present, and it can only become the reported primary pattern when no stronger pattern was also detected.

NO_MATERIAL_CHANGENo material change detected

What it means: None of the compared metrics moved by more than their material threshold, and no concentrated entity-level loss was found on either queries or pages.
Evidence the engine checks: The click, impression, and CTR percent changes and the position change, each measured against its material threshold.
What it does not mean: It doesn't mean literally nothing moved — small movements below the material thresholds are treated as noise by design, not as evidence of a change.
Why it matters: It's the only pattern ever reported alone, with no secondary patterns — RankingDiff's explicit way of saying the two periods you compared aren't meaningfully different by these measures.

The thresholds behind each pattern

Every number below lives in one place in the engine's source and is applied consistently across every pattern that uses it.

Material click change	15%	Below this, relative daily-rate click movement is treated as ordinary noise, not a signal.
Material impression change	15%	Mirrors the click threshold for search-appearance swings.
Material CTR change	10% (relative)	CTR is a ratio, so a relative move — e.g. 5.0% to 4.5% — is the meaningful unit, not an absolute-point change.
Material position change	0.5 places	The smallest impression-weighted position shift treated as worth flagging.
High concentration	50% (top 3)	When the top three pages or queries own half or more of all lost clicks, the decline isn't broad.
Widespread decline share	30%	Share of retained-cohort impressions that must sit behind a declining entity for the decline to count as widespread rather than isolated.
Limited entity count	30 retained entities	Below this, cohort-level shares are too sparse to trust.
Minimum clicks for analysis	50 (before period)	Raw click floor below which the analysis is flagged as data-limited.
Minimum impressions for analysis	1,000 (before period)	Raw impression floor below which the analysis is flagged as data-limited.
High-confidence retained count	50 retained entities	A cohort this large is where cross-entity agreement becomes meaningful enough to let confidence reach "high".
Composition net-to-gross ratio	50%	The net change must be no more than half of the smaller gross side (gains or losses) for it to count as a composition change.
Confidence bands	70+ high · 45–69 medium · below 45 low	The score thresholds that set the reported confidence level.

How the confidence score is built

Every analysis starts at a baseline score of 50, then moves up or down based on specific, documented factors: the size of the retained query cohort (a large cohort adds up to 20 points; a small one subtracts up to 35), whether data quality was flagged as limited (−8), the strength of the detected primary pattern (up to +20, plus a further +6 if that pattern is strongly expressed), whether a contradictory aggregate signal is present alongside a different primary pattern (−8), and whether a concentrated-loss pattern is sharply concentrated (+6).

The final score is clamped to 0–100 and mapped to a level: 70 and above is "high", 45–69 is "medium", and below 45 is "low". This score is confidence in the pattern that was detected — not confidence in any explanation for why it happened.

What this engine does not do

RankingDiff identifies patterns in the data. It does not claim to know the external cause.

Every pattern above separates what was observed from the pattern that represents, and stops there. The engine will never conclude, even as a secondary remark, that:

A Google algorithm update caused the change
A manual action or penalty occurred
AI Overviews caused the change
Competitors caused the change
A technical, indexing, or crawling issue caused the change
Seasonality caused the change

A worked example

Example · Illustrative demo data · Not a customer case study

This is the exact bundled dataset behind the "Try example analysis" button on the homepage — a small set of queries and pages for an illustrative vegan bakery site, invented for demonstration only. It runs through the identical pipeline described above, with no shortcuts, and the numbers below are that pipeline's real, live output for this input.

Input

12 queries and 10 pages, compared across a 28-day before period and a 28-day after period of equal length.

What changed (aggregate query metrics)

Metric	Before	After	Change
Clicks/day	168/day	134.9/day	-20%
Impressions/day	2,519.6/day	2,483.6/day	-1%
CTR	6.7%	5.4%	-19%
Position	6.5	7.7	+1.2

Query and page changes

The loss is not spread evenly. A small number of pages and queries account for almost all of the gross click loss:

Largest losses — pages

/blog/vegan-bakery-guide: −20.4/day
/blog/gluten-free-recipes: −13.6/day
/locations/austin-store: −4.6/day

Largest losses — queries

vegan bakery near me: −25/day
gluten free cupcakes: −15.4/day
vegan bakery austin: −5.1/day

Detected pattern (primary)

POSITION_ASSOCIATED_DECLINE

Evidence (the engine's own output, unedited)

Query clicks decreased 20% while impressions were broadly stable (1%). Impression-weighted position moved +1.2 places (worse), and 41% of retained-query impressions sit behind queries whose position worsened materially.

The observed click decline is associated with worsening observed position across retained queries.

The engine's observed / interpretation / next-investigation text is generated in English only, regardless of site language — shown here exactly as it would appear on your own Results page.

Secondary patterns also present

HIGHLY_CONCENTRATED_PAGE_LOSS
HIGHLY_CONCENTRATED_QUERY_LOSS
LIMITED_DATA

Confidence

37/100 · low

What RankingDiff deliberately does not conclude from this example

That any Google algorithm update, manual action, or penalty occurred
That AI Overviews, a SERP feature, or a competitor caused the position change
That the specific pages or queries above will behave the same way for a different site
Anything about causes beyond what the pattern's own evidence states

Ready to run this against your own data?

Compare your own Search Console periods

— it runs the same pipeline, in your browser.