URL Inspection API: Audit Indexing at Scale

Search Console's Page Indexing report groups thousands of URLs and won't show you the detail. The URL Inspection API does: one URL at a time, it returns the canonical Google picked, whether a noindex is blocking the page and when it was last crawled.

It is the same data you see when you paste a URL into the search bar at the top of the panel, only over HTTPS and in bulk: a scheduled query you store in a database and compare day after day, instead of a manual review.

What the URL Inspection API does (and doesn't)

The call is a POST to https://searchconsole.googleapis.com/v1/urlInspection/index:inspect with two required fields: the URL you want to check and the property it belongs to. The response carries an inspectionResult block with the indexing state, the canonical, the last crawl date and the sitemaps that reference the page.

There is one detail most people miss: the API returns the state of the copy that is in Google's index, not a live test. The official reference is blunt about it - you cannot test the indexability of a live URL. The "Test live URL" button in the panel has no equivalent in the API.

The three things it can't do

It does not submit anything for indexing and does not request a recrawl. That is either the manual request indexing button or the Indexing API, which only accepts JobPosting and BroadcastEvent inside a VideoObject.
It does not read properties outside the credential: the service account has to be added to that specific property.
It does not return clicks or impressions. That is Search Analytics, or the BigQuery export if you want to join the two datasets.

URL Inspection API vs Indexing API

This is the most common - and most expensive - mix-up: one reads state, the other notifies. If you build a script against index:inspect hoping to speed up indexing, nothing will happen, because none of those requests tells Google the content changed.

The real value of inspection is different: it is the only source that tells you, per URL and in bulk, that Google chose a different canonical than the one you declared.

Quota: 2,000 inspections per day, per property

The official limits are 2,000 queries per day and 600 per minute per site. At the Google Cloud project level the ceiling is far higher (10,000,000 per day and 15,000 per minute), so the bottleneck is never the account: it is the property.

And it really is per property. If you have the same site as a domain property and as a URL-prefix property, each one carries its own quota of 2,000. Google confirmed this when it announced the API in January 2022.

Why 2,000 is rarely enough (and why that's by design)

A 50,000-URL site would need 25 days for a full pass. The API was built for sampling and debugging, not for walking the whole site: accept that up front and you design the sample before writing the script.

Setting up access in Google Cloud

The steps, in order:

Create a Google Cloud project and enable the Google Search Console API.
Create OAuth2 credentials: a service account for scheduled scripts, the desktop flow for local tests.
Request the https://www.googleapis.com/auth/webmasters.readonly scope.
Add the service account email as a user of the property in Search Console, with read permission or higher.

Step 4 is the one people forget: without it the API returns a permission error even though the token is valid.

Getting siteUrl right: trailing slash and sc-domain

This error breaks most first attempts. A URL-prefix property is written in full and with a trailing slash: https://www.example.com/. A domain property is written with no scheme and no slash: sc-domain:example.com. Mix the two formats and the request fails with a permission error that never says the problem is the format.

The inspected URL has to sit under that property. Query a URL from another subdomain with the main property's credential and it fails the same way.

The call: endpoint, request body and a Python script

The minimum body is two fields. The third, languageCode, is optional (BCP-47, defaults to en-US) and only changes the language of the translated messages in the response.

from google.oauth2 import service_account
from googleapiclient.discovery import build
import os, time

SCOPES = ["https://www.googleapis.com/auth/webmasters.readonly"]
creds = service_account.Credentials.from_service_account_file(
    os.environ["GSC_SA_JSON"], scopes=SCOPES)
service = build("searchconsole", "v1", credentials=creds)

body = {
    "inspectionUrl": "https://example.com/blog/example-post/",
    "siteUrl": "sc-domain:example.com",
}
res = service.urlInspection().index().inspect(body=body).execute()
r = res["inspectionResult"]["indexStatusResult"]
print(r["verdict"], r.get("googleCanonical"), r.get("lastCrawlTime"))
time.sleep(0.1)

Two production notes: keep the service account JSON outside the repository and read its path from an environment variable; and keep the sleep short but present, because 2,000 URLs with no pause push you toward the 600-per-minute limit.

Reading the response field by field

Everything useful lives in inspectionResult.indexStatusResult. These are the fields worth storing:

verdict: PASS, FAIL or NEUTRAL. PARTIAL is reserved and no longer in use.
coverageState: the text you see in the panel.
robotsTxtState: ALLOWED or DISALLOWED.
indexingState: INDEXING_ALLOWED, BLOCKED_BY_META_TAG or BLOCKED_BY_HTTP_HEADER.
pageFetchState: SUCCESSFUL, SOFT_404, NOT_FOUND, ACCESS_DENIED, ACCESS_FORBIDDEN, SERVER_ERROR, REDIRECT_ERROR, BLOCKED_4XX, INTERNAL_CRAWL_ERROR, INVALID_URL or BLOCKED_ROBOTS_TXT.
lastCrawlTime: RFC 3339 timestamp, absent if the page was never crawled successfully.
googleCanonical and userCanonical: the one Google picked and the one you declared.
crawledAs: MOBILE or DESKTOP.
sitemap and referringUrls: how Google found the page.

mobileUsabilityResult is marked as deprecated. Do not turn it into a KPI in a new dashboard: the field no longer updates and it will give you false positives forever.

coverageState is localized text, not a status code

Send languageCode: "es-ES" and coverageState comes back in Spanish. An if coverageState == "..." comparison breaks the moment you change languages or Google rewords the message. Filter on the enums (verdict, indexingState, pageFetchState) and use the text for display only.

The four diagnoses worth the quota

Google picked a different canonical

When googleCanonical differs from userCanonical you have content competing with itself: contradictory canonicals across versions, duplicated parameters or two near-identical pages. It is the one signal no aggregated report gives you per URL.

Crawled, currently not indexed

It shows up as indexingState: INDEXING_ALLOWED with verdict: NEUTRAL. This is not a technical block: it is a quality or thin-content problem. You fix it with content, not with tags.

Blocked by noindex

The distinction matters here. BLOCKED_BY_META_TAG is the noindex sitting in the page HTML; BLOCKED_BY_HTTP_HEADER is the one sent in the server header, which never shows up in the page source - exactly where nobody looks for it.

404s, soft 404s and redirect errors

pageFetchState separates three things an aggregated report makes look alike: NOT_FOUND (a real 404), SOFT_404 (returns 200 but looks empty) and REDIRECT_ERROR (a loop or a broken chain). With that split you prioritize by template instead of by anecdote.

Common errors and what they actually mean

401: wrong credentials or scopes; almost always the missing webmasters.readonly.
403: the account has no permission on that property, or the URL does not sit under it.
429: quota exceeded. Exponential backoff, not retry loops.
400: malformed body, inspectionUrl without https://, or siteUrl with a slash where it does not belong.

If richResultsResult or ampResult come back empty, that is not a script failure: the analysis simply does not apply to that page.

How to spend your 2,000 inspections a day

An order that works: new templates and recent deploys; URLs with impressions but no clicks; pages where the canonical or a redirect changed; a fixed random sample of the rest to catch regressions; and the URLs you just asked Google to index.

Store history (date, url, verdict, canonical) in a table. Without a time series the API is a snapshot, and you cannot answer the question that matters: since when has it been like this.

When the API is the wrong tool

For volume, the Page Indexing report and the sitemap already give you the aggregate. To see what Googlebot really crawls, server logs are better: every request, no quota. And to notify a change, use IndexNow or the manual indexing request; inspection does not notify anyone.

If the real question is why a page will not get in, cross the response with what you already know about meta robots and noindex, 404 and soft 404 errors and orphan pages. And if you want inspection data next to clicks and impressions, the Search Console to BigQuery export is the natural path.

The URL Inspection API is, in the end, a per-URL magnifying glass. It answers the questions the aggregated report cannot: which canonical Google picked, why a page is not getting in, and since when.

Sources: urlInspection.index.inspect reference, API usage limits and the URL Inspection API announcement (January 2022).