Google generates organic search listings from its database of hundreds of billions of web pages. Indexing a page in that database is Google’s first step in determining rankings.
“Indexation” in Google-speak means it has added the URL to the database along with key information, on-page (headings, body text, meta tags) and off (internal and external links, text surrounding those links, author info). Google uses that info in search results and regularly updates the index.
A company could block a page from Google’s index via a robots.txt command, although it’s not foolproof, and Google could still index it using off-page info.
Given the enormity of its index, Google may require a few days to discover and index a new page. And new pages are indexed quicker than updates.
How to get a page indexed?
Google will typically discover and quickly index pages of websites with less than a few thousand URLs, provided there are internal links to each of those pages. External links will speed up discovery, as will the submission of an XML sitemap in Search Console.
Once indexed, Google will regularly revisit your site for changes.
Google allows requests to recrawl a site. But frequent requests are often indicators of technical problems. So instead of a recrawl, audit your site to ensure internal links are valid and easily crawled. A third-party crawling tool such as Screaming Frog can help.
XML sitemaps are especially helpful for large, database-driven sites with thousands of product pages and hundreds of categories. Sitemaps will enable Google to access deeper pages easier and report back (via Search Console) with structural or indexation problems.
How to confirm indexing?
There are two ways to tell if Google indexed a page:
- Search Google for site:full-url (e.g., site:practicalecommerce.com). If indexed, the URL will appear in organic results.
- Check the “Pages” report on Search Console.
Note that Google can index a page and not cache it.
Keep an eye on the “Pages” tab for indexing glitches. Scroll down to the “Crawled – currently not indexed” report for a list of pages not in Google’s index and the cause. The report typically includes a few non-indexed pages. However, if the number is growing or more than a few, review your site for underlying problems.
When browsing that report, I often find URLs that are actually indexed, indicating the report is outdated.
Nonetheless, a non-indexed URL can signal a wider problem. So, again, rule out that possibility before requesting a recrawl.
The “Crawled – currently not indexed” report lists pages not in Google’s index and the cause. Click image to enlarge.
How to speed up indexing?
The most common (and valid) reason for faster indexing is verifying a fix — i.e., Google reported an error via Search Console, and you fixed it. In that case, request indexing through Search Console’s “URL inspection” tool.
Google will email when it verifies the fix and indexes the page. It usually takes a couple of days.
Use the tool only occasionally, certainly not daily. When re-platforming an entire site to a new domain or content management system, don’t attempt to use this tool to re-index page by page. Instead, submit an updated sitemap in Search Console. That will prompt Google to index your site quickly.
After fixing errors on a page, the “URL inspection” tool can speed up its indexing. Click image to enlarge.
