XML Sitemaps: Your Site’s Table of Contents for Google
If robots.txt tells crawlers where not to go, the XML sitemap does the opposite: here is everything worth crawling, and here’s when it last changed. A clean sitemap gets new pages discovered in hours instead of days — a dirty one trains Google to distrust it.
What a Sitemap Is (and Isn’t)
An XML sitemap is a machine-readable file listing the URLs you want search engines to crawl and index. From Lesson 1.1: it’s one of the three discovery paths (links from known pages, internal links, sitemaps) — and it’s the only one you fully control, which makes it especially valuable for new pages (no links pointing at them yet) and new sites (nothing linking in at all).
Two honest limits up front:
- A sitemap is a suggestion, not a command — inclusion doesn’t guarantee crawling, and crawling doesn’t guarantee indexing (Lesson 1.1’s “crawled — currently not indexed” applies regardless).
- A sitemap doesn’t replace internal linking (Lesson 3.4). Pages reachable only via the sitemap are still functionally orphans — discoverable, but receiving no authority and no context. The sitemap accelerates discovery; links make pages matter.
Here’s what one looks like — you never write this by hand, but you should be able to read it:
<?xml version="1.0" encoding="UTF-8"?>
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
<url>
<loc>https://yourdomain.com/eggless-cakes-pune/</loc>
<lastmod>2026-07-01</lastmod>
</url>
<url>
<loc>https://yourdomain.com/photo-cake-guide/</loc>
<lastmod>2026-06-18</lastmod>
</url>
</urlset>
Two tags carry the value: <loc> (the URL) and <lastmod> (when it last meaningfully changed). You may see <priority> and <changefreq> in older sitemaps — Google ignores both; don’t spend a minute on them.
The One Rule: Indexable, Canonical, 200-OK URLs Only
Everything about sitemap quality reduces to one rule: a sitemap contains only pages you want indexed, in their canonical form, returning status 200. Each violation sends Google a contradiction:
| In the sitemap but… | The contradiction |
|---|---|
| URL is noindexed | “Please index this” + “don’t index this” — pick one (Lesson 4.1’s distinction, violated in one file). |
| URL redirects (301/302) | You listed an address that immediately forwards elsewhere — list the destination instead. |
| URL is a 404 | Inviting Google to crawl pages that don’t exist — pure crawl waste, at scale a trust problem. |
| URL is a non-canonical duplicate | You’re nominating a version you’ve told Google (via canonical tags — next lesson) isn’t the real one. |
| URL is blocked in robots.txt | “Crawl this” + “don’t crawl this” simultaneously. |
A sitemap full of contradictions doesn’t just waste crawls — it degrades the signal value of the whole file. Keep it clean and Google treats it as a reliable source; that reliability is the entire point.
Sitemap Index Files: How Real Sites Organise
One sitemap file caps at 50,000 URLs / 50MB, but well before that limit, organised sites split sitemaps by content type and bundle them in a sitemap index — a sitemap of sitemaps:
<sitemapindex xmlns="http://www.sitemaps.org/schemas/sitemap/0.9"> <sitemap><loc>https://yourdomain.com/post-sitemap.xml</loc></sitemap> <sitemap><loc>https://yourdomain.com/page-sitemap.xml</loc></sitemap> <sitemap><loc>https://yourdomain.com/product-sitemap.xml</loc></sitemap> </sitemapindex>
The practical benefit isn’t the size limit — it’s diagnosis. When Search Console reports indexing problems per sitemap, “products: 40% indexed, posts: 95% indexed” instantly tells you where the problem lives. WordPress SEO plugins (Rank Math included) generate exactly this structure automatically: a sitemap_index.xml splitting posts, pages, and any custom types.
The WordPress Setup (Ten Minutes, Once)
- Confirm generation: visit yourdomain.com/sitemap_index.xml. Rank Math generates and maintains it automatically — every publish, update and delete is reflected without your involvement.
- Configure inclusions: in the plugin’s sitemap settings, include only content types that deserve indexing. Typical exclusions: tag archives, author archives on single-author sites, media/attachment pages (a classic WordPress junk-URL source), and any utility post types.
- Check the exclusion sync: good plugins automatically drop noindexed pages from the sitemap — that’s the one-rule enforcement working for you. Spot-check: noindex a test page and confirm it leaves the sitemap.
- Reference it in robots.txt — the Sitemap: line from Lesson 4.1 (Rank Math adds this automatically too).
Submitting to Search Console (and Reading the Report)
Submit once: GSC → Sitemaps → enter sitemap_index.xml → Submit. Google remembers it and refetches periodically forever. The report then becomes a permanent diagnostic:
- “Success” status confirms Google can fetch and parse the file — not that everything got indexed.
- Discovered vs indexed counts per sitemap show your real indexing rate. A large persistent gap points to quality issues (thin/duplicate content — Lesson 1.1’s selective indexing) or the contradictions from the table above.
- Page indexing report → filter by sitemap lists exactly which submitted URLs aren’t indexed and why — “Crawled – currently not indexed”, “Duplicate without user-selected canonical”, “Not found (404)”. Each reason maps to a lesson in this module.
Key Takeaways
- The sitemap is the discovery path you fully control — fastest value on new pages and new sites, but no substitute for internal links.
- Only two tags matter: loc and lastmod — and lastmod must tell the truth or Google learns to ignore it.
- The one rule: indexable, canonical, 200-OK URLs only — no noindexed, redirecting, 404 or blocked URLs.
- Sitemap index files split by content type turn GSC into a per-section indexing diagnostic.
- WordPress + your SEO plugin automates generation, updates and noindex-sync — configure inclusions once, exclude junk types like media pages.
- Submit sitemap_index.xml to GSC once, then read discovered vs indexed as your ongoing site-health gauge.