XML Sitemap Protocol & Indexation Best Practices
An XML sitemap acts as an explicit indexation roadmap for search engines. It provides search engine crawlers with a structured directory of canonical URLs, modification timestamps (<lastmod>), and optional media extensions (images, videos, or news articles).
Sitemap Protocol Specifications & Limits
Standardized under the Sitemaps.org standard and supported by all major search engines:
- Single File Limit: A standard
<urlset>can contain a maximum of 50,000 URLs and cannot exceed 50 MB uncompressed. - Compression: Sitemaps can be gzip-compressed (e.g.,
sitemap.xml.gz) to save server bandwidth; search engines automatically decompress them. - Character Encoding: Must use UTF-8 encoding with entity escaping (e.g.,
&for&,"for").
Sitemap Index Architecture (<sitemapindex>)
Websites containing more than 50,000 URLs, or with distinct content categories, must split records across multiple sitemap files referenced by a parent sitemap index:
<?xml version="1.0" encoding="UTF-8"?>
<sitemapindex xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
<sitemap>
<loc>https://www.example.com/sitemap-posts.xml</loc>
<lastmod>2026-08-20T10:00:00+00:00</lastmod>
</sitemap>
<sitemap>
<loc>https://www.example.com/sitemap-products.xml</loc>
<lastmod>2026-08-25T14:30:00+00:00</lastmod>
</sitemap>
</sitemapindex>The Critical Role of <lastmod>
Google officially disregards <priority> and <changefreq> tags because webmasters historically inflated them. However, Google actively uses the <lastmod> timestamp to determine whether content has been modified and requires re-crawling.
Rule: Only update <lastmod> when meaningful content changes occur. Setting all URLs to today's date automatically will cause search engines to distrust your timestamps.
Common Sitemap Errors & How to Fix Them
1. Including Non-Canonical or Redirecting URLs
A sitemap should strictly contain HTTP 200 indexable canonical URLs. Never include URLs that return:
- 301/302 redirects (verify with our Redirect Checker)
- 404/410 errors (verify with our HTTP Status Checker)
- URLs marked with noindex directives
- URLs with cross-page canonical tags pointing elsewhere
2. Off-Domain URL Declarations
Sitemaps cannot contain URLs on domains or subdomains other than the one hosting the sitemap, unless cross-domain ownership is verified inside search engine webmaster tools.
Frequently Asked Questions
Where should I place my sitemap?
Place your main sitemap at the domain root (e.g., https://example.com/sitemap.xml) and reference its absolute URL inside your robots.txt file using the Sitemap: directive.
How do I submit my sitemap to Google?
Log into Google Search Console, navigate to Index > Sitemaps, enter your sitemap filename, and click Submit. This triggers Googlebot to queue the file for parsing.
Does having a sitemap guarantee that all my pages will be indexed?
No. A sitemap assists in URL discovery, but search engines ultimately determine indexation based on content quality, technical accessibility, and authority. Use our Google Index Checker to inspect specific URL indexation signals.