Glossary
XML sitemap
An XML sitemap is a file listing the URLs on a site that its owner considers worth crawling, optionally with the date each was last modified. It is a hint to search engines about what exists, not an instruction to index anything: inclusion does not guarantee indexing and exclusion does not prevent it. Its practical value is highest on large sites, on sites with poor internal linking, and on sites whose pages are discovered slowly.
What a sitemap contains
A list of `<url>` entries, each with a `<loc>` holding an absolute URL. The URL must be on the same host as the sitemap unless cross-submission has been established.
An optional `<lastmod>` in W3C datetime format. Google has stated it uses this value when it is consistently accurate, and ignores it when it is not.
`<changefreq>` and `<priority>` are part of the protocol and are ignored by Google. Setting them is harmless and pointless.
The file is limited to 50 000 URLs and 50 MB uncompressed. Beyond that, a sitemap index file lists several sitemaps, and index files can be nested.
What belongs in it, and what does not
Only canonical URLs. Listing a URL that carries a canonical tag pointing elsewhere sends two contradictory signals about the same page.
Only URLs that return 200. Redirects and errors in a sitemap waste crawl requests and are reported as errors in Search Console.
Only indexable URLs. A URL that is listed and also carries `noindex` is another contradiction, and Google reports it as one.
Everything you want found, including pages that are poorly linked internally. A page reachable only from a sitemap is a page with a weak position, but it is at least discoverable.
lastmod, and why accuracy matters
`lastmod` should reflect the date the page's content meaningfully changed, not the date the file was regenerated.
A sitemap where every URL carries today's date on every generation is not informative, and search engines stop trusting it as a result.
Used honestly, it lets a crawler prioritise what actually changed, which matters on a site with tens of thousands of pages and a limited crawl rate.
The dates in a sitemap and the `Last-Modified` header on the page itself should agree; disagreement is noticed.
How to publish and verify one
Reference it from robots.txt with a `Sitemap:` line, which every major crawler reads, and submit it in Search Console for reporting.
Fetch the URL directly and confirm it parses as XML and is served with an XML content type.
Spot-check a sample of the listed URLs: each should return 200, be canonical to itself, and be indexable.
Search Console's sitemap report gives the count discovered against the count submitted, and the gap is the fastest indicator of a structural problem.
Frequently asked questions
- Does adding a page to a sitemap get it indexed?
- No. It helps the page be discovered. Whether it is indexed depends on the page itself and on the search engine's own judgement.
- Do priority and changefreq do anything?
- Google ignores both. Only the URL and, when it is accurate, lastmod are used.
- How many URLs can one sitemap hold?
- 50 000 URLs or 50 MB uncompressed. Larger sites split across several files listed in a sitemap index.
Sources
Related
VeriFixScan crawls a site and applies its checks to every page it reaches, keeping the evidence behind each finding. Scanning one website is free.
Scan a website