Glossary
HTTP status code
An HTTP status code is the three-digit number a server puts at the start of every response to say what happened to the request. The first digit gives the class: 1xx informational, 2xx success, 3xx redirection, 4xx the request was faulty, 5xx the server failed. Automated consumers — browsers, caches, link checkers, search engine crawlers — act on the code and not on the words in the page, which is why a page that says one thing and returns another is a defect even when it looks correct to a human.
The five classes
1xx informational: the request was received and processing continues. `100 Continue` and `101 Switching Protocols` are the two a site may emit; neither is a final answer and neither ends the exchange.
2xx success: the request was understood and fulfilled. `200 OK` is the ordinary answer, `204 No Content` says there is nothing to return, and `206 Partial Content` answers a range request such as a video seek.
3xx redirection: the resource is somewhere else, or the copy the client holds is still good. `301`, `302`, `307` and `308` point elsewhere; `304 Not Modified` says the cached copy may be reused.
4xx client error: the request was wrong in some way. `400` malformed, `401` not authenticated, `403` authenticated but not allowed, `404` no such resource, `410` deliberately removed, `429` too many requests.
5xx server error: the request was fine and the server failed anyway. `500` an unhandled fault, `502` a bad answer from an upstream, `503` temporarily unavailable, `504` an upstream timed out.
Why the code matters more than the page
A status code is the only part of the answer every automated client agrees on. A browser decides whether to cache, a proxy decides whether to store, a link checker decides whether the link works, and a search engine decides whether to keep the URL indexed — all from the number.
The page body is for people. It carries no machine-readable claim about whether the request succeeded, so a body that says "page not found" under a `200` is read by every machine as a successful page with that text on it.
That mismatch has a name: a soft 404. It is common in applications that render the not-found view in the browser without the server ever being told the route was invalid.
The rule that avoids all of this is short: decide the status on the server, and let the page describe it rather than replace it.
Which codes a crawl actually surfaces
The entry URL's code decides whether the rest of the scan is possible at all. A `5xx` or a connection failure there means nothing downstream can be measured, so it is reported as a state rather than as a list of findings.
Link checking produces the widest spread. Internal links resolving to `404` or `410` are broken; `403` usually means a protected area or a bot filter rather than a dead page; `429` means the check was rate limited and says nothing about the target.
Redirects are followed and counted. A single `301` is normal; a chain of several is a latency and signal-dilution problem, and a chain that returns to its own start is a loop that never resolves.
Codes seen on subresources — images, scripts, stylesheets — matter differently: a `404` on a stylesheet changes what the page looks like, not whether it is indexable.
How to read one yourself
`curl -I https://example.com/some-path` prints the status line and headers without downloading the body. Add `-L` to follow redirects and `-v` to see each hop's code in order.
In a browser, the Network panel lists the status of every request the page made, which is the fastest way to find a subresource failing under a page that renders correctly.
Check a URL you know is invalid on each distinct part of the site. Applications frequently return the right code for static routes and `200` for application routes, and the difference only shows when you look.
Search Console's Page indexing report groups URLs by the outcome Google observed, which is where a status problem that only appears to crawlers becomes visible.
Frequently asked questions
- Which status code should a removed page return?
- 404 or 410. Both tell clients the resource is gone; 410 adds that the removal is deliberate and permanent, which tends to remove the URL from a search index sooner.
- Is a 3xx status an error?
- No. A redirect is a normal instruction to look elsewhere. It becomes a problem when there are several in a row, when it loops, or when it points somewhere unrelated to what was requested.
- Can one URL return different codes to different clients?
- Yes. Geographic routing, bot filtering, rate limiting and authentication all change the answer. That is why a code observed once is evidence and not proof.
Sources
Related
VeriFixScan crawls a site and applies its checks to every page it reaches, keeping the evidence behind each finding. Scanning one website is free.
Scan a website