Open Search Console's Pages report and work the "Why pages aren't indexed" list by class: 301 the 404s that had links or traffic, give soft 404s real content or a real 404, chase 5xx errors to hosting or an overzealous firewall, and unblock anything robots.txt caught by accident. Validate each fix and recheck monthly.
Where the Errors Actually Live
Search Console, Pages report, "Why pages aren't indexed." That table is the master list, sorted by count: Not found (404), Soft 404, Server error (5xx), Blocked by robots.txt, Redirect error, and the rest. Each row expands into the affected URLs. Pair it with a Screaming Frog crawl, which finds the internal links pointing at those dead URLs: Search Console tells you a page is broken, the crawl tells you who keeps sending people there. URL Inspection handles single-page questions; the Crawl stats report under Settings shows whether errors are eating a meaningful share of Googlebot's requests.
Plain 404s, Sorted by Intent
A 404 is not automatically a problem. A URL that should not exist returning "not found" is the system working. Sort each dead URL into one of three bins. The page moved: 301 it to the new address. The page died but has a close substitute: 301 it there instead. The page is gone and has no replacement: let the 404 stand, or return a 410 if you want to say "gone on purpose." What you never do is redirect every dead URL to the homepage; Google treats those as soft 404s and users treat them as a bait and switch.
A Menu Migration, By the Numbers
The classic cannabis case. A store switches menu providers and the URL pattern changes from /shop/product-name to /menu/product-name. Overnight, roughly 300 product URLs start returning 404s — every Leafly mention, every Reddit link, every customer bookmark now hits a dead end, and the Pages report fills up within a couple of weeks. The fix took one pattern-matching rewrite rule: anything starting /shop/ 301s to the same slug under /menu/, with a fallback to the category page for products that did not carry over. Errors drained out of the report over the following month. Every menu-provider switch needs this mapping written before launch day, not after.
Soft 404s: The Sneaky Class
A soft 404 is a page that returns a 200 success code while showing nothing useful — an empty "no products found" category, an out-of-stock item with a bare title, a "page not found" message served as a normal page. The status code says fine, the content says empty, and Google flags the contradiction. Either make the page real (restock content, add copy) or make it honest (a true 404 or 410). Dispensary menus generate these constantly as inventory churns, which makes this the class worth rechecking most often.
Server Errors and Accidental Blocks
5xx errors mean your server failed before content ever mattered. On cannabis sites the usual suspects are underpowered shared hosting buckling on drop-day traffic and security plugins or firewalls that rate-limit Googlebot into 503s. Your host's logs settle which one it is. Blocked by robots.txt has an even shorter diagnosis: read the file. If a valuable path is listed, someone blocked it by accident, often a leftover Disallow from staging.
Fix, Validate, Repeat Monthly
Each error class in the Pages report has a Validate Fix button; press it after fixing the batch and Google recrawls the affected URLs over days to a few weeks, reporting pass or fail per URL. Then put a monthly reminder on the calendar. Errors are not a project you finish; every inventory change, plugin update, and template edit mints new ones, and the report only helps if someone reads it.
Key takeaways
- The Pages report's "Why pages aren't indexed" table is the master error list; Screaming Frog finds the internal links feeding it.
- 301 dead URLs to real equivalents; never blanket-redirect everything to the homepage.
- Menu-provider switches need a URL mapping written before launch, or hundreds of product links die at once.
- Soft 404s come from empty categories and stripped product pages; make them real or make them honest.
- Validate fixes in Search Console and recheck monthly, because errors regenerate.
Frequently asked questions
Are 404 errors bad for my rankings?
Not by themselves — Google expects sites to have 404s, and a URL that never existed returning one is correct behavior. They hurt when the dead URL had links, traffic, or internal links still pointing at it. Fix those with 301s and link updates; ignore the rest.
What is a soft 404?
A page that returns a 200 success code while showing essentially nothing — an empty category, an out-of-stock product with no content, a 'not found' message served as a normal page. Google flags these in the Pages report because the status code and the content disagree.
How fast do fixes show up in Search Console?
After you fix a class of errors, use the Validate Fix button in the Pages report; Google recrawls the affected URLs over days to a few weeks. A single urgent page moves faster through URL Inspection's request-indexing option.
Want this done for your brand?
We build cannabis & CBD search visibility with full transparency, for dispensaries, brands, and MSOs.
Get a free cannabis SEO audit ›