Removed pages returning 404 instead of a redirect

What Is This Issue

This issue checks whether pages that have been moved to a new URL or deleted return proper 301 redirects (for moved pages) or appropriate error pages (for deleted pages), rather than abrupt 404 errors. A 301 redirect passes link equity to the new URL, while abrupt 404s on moved pages lose accumulated PageRank.

A passing implementation means:

• Pages that have moved to a new URL return a 301 (permanent) redirect to the new location
• The redirect chain is short (ideally direct, not multiple hops)
• Deleted pages that don’t have a relevant replacement return 404 or 410 (Gone) status
• No moved pages return 404 abruptly without a redirect

Example:

• Old URL: https://example.com/old-page
• New URL: https://example.com/new-page
• ✅ Old URL returns 301 → https://example.com/new-page
• ❌ Old URL returns 404 Not Found (link equity lost!)

Why Is This Important

Link Equity Preservation: 301 redirects pass the vast majority of link equity (PageRank) from the old URL to the new URL. Abrupt 404s lose this valuable ranking power.

User Experience: Users who bookmark or find old URLs in search results should be automatically taken to the new location, not encounter a 404 error.

Crawl Efficiency: Search engines will continue crawling old URLs that return 404, wasting crawl budget. 301 redirects tell them to update their index to the new URL.

SEO Rankings: Pages that have moved but don’t redirect may lose their rankings because search engines see the content as “gone” rather than “moved.”

Historical Value: Old URLs may have accumulated backlinks over time. 301 redirects ensure those links continue to benefit your site.

SEO Health Score: Proper redirect management is a fundamental technical SEO practice that significantly improves the site’s health score.

How XeoPix Detects This

XeoPix performs the following checks:

1. Crawls the website - The crawler follows links throughout the site, discovering URLs.

2. Identifies potential moved pages - XeoPix looks for patterns that suggest a page may have moved:

   • URL structure changes
   • Pages that return 404 but have similar content elsewhere
   • Historical data from previous crawls (if available)

3. Checks HTTP status codes - For each URL, XeoPix records the HTTP status code:

   • 200: OK (page exists)
   • 301/302: Redirect (follows to destination)
   • 404: Not Found
   • 410: Gone

4. Validates redirect type - If a redirect is found, XeoPix checks if it’s a 301 (permanent) vs 302 (temporary). For moved pages, 301 is preferred.

5. Reports issues - The issue is flagged if:

   • A page appears to have moved but returns 404 instead of 301
   • Redirect chains are too long (3+ hops)
   • 302 redirects are used for permanently moved content

How To Fix

1. Identify moved pages - Review your site for pages that have been:

   • Moved to a new URL structure
   • Merged into other pages
   • Replaced with updated versions

2. Implement 301 redirects - Set up server-side 301 redirects from old URLs to new URLs:

   • Apache: Redirect 301 /old-page https://example.com/new-page
   • Nginx: rewrite ^/old-page/?$ https://example.com/new-page permanent;
   • CMS: Use redirect plugins/modules

3. Avoid redirect chains - Don’t chain multiple redirects (A → B → C). Redirect directly to the final destination.

4. Handle deleted pages - For pages that are deleted without a replacement:

   • Return 404 (Not Found) or 410 (Gone)
   • Consider a custom 404 page with helpful navigation
   • Don’t redirect to the homepage (this confuses search engines)

5. Update internal links - Change internal links to point directly to the new URLs instead of relying on redirects.

6. Monitor with tools - Use Google Search Console to identify 404 errors and set up proper redirects.

What We Store

Storage Level

Page Level — This issue is evaluated for each individual page.

Database Table / Prisma Model

PageUrlParameterAudit

Stored Fields

Detection Dependencies

• The following data sources are required to evaluate this issue:
• URL — The crawler analyzes the URL structure and extracts query parameters
• HTML Document — The page URL is parsed to identify parameters

Examples

Example 1: Proper 301 Redirect

Problematic State (Fails): Old page returns 404:

• https://example.com/old-services returns 404 Not Found
• Page has valuable backlinks that are now wasted

Corrected State (Passes): Implement 301 redirect:

• https://example.com/old-services → 301 → https://example.com/services

Example 2: Redirect Chain

Problematic State (Fails): Multiple redirects in chain:

/old-page → /intermediate-page → /new-page

Corrected State (Passes): Redirect directly to final destination:

/old-page → /new-page

Example 3: Deleted Page Without Replacement

Problematic State (Fails): Page deleted but redirects to homepage (confusing for search engines):

• https://example.com/discontinued-product → 301 → https://example.com/

Corrected State (Passes): Return 404 or 410:

• https://example.com/discontinued-product returns 404 Not Found
• Consider a custom 404 page with links to similar products

Unit Test

Test File

xeopix-crawling-v2/__tests__/seo-audit-checks/crawlBehaviour/issue-220-moved-deleted-redirect.test.js

Purpose

This unit test validates that the checkMovedDeletedPagesRedirect() function correctly detects pages that have been moved or deleted but do not return proper 301 redirects, reporting issues for pages that return 404 or still serve content at the old URL.

Tested Function

checkMovedDeletedPagesRedirect() from seo-audit-checks.js

Issue Information

• Issue Number: 220
• Issue Code: MOVED_DELETED_RETURN
• Toggle Group: crawlBehaviour

Test Scenarios

Positive Test Cases

None (every historical URL that is checked will produce an issue if it is not properly redirected)

Negative Test Cases

• Historical URL returns HTTP 404 → MOVED_DELETED_RETURN issue created with statusCode 404
• Historical URL still returns HTTP 200 → MOVED_DELETED_RETURN issue created with status still-live

Boundary Cases

None

Edge Cases

• Empty history list → no issues detected, notFoundCount is 0

Expected Outcome

Pass

A MOVED_DELETED_RETURN issue is not reported when the history list is empty (no URLs to check).

Fail

A MOVED_DELETED_RETURN issue is reported when a historical URL:

• Returns HTTP 404 instead of a 301 redirect (statusCode 404)
• Still returns HTTP 200 without redirecting (still-live)

Validation

• Verifies detection of 404 responses on historical URLs
• Validates detection of still-live pages that should have been redirected
• Checks empty input handling produces no false issues
• Validates payload structure (notFoundCount, details.statusCode, details.status)

Related Production Files

• xeopix-crawling-v2/seo-audit-checks.js
• xeopix-crawling-v2/issueCodes.js

Coverage Summary

• 3 test cases (two negative, one edge case)
• Covers both 404 and still-live detection scenarios
• Validates empty history list safe handling
• Includes API mocking for URL ID resolution

References

• 301 Redirects  — Google Search Central
• HTTP Redirections  — MDN
• HTTP Semantics  — RFC Editor