Robots meta tag missing snippet and preview directives

What Is This Issue

This issue checks whether your pages have a properly configured <meta name="robots"> tag with specific directives that control how search engines display your content in search results.

A passing implementation includes a meta robots tag with the following recommended settings:

max-snippet:-1 (no limit on snippet length)
max-image-preview:large (allow large image thumbnails)
max-video-preview:-1 (no limit on video preview duration)

Example of correct implementation:


<meta
  name="robots"
  content="max-snippet:-1, max-image-preview:large, max-video-preview:-1"
/>

Without this tag, search engines may use default settings that could limit how your content appears in SERPs.

Why Is This Important

The meta robots tag directly impacts how your pages appear in search engine results pages (SERPs):

Rich snippets: The max-snippet directive controls how much text content can appear in search snippets. Setting it to -1 allows search engines to show as much as they deem relevant.
Visual appeal: The max-image-preview:large directive enables large thumbnail images next to your search results, which can significantly increase click-through rates.
Video visibility: The max-video-preview directive controls whether video content can show preview clips in search results, helping video content stand out.
CTR improvement: Properly configured meta robots tags can make your search listings more visually appealing and informative, leading to higher click-through rates.

Resolving this issue improves your SEO health score by ensuring your content has the best possible presentation in search results, which can lead to increased organic traffic.

How XeoPix Detects This

XeoPix performs the following checks to detect this issue:

HTML parsing: The crawler extracts the <head> section from the page’s HTML content.
Meta tag search: It looks for a <meta> tag with the attribute name="robots".
Content validation: If the tag is found, XeoPix checks the content attribute for the presence of:
- max-snippet directive (value can be -1 or a number)
- max-image-preview directive (value should be large, standard, or none)
- max-video-preview directive (value can be -1 or a number)
Pass/Fail determination:
- Passes: If all three directives are present in the meta robots tag content
- Fails: If the meta robots tag is missing, or if any of the three directives are missing

The detection focuses on whether the tag exists and contains the recommended directives, not on the specific values used.

How To Fix

Follow these steps to implement the meta robots tag correctly:

Add the meta tag to your HTML head section: Place the following line inside the <head> section of your HTML document:
```
<meta
  name="robots"
  content="max-snippet:-1, max-image-preview:large, max-video-preview:-1"
/>
```
For specific page types: You can adjust the values based on your needs:
- For pages where you want to limit snippets: Use max-snippet:50 (limits to 50 characters)
- For pages with sensitive images: Use max-image-preview:standard instead of large
- For pages without video: The max-video-preview directive won’t have any effect
Verify implementation: Use View Page Source or browser developer tools to confirm the tag appears correctly in the rendered HTML.
Test with search engines: Use Google Search Console’s URL Inspection tool to see how Google interprets your robots meta directives.

Note: This tag should be present on all indexable pages of your website.

What We Store

Storage Level

Page Level — This issue is evaluated for each individual URL.

Database Table / Prisma Model

PageHtmlHeadAudit

Stored Fields

Field	Type	Description
robotsMetaContent	String?	The content attribute of the robots meta tag
maxSnippet	String?	The max-snippet directive value
maxImagePreview	String?	The max-image-preview directive value
maxVideoPreview	String?	The max-video-preview directive value

Detection Dependencies

The following data sources are required to evaluate this issue:
HTML Document — The crawler parses the HTML head section to find the robots meta tag
Meta Tag Extraction — The <meta name="robots"> tag and its content are extracted
Directive Parsing — Individual directives (max-snippet, max-image-preview, max-video-preview) are parsed from the content

Examples

Example 1: Correct Implementation

Scenario: A properly configured page with meta robots tag.

Correct State (Passes):


<head>
  <meta charset="UTF-8" />
  <meta name="viewport" content="width=device-width, initial-scale=1" />
  <meta
    name="robots"
    content="max-snippet:-1, max-image-preview:large, max-video-preview:-1"
  />
  <title>Page Title</title>
</head>

Example 2: Missing Meta Robots Tag

Scenario: Page has no meta robots tag.

Problematic State (Fails):


<head>
  <meta charset="UTF-8" />
  <meta name="viewport" content="width=device-width, initial-scale=1" />
  <title>Page Title</title>
</head>

Why it fails: Without the meta robots tag, search engines may use default settings that limit how your content appears in search results.

Corrected State (Passes):


<head>
  <meta charset="UTF-8" />
  <meta name="viewport" content="width=device-width, initial-scale=1" />
  <meta
    name="robots"
    content="max-snippet:-1, max-image-preview:large, max-video-preview:-1"
  />
  <title>Page Title</title>
</head>

Example 3: Incomplete Meta Robots Tag

Scenario: Meta robots tag is present but missing required directives.

Problematic State (Fails):


<head>
  <meta name="robots" content="max-snippet:-1" />
  <title>Page Title</title>
</head>

Why it fails: Missing max-image-preview and max-video-preview directives, which may limit how images and videos appear in search results.

Corrected State (Passes):


<head>
  <meta
    name="robots"
    content="max-snippet:-1, max-image-preview:large, max-video-preview:-1"
  />
  <title>Page Title</title>
</head>

Example 4: Restrictive Meta Robots Settings

Scenario: Page uses restrictive settings that limit search result appearance.

Problematic State (Fails):


<head>
  <meta
    name="robots"
    content="max-snippet:0, max-image-preview:none, max-video-preview:0"
  />
  <title>Page Title</title>
</head>

Why it fails: These settings prevent search engines from showing snippets, images, and video previews, which can significantly reduce click-through rates.

Corrected State (Passes):


<head>
  <meta
    name="robots"
    content="max-snippet:-1, max-image-preview:large, max-video-preview:-1"
  />
  <title>Page Title</title>
</head>

Unit Test

Test File

__tests__/seo-audit-checks/htmlHeadTags/issue-106-meta-name-robots.test.js

Purpose

Validates the detection of missing or incomplete <meta name="robots"> declarations in the HTML <head>. Ensures pages are flagged when the robots meta tag is missing or lacks the required preview directives (max-snippet:-1, max-image-preview:large, max-video-preview:-1).

Tested Function

runHtmlHeadTags() from toggleGroups/htmlHeadTags.js

Issue Information

Issue Number: 106
Issue Code: META_NAME_ROBOTS
Toggle Group: htmlHeadTags

Test Scenarios

Positive Test Cases

Page has all three required directives (max-snippet:-1, max-image-preview:large, max-video-preview:-1) — no issue reported
Directives in different order — no issue reported
Case-insensitive directive names (MAX-SNIPPET:-1, MAX-IMAGE-PREVIEW:LARGE, MAX-VIDEO-PREVIEW:-1) — no issue reported
Directives with extra whitespace — no issue reported

Negative Test Cases

Page has no robots meta tag — issue reported with message "Meta robots tag is missing"
Robots tag is missing max-snippet:-1 — issue reported with missingDirectives containing "max-snippet:-1"
Robots tag is missing max-image-preview:large — issue reported with missingDirectives containing "max-image-preview:large"
Robots tag is missing max-video-preview:-1 — issue reported with missingDirectives containing "max-video-preview:-1"
max-image-preview is set to standard instead of large — issue reported with missingDirectives containing "max-image-preview:large"

Boundary Cases

None present.

Edge Cases

Empty HTML (<html></html>): issue reported (meta robots tag is missing)
Case-insensitivity: directive names are matched case-insensitively
Whitespace handling: directives with extra surrounding whitespace are correctly parsed
Incorrect value: max-image-preview:standard is treated as missing max-image-preview:large

Expected Outcome

Pass

The issue should be reported when:

The meta robots tag is completely absent
Any of the three required directives is missing
max-image-preview is present but not set to large

Fail

The issue should not be reported when:

All three directives (max-snippet:-1, max-image-preview:large, max-video-preview:-1) are present
Directives are in any order
Directive names use any case
Directives contain extra whitespace

Validation

Correct detection of missing meta robots tag
Correct identification of missing individual required directives
Correct detection of incorrect directive values (e.g., standard instead of large)
Case-insensitive directive name matching
Whitespace-tolerant directive parsing
Cache mechanism prevents duplicate issue entries on repeated calls

toggleGroups/htmlHeadTags.js
issueCodes.js

Coverage Summary

Positive cases: 4 (all directives, different order, case-insensitive, whitespace)
Negative cases: 5 (missing tag, missing max-snippet, missing max-image-preview, missing max-video-preview, wrong value)
Edge cases: 2 (empty HTML, case-insensitivity, whitespace)
Cache validation: 1

References

Robots meta tag — Google Search Central
Meta element — MDN
Snippets — Google Search Console Help

Robots meta tag missing snippet and preview directives

What Is This Issue

Why Is This Important

How XeoPix Detects This

How To Fix

What We Store

Storage Level

Database Table / Prisma Model

Stored Fields

Detection Dependencies

Examples

Example 1: Correct Implementation

Example 2: Missing Meta Robots Tag

Example 3: Incomplete Meta Robots Tag

Example 4: Restrictive Meta Robots Settings

Unit Test

Test File

Purpose

Tested Function

Issue Information

Test Scenarios

Positive Test Cases

Negative Test Cases

Boundary Cases

Edge Cases

Expected Outcome

Pass

Fail

Validation

Related Production Files

Coverage Summary

References