No llms-full.txt published

What Is This Issue

This issue checks whether a website publishes an llms-full.txt file at the root domain, which is an extended version of llms.txt that provides complete page content for AI ingestion. This emerging standard helps AI systems understand and cite website content more effectively.

A passing implementation means:

An llms-full.txt file is accessible at https://example.com/llms-full.txt
The file contains structured content from the website in a format optimized for LLM consumption
The file is kept up-to-date with the site’s current content

Example location:


https://example.com/llms-full.txt

Why Is This Important

AI Content Ingestion: Unlike llms.txt which provides a summary, llms-full.txt provides complete content, allowing AI systems to have full context about your website.

Improved AI Citations: When AI systems have complete content access, they can provide more accurate citations and references to your pages.

Emerging Standard: As AI search engines and LLMs become more prevalent, having structured content files demonstrates technical sophistication and forward-thinking SEO.

Content Control: By providing an llms-full.txt file, you control exactly what content AI systems see, rather than relying on their crawlers to interpret your pages.

Competitive Advantage: Early adoption of this standard may provide advantages as AI search becomes more prominent.

SEO Health Score: Publishing llms-full.txt demonstrates comprehensive AI optimization and improves the technical SEO score.

How XeoPix Detects This

XeoPix performs the following checks:

Attempts to fetch the file - The crawler requests https://example.com/llms-full.txt (and common variations like llmsfull.txt).
Validates existence - XeoPix checks if the file returns a 200 OK status.
Validates content - The system performs basic validation to ensure the file:
- Contains actual content (not empty)
- Is in a text-based format (not binary)
- Has reasonable file size (not just a placeholder)
Checks format - While the format is still emerging, XeoPix may check for basic structure (markdown headings, clear sections, etc.).
Reports status - The issue passes if a valid llms-full.txt file is detected at the root domain; otherwise, it’s flagged as needing attention.

How To Fix

Understand the format - The llms-full.txt file should contain structured, complete content from your website in a format optimized for LLM parsing (typically markdown or plain text with clear structure).
Create the file - Generate an llms-full.txt file that includes:
- Complete page content (not just summaries)
- Clear headings and structure
- Important metadata (publication dates, authors, etc.)
- Clean, parseable formatting
Place at root domain - Upload the file to your website’s root directory so it’s accessible at:
```
https://example.com/llms-full.txt
```

Reference from llms.txt - If you have an llms.txt file, you can reference the full version:


# Website Name
> Summary description

## Full Content
- [Complete Content](llms-full.txt)

Keep updated - Ensure the file is regenerated when site content changes significantly.
Validate accessibility - Verify the file is accessible and returns proper content-type headers.

What We Store

Storage Level

Site Level — This issue is evaluated at the site/domain level.

Database Table / Prisma Model

SiteCrawlBehaviourData

Stored Fields

Field	Type	Description
robotsTxtData	Json?	Parsed robots.txt content including all directives

Detection Dependencies

The following data sources are required to evaluate this issue:
robots.txt — The crawler fetches and parses the robots.txt file to check for blank/disallowed content
HTTP Response — The crawler checks for robots.txt at https://domain.com/robots.txt

Examples

Example 1: Basic llms-full.txt

Problematic State (Fails): No llms-full.txt file exists. AI systems must crawl individual pages to understand content.

Corrected State (Passes): Create llms-full.txt at root domain:


# My Website
 
## Homepage
 
Welcome to my website. We provide SEO services...
 
## About Us
 
Our company was founded in 2020...
 
## Services
 
### SEO Audit
 
We provide comprehensive SEO audits...
 
### Content Marketing
 
Our content marketing services...

Example 2: Referencing from llms.txt

Problematic State (Fails): llms.txt exists but doesn’t reference the full version.

Corrected State (Passes): Add reference in llms.txt:


# My Website
 
> Complete SEO services and resources
 
## Full Content
 
- [Complete Site Content](llms-full.txt)
 
## Key Pages
 
- [Services](/services)
- [Blog](/blog)

Example 3: Keeping Content Updated

Problematic State (Fails): llms-full.txt is outdated and doesn’t reflect recent content changes.

Corrected State (Passes): Set up automated generation to keep llms-full.txt updated when content changes:

Integrate with CMS to regenerate on publish
Schedule weekly regeneration
Include latest blog posts and pages

Unit Test

Test File

xeopix-crawling-v2/__tests__/seo-audit-checks/crawlBehaviour/issue-199-llms-full-txt.test.js

Purpose

This unit test validates that the checkLlmsFullTxt() function correctly detects whether an llms-full.txt file exists at the root domain, reporting its status (found, not-found, or error).

Tested Function

checkLlmsFullTxt() from seo-audit-checks.js

Issue Information

Issue Number: 199
Issue Code: LLMS_FULL_TXT
Toggle Group: crawlBehaviour

Test Scenarios

Positive Test Cases

llms-full.txt exists and returns HTTP 200 with content → hasLlmsFullTxt is true, issue created with status found

Negative Test Cases

llms-full.txt is missing (HTTP 404) → hasLlmsFullTxt is false, issue created with status not-found

Boundary Cases

None

Edge Cases

Request failure (network error) → hasLlmsFullTxt is false, issue created with status error

Expected Outcome

Pass

An LLMS_FULL_TXT issue is always reported for this check, with status found when the file exists, not-found when it is missing, or error when the request fails.

Fail

This issue is always reported regardless of the outcome. The status field differentiates between found, not-found, and error.

Validation

Verifies detection when llms-full.txt exists with content
Validates detection when the file is missing (404)
Checks graceful handling of network request failures
Validates payload structure (hasLlmsFullTxt, details.status)

xeopix-crawling-v2/seo-audit-checks.js
xeopix-crawling-v2/issueCodes.js

Coverage Summary

3 test cases (found, not-found, and network error)
Covers all three possible status outcomes
Validates HTTP success, HTTP error, and network error scenarios

References

llms.txt Documentation — llms.txt Initiative
AI Search Optimization — Google Search Central