llms.txt Guide for AI Crawlers
Looking for Magento 2 OpenAI commerce integration? View the full Magento 2 OpenAI commerce integration page to explore features, configuration options, and pricing.
The OpenAI Commerce extension generates an llms.txt file — a Markdown document placed at your web root that guides AI crawlers and LLMs to your store's most important content. Think of it as robots.txt for AI: it tells language models where to find your store description, categories, CMS pages, and product feed.
How It Works
Each store view generates its own file with a store-code suffix:
| Store Code | Physical File | Public URL |
|---|---|---|
default |
pub/llms-default.txt |
https://store.com/llms.txt |
nl |
pub/llms-nl.txt |
https://store.nl/llms.txt |
A frontend router intercepts requests to the configured filename (default llms.txt) and serves the correct store-specific file based on the current store context. This means every store view can have its own content while all sharing the same clean public URL.
Configuration
Location: Magmodules → OpenAI - LLM Configuration → llms.txt
Main Settings
| Setting | Description |
|---|---|
| Enable | Enable llms.txt generation for this store view |
| Filename | The public URL path visitors request (default: llms.txt). The physical file automatically gets a store-code suffix (e.g. llms-default.txt) |
| Title | H1 heading in the generated file. Leave empty to use the store name |
| Description | Optional summary rendered as a blockquote below the title |
| Include Categories | Add links to categories from your navigation menu (up to 200) |
| Include CMS Pages | Add links to your CMS pages |
| Link Product Feed | Include a link to the OpenAI Commerce product feed |
Files Table
The admin table shows all store views with:
- File status — whether the file has been generated
- Generate action — regenerate the file for a specific store
- View action — open the generated file in a new tab (when available)
Cron
| Setting | Description |
|---|---|
| Enable | Automatically regenerate llms.txt files on a schedule |
| Frequency | Daily, or use a custom crontab expression |
| Custom Frequency | Crontab expression (e.g. 0 5,17 * * * for 5 AM and 5 PM) |
Generating Files
Via Admin
Use the Generate action in the Files table to generate the file for a specific store view.
Via CLI
bin/magento mm-openai:feed:create --type=llms --store-id=1
Via Cron
Enable the cron under Cron settings. Files are regenerated automatically for all enabled store views.
Multistore Setup
- Switch to the desired store view scope in admin
- Set Enable to Yes
- Configure the content settings (title, description, includes)
- Save and generate via the Files table
- Repeat for each store view
Each store view can have different content — for example, one store might include categories while another only includes CMS pages.
Example Output
# My Store
> Your one-stop shop for quality products
## Product Feed
- [OpenAI Commerce Feed](https://store.com/media/mm-openai/openai.json)
## Categories
- [Electronics](https://store.com/electronics)
- [Clothing](https://store.com/clothing)
## Pages
- [About Us](https://store.com/about-us)
- [Contact](https://store.com/contact)
Troubleshooting
| Issue | Solution |
|---|---|
| 404 when visiting llms.txt | Check that llms.txt is enabled for the store view and the file has been generated |
| File not updating | Regenerate manually via the Files table or check that cron is running |
| Wrong content for store | Verify you generated the file for the correct store view |
| File permissions error | Ensure pub/ directory is writable by the web server |
Need More Help?
Documentation:
- All Help Articles - Complete documentation overview
Support:
- Contact Support - Get help from our team
For a complete overview of features and configuration options, see Magento 2 OpenAI commerce integration.