Configure Sitemap & Robots in Magento 2

Do's:

• ✅ Submit your sitemap to Google Search Console
• ✅ Use realistic update frequencies that match actual content changes
• ✅ Include product images in your sitemap for visual search
• ✅ Enable noindex exclusion to keep your sitemap clean
• ✅ Set homepage priority to 1.0
• ✅ Regenerate sitemap after major catalog changes
• ✅ Use cron for automatic daily regeneration

Don'ts:

• ❌ Set all pages to priority 1.0 (defeats the purpose)
• ❌ Use "always" frequency for static pages
• ❌ Include noindex pages in your sitemap
• ❌ Forget to regenerate after URL changes
• ❌ Exceed 50,000 URLs per sitemap file

Assign priorities based on page importance:

Match frequency to actual content changes:

Do's:

• ✅ Use INDEX,FOLLOW for pages you want ranked
• ✅ Use NOINDEX,FOLLOW for filter pages and search results
• ✅ Keep internal link equity flowing with FOLLOW
• ✅ Set consistent robots across similar page types

Don'ts:

• ❌ Block important pages with NOINDEX
• ❌ Use NOFOLLOW on your own internal links
• ❌ Forget to check robots settings after updates

Use NOINDEX for pages that:

• Duplicate content (filtered category views)
• Have thin content (empty categories)
• Are utility pages (search results, account pages)
• Create crawl budget waste

Keep these pages as NOINDEX,FOLLOW so link equity still flows.

• Pages marked NOINDEX should be excluded from sitemap
• Enable "Exclude No-Index" options in sitemap settings
• Verify excluded pages don't appear in generated sitemap
• Check sitemap after changing robots defaults

1. Generate your sitemap via admin or CLI
2. Copy the sitemap URL from the Feeds section
3. Go to Google Search Console
4. Navigate to Sitemaps
5. Enter your sitemap URL and submit

Check Search Console regularly for:

• Indexing errors
• Coverage issues
• Sitemap processing status

For stores with 10,000+ products:

• Set appropriate file limits (50,000 URLs max)
• Use daily cron generation during low-traffic hours
• Consider category filtering if not all products need indexing

Optimize crawl budget by:

• Excluding low-value pages via NOINDEX
• Using realistic update frequencies
• Keeping sitemap focused on important pages