Configure the Robot Tags

Robot meta tags are essential for controlling how search engines crawl and index your Magento 2 store. This configuration screen allows you to define default robot directives for category pages, product pages, and CMS pages, giving you precise control over your SEO strategy.

Understanding Robot Tags

Robot tags instruct search engines on whether to index and follow links on specific pages. Common robot directives include:

• INDEX, FOLLOW: Allows indexing of the page and following the links.
• NOINDEX, FOLLOW: Prevents the page from being indexed but allows search engines to follow links on the page.
• INDEX, NOFOLLOW: The page is indexed, but links are not followed.
• NOINDEX, NOFOLLOW: Neither the page is indexed nor its links followed.

How to Configure Robot Tags

General Settings

Settings Section

Logging and Debugging

Best Practices for Robot Tags in Magento 2

• Use NOINDEX for duplicate or low-priority pages to optimize crawl budgets.
• Apply INDEX, FOLLOW to essential pages like product listings, categories, and high-conversion landing pages.
• Monitor pages using tools like Google Search Console to ensure the correct pages are indexed.