Configuring your robots.txt for Google Product Search

Please make sure that the user-agent 'googlebot' is not being blocked from crawling a merchant's website or product pages. If you do not add this line, or have  googlebot blocked, your items can be removed from Google Product Search listings. To ensure the 'googlebot' is not being blocked, please add the following two lines of text to the end of your robots.txt file:


User-agent: googlebot


Article is closed for comments.