Stay organized with collections
Save and categorize content based on your preferences.
Wednesday, January 09, 2008
Confused about the best uses of robots.txt, nofollow, URL removal tool? Wondering
how to keep some of your pages off the web? Our webspam lead, Matt Cutts, talks about the best
ways to stop Google from crawling your content, and how to remove content from the Google index
once we've crawled it.
[[["Easy to understand","easyToUnderstand","thumb-up"],["Solved my problem","solvedMyProblem","thumb-up"],["Other","otherUp","thumb-up"]],[["Missing the information I need","missingTheInformationINeed","thumb-down"],["Too complicated / too many steps","tooComplicatedTooManySteps","thumb-down"],["Out of date","outOfDate","thumb-down"],["Samples / code issue","samplesCodeIssue","thumb-down"],["Other","otherDown","thumb-down"]],[],[],["Matt Cutts discusses how to prevent Google from crawling website content and how to remove content from the Google index after it's been crawled. The discussion covers the best practices for using robots.txt, `nofollow`, and the URL removal tool. Feedback on the video is encouraged through the Webmaster Help Group. Additional information on content removal can be found in the provided Help Center articles. Robots.txt has existed since at least 1996.\n"]]
