Stay organized with collections
Save and categorize content based on your preferences.
Overview of crawling and indexing topics
The topics in this section describe how you can control Google's ability to find and parse
your content in order to show it in Search and other Google properties, as well as how to
prevent Google from crawling specific content on your site.
Here's a brief description of each page. To get an overview of crawling and indexing, read
our How Search works guide.
Learn what URL canonicalization is and how to tell Google about any duplicate pages on
your site in order to avoid excessive crawling. Learn how Google auto-detects duplicate
content, how it treats duplicate content, and how it assigns a canonical URL to
any duplicate page groups found.
There are some differences and limitations that you need to account for when designing your
pages and applications to accommodate how crawlers access and render your content.