Important: This is an old version of this page. For the latest version, use the links in the left-side navbar.
The Webmaster Tools Data API allows client applications to view and update information in Webmaster Tools. Your client application can use the Google Webmaster Tools Data API to:
- View a list of sites in your account
- Add and remove sites from your account
- Verify site ownership
- Modify site settings
- Submit and delete Sitemaps
- Manage messages sent to your account by Google via the Message Center in Webmaster Tools.
In addition to providing some background on the capabilities of the Webmaster Tools API, this document provides examples for interacting with the API using the Java client library. If you're interested in understanding more about the underlying protocol that the library uses to interact with the Webmaster Tools Data API, see the Developer's Guide.
Contents
- Audience
- Getting started
- Authenticating to the Webmaster Tools service
- Managing sites
- Managing site settings
- Understanding keywords
- Managing Sitemaps
- Managing Message Center messages
Audience
This document is intended for programmers who want to write Java client applications that can interact with Webmaster Tools. It provides a series of examples of basic Data API interactions using the Java client library.
For Webmaster Tools Data API reference information, see the reference guide.
This document assumes that you understand the general ideas behind the Google Data APIs protocol.
Getting started
The Java client library requires Java 1.5. After downloading the client library, you'll find the classes you need to get started in the java/lib/gdataclient-1.0.jar
file and the Webmaster Tools API.
Creating a Webmaster Tools account
You may want to sign up for a Webmaster Tools account for testing purposes. Webmaster Tools uses Google Accounts, so if you already have a Google account, you're all set.
Running the sample code
A working sample client, containing the sample code shown in this
document, is available in the Java client library distribution under
the directory gdata/java/sample/webmastertools/WebmasterToolsClient.java
.
Build and execution instructions are included in the same directory in
the README.txt
file.
The sample client performs several operations to demonstrate the use of the Webmaster Tools Data API.
To compile your code, such as in the samples in this document, you'll need to use the following import statements:
import com.google.gdata.client.webmastertools.WebmasterToolsService;
import com.google.gdata.data.OutOfLineContent;
import com.google.gdata.data.webmastertools.CrawlRate;
import com.google.gdata.data.webmastertools.DomainPreference;
import com.google.gdata.data.webmastertools.SitemapsEntry;
import com.google.gdata.data.webmastertools.SitemapsFeed;
import com.google.gdata.data.webmastertools.SitemapsRegularEntry;
import com.google.gdata.data.webmastertools.SitesEntry;
import com.google.gdata.data.webmastertools.SitesFeed;
import com.google.gdata.data.webmastertools.VerificationMethod;
import com.google.gdata.util.AuthenticationException;
import com.google.gdata.util.ServiceException;
import com.google.gdata.client.sample.util.SimpleCommandLineParser;
import java.io.IOException;
import java.net.MalformedURLException;
import java.net.URL;
import java.net.URLEncoder;
Authenticating to the Webmaster Tools service
You can only access private feeds using the Webmaster Tools Data API. If you want to modify your Webmaster Tools account then your client needs to authenticate before requesting private feeds. It can authenticate using either of two approaches: ClientLogin username/password authentication or AuthSub proxy authentication.
For a more detailed discussion of these authentication systems, see the Google Account Authentication document and the Google Data APIs Authentication Overview.
AuthSub proxy authentication
AuthSub proxy authentication is used by web applications that need to authenticate their users to Google accounts. The website operator and the client code don't have access to the username and password for the Webmaster Tools user; instead, the client obtains special AuthSub tokens that allow the client to act on a particular user's behalf. See the AuthSub documentation for more detailed information.
When a user first visits your application, they have not yet been authenticated. In this case, you need to display some information and a link directing the user to a Google page to authenticate your request for access to their (service object)s. The Java client library provides a function to generate the Google page's URL from the AuthSubUtil
class. The code below retrieves the URL of the AuthSubRequest page:
String next = "http://www.example.com/welcome.html"; String scope = "http://www.google.com/webmasters/tools/feeds"; boolean secure = false; boolean session = true; String authSubLogin = AuthSubUtil.getRequestUrl(next, scope, secure, session);
The AuthSubRequest URL might look like this:
https://www.google.com/accounts/AuthSubRequest?scope=http%3A%2F%2Fwww.google.com%2Fwebmasters%2Ftools%2Ffeeds %2F&session=1&secure=0&next=http%3A%2F%2Fwww.example.com%2Fwelcome.html
The following query parameters are included in the AuthSubRequest URL:
- scope
- Indicates that the application is requesting a token to access Webmaster Tools feeds.
- session
- Indicates whether the token returned can be exchanged for a multi-use (session) token.
- secure
- Indicates whether the client is requesting a secure token.
- next
- The URL of the page that Google should redirect the user to after authentication.
After the user logs in, the AuthSub system redirects them to the URL you specified in the next
query parameter of the AuthSubRequest URL. The AuthSub system appends an authentication token to that URL, as the value of the token query parameter. For example:
http://www.example.com/welcome.html?token=yourAuthToken
This token value represents a single-use AuthSub token. In this example, since session = true
was specified, this token can be exchanged for an AuthSub session token by calling the AuthSubSessionToken
service with the single-use token in an Authorization header, as follows, where urlFromAuthSub
is the URL that AuthSub appended the token to:
String token = AuthSubUtil.getTokenFromReply(urlFromAuthSub); String sessionToken = AuthSubUtil.exchangeForSessionToken(token, null);
Your application can then use the session token value in subsequent interactions with Webmaster Tools. The client library automatically sends the token along with requests.
ClientLogin username/password authentication
Use ClientLogin authentication if your client is a standalone, single-user "installed" client (such as a desktop application). Just call the setUserCredentials
method on your WebmasterToolsService
object and all subsequent interactions with Webmaster Tools will be authenticated:
WebmasterToolsService myService = new WebmasterToolsService("exampleCo-exampleApp-1"); myService.setUserCredentials("user@domain.com", "secretPassword");
In the snippet above we pass one parameter to the WebmasterToolsService
constructor. The parameter is the name of our application in the form <companyName>-<applicationName>-<versionNumber>
.
For more information about ClientLogin authentication, including sample requests and responses, see the Account Authentication for Installed Applications documentation.
Managing sites
Retrieving a list of sites
The Webmaster Tools Data API provides a feed that lists the sites for a particular user. The following sample code uses an authenticated WebmasterToolsService
object to retrieve the metafeed and then prints each site's domain.
public static void printUserSites(WebmasterToolsService myService)
throws ServiceException, IOException {
try {
System.out.println("Printing user's sites:");
// Request the feed
URL feedUrl = getSitesFeedUrl();
SitesFeed sitesResultFeed = myService.getFeed(feedUrl, SitesFeed.class);
// Print the results
for (SitesEntry entry : sitesResultFeed.getEntries()) {
System.out.println("\t" + entry.getTitle().getPlainText());
}
} catch (MalformedURLException e) {
throw new IOException("URL for sites feed is malformed.");
}
}
Adding a site
You can use the Java client library to add new sites to your account. First, create a SitesEntry
object to represent the item
. Then you can set the URL of the SitesEntry. Finally, use the WebmasterToolsService
object to add the site. Here's an example of how to add a new site:
public static SitesEntry insertSite(WebmasterToolsService myService,
String siteUrl) throws IOException, ServiceException {
SitesEntry entry = new SitesEntry();
OutOfLineContent content = new OutOfLineContent();
content.setUri(siteUrl);
entry.setContent(content);
System.out.println("Site: " + siteUrl + " now being added.");
return myService.insert(getSitesFeedUrl(), entry);
}
The insert
method used above takes the service's post URL as a parameter. Then the method returns the entry as it was stored by Webmaster Tools. The entry returned includes various elements added by Webmaster Tools, such as site information.
If your request fails for some reason, Webmaster Tools may return a different status code. For information about the status codes, see the protocol reference document.
Verifying site ownership
Webmaster Tools will provide key information only to users who have verified their ownership of the site(s). There are two methods of verification: using an HTML page, or using a meta tag. The HTML page file name and the meta tag value are unique to each site and are included in the Sites feed.
To verify using an HTML page, you will need to create an HTML page and upload it to the highest-level directory on your server. The file must have the filename specified in the Sites feed:
<wt:verification-method type="htmlpage" in-use="false">456456-google.html</wt:verification-method>
To verify using a meta tag, you'll need to update your home page with the meta tag specified in the Sites feed:
<wt:verification-method type="metatag" in-use="false"><meta name="verify-v1" content="a2Ai"/></wt:verification-method>
You can obtain your verification methods by calling the
listVerificationValues
on your SitesEntry
.
public static void listVerificationValues(WebmasterToolsService myService,
String siteUrl) throws IOException, ServiceException {
// Request the entry
String siteId = URLEncoder.encode(siteUrl, "UTF-8");
URL feedUrl = new URL(getSitesFeedUrl() + siteId);
SitesEntry entry = myService.getEntry(feedUrl, SitesEntry.class);
// Print verification options
for (VerificationMethod method : entry.getVerificationMethods()) {
System.out.println("Verification method: " + method.getMethodType());
if (method.getMethodType() == VerificationMethod.MethodType.METATAG) {
System.out.println("Meta verification tag value: " + method.getValue());
} else if (
method.getMethodType() == VerificationMethod.MethodType.HTMLPAGE) {
System.out.println("HTML verification page is: " + method.getValue());
}
}
}
Once you've created the file or updated your home page, you'll need to submit a verification request.
public static SitesEntry verifySite(WebmasterToolsService myService,
String siteUrl) throws IOException, ServiceException {
VerificationMethod method = new VerificationMethod();
method.setMethodType(VerificationMethod.MethodType.HTMLPAGE);
// Or method.setMethodType(VerificationMethod.MethodType.METATAG);
method.setInUse(true);
// Create the new SitesEntry to be verified
SitesEntry entry = new SitesEntry();
entry.addVerificationMethod(method);
String siteId = URLEncoder.encode(siteUrl, "UTF-8");
System.out.println("Now verifying site: " + siteUrl);
URL updateUrl = new URL(getSitesFeedUrl() + siteId);
return myService.update(updateUrl, entry);
}
Deleting sites
To delete a site, call the delete
method on your SitesEntry
object:
public static void deleteSite(WebmasterToolsService myService, String siteUrl)
throws IOException, ServiceException {
String siteId = URLEncoder.encode(siteUrl, "UTF-8");
URL feedUrl = new URL(getSitesFeedUrl() + siteId);
SitesEntry entry = myService.getEntry(feedUrl, SitesEntry.class);
System.out.println("Now deleting site: " + siteUrl);
entry.delete();
}
Managing site settings
Before you can see or update the current site settings for a particular site, you must verify your site. The site settings information is included in the sites feed and can be accessed using the methods in SitesEntry
.
public static void printSiteSettings(WebmasterToolsService myService)
throws IOException, ServiceException {
try {
System.out.println("Printing site settings:");
// Get the sites feed
URL feedUrl = getSitesFeedUrl();
SitesFeed sitesResultFeed = myService.getFeed(feedUrl, SitesFeed.class);
for (SitesEntry entry : sitesResultFeed.getEntries()) {
// Print site name and settings
System.out.println("\t" + entry.getTitle().getPlainText());
System.out.println("\t\tGeographic location:"
+ entry.getGeolocation());
System.out.println("\t\tDesired Crawl Rate:"
+ entry.getCrawlRate());
System.out.println("\t\tPreferred Domain Association:"
+ entry.getPreferredDomain());
System.out.println("\t\tEnhanced Image Search:"
+ entry.getEnhancedImageSearch());
}
} catch (MalformedURLException e) {
throw new IOException("URL for site URL is malformed.");
}
}
Changing site settings
To change the site settings, you must create a new SitesEntry
for each desired update.
The entry returned is the same one you sent, but it also includes the new setting. You can only update one setting at a time. Entries with multiple updates, as well as unsuccessful or unauthorized updates, will receive an error.
public static SitesEntry updateSiteSettings(WebmasterToolsService myService,
String siteUrl) throws IOException, ServiceException {
System.out.println("Site: " + siteUrl);
String siteId = URLEncoder.encode(siteUrl, "UTF-8");
URL updateUrl = new URL(getSitesFeedUrl() + siteId);
try {
// Update geographic location
System.out.println("Updating geographic location...");
SitesEntry entryUpdate = new SitesEntry();
entryUpdate.setGeolocation(SAMPLE_LOCATION);
myService.update(updateUrl, entryUpdate);
// Update desired crawl rate
System.out.println("Updating desired crawl rate...");
entryUpdate = new SitesEntry();
entryUpdate.setCrawlRate(SAMPLE_RATE);
myService.update(updateUrl, entryUpdate);
// Update preferred domain
System.out.println("Updating preferred domain...");
entryUpdate = new SitesEntry();
entryUpdate.setPreferredDomain(SAMPLE_PREFERRED_DOMAIN);
myService.update(updateUrl, entryUpdate);
// Update enhanced image search and return the last updated entry
System.out.println("Updating enhanced image search...");
entryUpdate = new SitesEntry();
entryUpdate.setEnhancedImageSearch(SAMPLE_ENHANCED_IMAGE_SEARCH);
return myService.update(updateUrl, entryUpdate);
} catch (ServiceException e) {
System.out.println("Please make sure that the site to update "
+ "contains a trailing forward slash");
throw(e);
}
}
Understanding keywords
Retrieving the Keywords feed
To retrieve a list of keywords Google found on your site or in the anchor text of links to your site, call the same getFeed
method used to retrieve the metafeed, but this time send the Keywords feed URL:
public void printKeywords(String siteId) { // Get the feed URL URL feedUrl; try { if (!siteId.endsWith("/")) { siteId = siteId.concat("/"); } String encodedSiteId = URLEncoder.encode(siteId, "UTF-8"); feedUrl = new URL("http://www.google.com/webmasters/tools/feeds/" + encodedSiteId + "/keywords/"); } catch (UnsupportedEncodingException e) { System.out.println("Encoding not supported"); return; } catch (MalformedURLException e) { System.out.println("Malformed URL"); return; } // Retrieve the keywords feed from the URL KeywordsFeed feed; try { feed = service.getFeed(feedUrl, KeywordsFeed.class); } catch (ServiceException e) { System.out.println("Service error"); return; } catch (IOException e) { System.out.println("IO Error"); return; } // Print the Keywords feed System.out.println("Keywords: "); for (Keyword keyword : feed.getKeywords()) { System.out.print("\tSource: " + keyword.getSource()); System.out.println("\tKeyword: " + keyword.getValue()); } System.out.println("Number of keywords: " + feed.getKeywords().size()); }
Managing Sitemaps
Retrieving a list of submitted Sitemaps
To retrieve all the Sitemaps for a site, call the same getFeed
method used to retrieve the metafeed, but this time send the Sitemap feed URL:
public static void printUserSitemaps(WebmasterToolsService myService, String siteUrl) throws ServiceException, IOException { try { // Request the feed String siteId = URLEncoder.encode(siteUrl, "UTF-8"); URL feedUrl = new URL("http://www.google.com/webmasters/tools/feeds/" + siteId + "/sitemaps/"); SitemapsFeed resultFeed = myService.getFeed(feedUrl, SitemapsFeed.class, null); // Print the results for (int i = 0; i < resultFeed.getEntries().size(); i++) { SitemapsEntry entry = resultFeed.getEntries().get(i); System.out.println("\t" + entry.getTitle().getPlainText()); } } catch (MalformedURLException e) { throw new IOException("URL for site URL is malformed."); } }
Adding Sitemaps
Once you have created a Sitemap in the approved format, you can submit it to Google. You can use the Java client library to add new Sitemaps to your account. First, create a SitemapsEntry
object to represent the Sitemap. Then you can set the url of the Sitemap. Finally, use the WebmasterToolsService
object to add the Sitemap. Here's an example of how to add a new Sitemap:
// Create the Sitemap entry to submit SitemapsRegularEntry entry = new SitemapsRegularEntry(); // sitemapUrl is the URL of the Sitemap to be submitted entry.setId(sitemapUrl); entry.setSitemapType("WEB"); String sitemapId = URLEncoder.encode(sitemapUrl, "UTF-8"); // Submit the Sitemap URL postURL = new URL("http://www.google.com/webmasters/tools/feeds/" + sitemapId + "/sitemaps"); return myService.insert(postUrl, entry);
Deleting Sitemaps
To delete a Sitemap call the delete
method on your SitemapsRegularEntry
object (or SitemapsNewsEntry
or SitemapsMobileEntry
):
public static void deleteSitemap(WebmasterToolsService myService, String siteUrl, String sitemapUrl) throws IOException, ServiceException { String siteId = URLEncoder.encode(siteUrl, "UTF-8"); String sitemapId = URLEncoder.encode(sitemapUrl, "UTF-8"); URL feedUrl = new URL("http://www.google.com/webmasters/tools/feeds/" + siteId + "/sitemaps/" + sitemapId); SitemapsRegularEntry entry = myService.getEntry(feedUrl, SitemapsRegularEntry.class); entry.delete(); }
Managing Message Center messages
Retrieving the Messages feed
The Webmaster Tools Data API provides a feed that lists messages sent by Google to the Message Center in Webmaster Tools. These messages contain important information about sites in the user's Webmaster Tools account. The following sample code uses an authenticated WebmasterToolsService
object to retrieve the feed.
public static MessagesFeed getMessagesFeed(WebmasterToolsService myService, String userLanguage) throws ServiceException, IOException { // Construct feed URL URL feedUrl; if (userLanguage == null) { feedUrl = new URL(MESSAGES_FEED_URI); } else { feedUrl = new URL(MESSAGES_FEED_URI + "?hl=" + userLanguage); } // Request feed from the service return myService.getFeed(feedUrl, MessagesFeed.class); }
The following sample code prints and formats the retrieved messages on the screen, including the status (read or unread), date, and subject of each message.
private static void printMessagesFeed(MessagesFeed feed) { System.out.println(feed.getTitle().getPlainText()); for (int i = 0; i < feed.getEntries().size(); i++) { MessageEntry entry = feed.getEntries().get(i); System.out.print(i + ":\t"); if (entry.getRead()) { System.out.print(" \t"); } else { System.out.print("new\t"); } System.out.print(entry.getDate().toUiString() + "\t"); System.out.println(entry.getSubject()); } }
Marking a message as read or unread
You can use the Java client library to mark the messages as read or unread.
- Create a MessageEntry instance with the updated information.
- Obtain the message URL (for messages, this is
http://www.google.com/webmasters/tools/feeds/messages
). - Use the
WebmasterToolsService
object to update the message.
Use the following code:
public static URL getMessageUrl(String messageId, String userLanguage) throws UnsupportedEncodingException, MalformedURLException { // Get correctly encoded message Id if (!messageId.endsWith("/")) { messageId = messageId.concat("/"); } String encodedMessageId = URLEncoder.encode(messageId, "UTF-8"); // Construct message URL if (userLanguage == null) { return new URL(MESSAGES_FEED_URI + encodedMessageId); } else { return new URL(MESSAGES_FEED_URI + encodedMessageId + "?hl=" + userLanguage); } }
To update a message, using the message Id:
public static void markMessageAsRead(WebmasterToolsService service, String messageId) throws IOException, ServiceException { // Create update entry MessageEntry updateEntry = new MessageEntry(); updateEntry.setRead(true); // Request update from service service.update(getMessageUrl(messageId, null), updateEntry); }
Deleting messages
To delete a message, call the delete
method on the previously retrieved MessageEntry instance:
public static void deleteMessage(WebmasterToolsService service, String messageId) throws IOException, ServiceException { // Retrieve entry and set to delete MessageEntry deleteEntry = service.getEntry( getMessageUrl(messageId, null), MessageEntry.class); deleteEntry.delete(); }