Webmaster Tools Developer's Guide - Java

Important: This is an old version of this page. For the latest version, use the links in the left-side navbar.

The Webmaster Tools Data API allows client applications to view and update information in Webmaster Tools. Your client application can use the Google Webmaster Tools Data API to:

  • View a list of sites in your account
  • Add and remove sites from your account
  • Verify site ownership
  • Modify site settings
  • Submit and delete Sitemaps
  • Manage messages sent to your account by Google via the Message Center in Webmaster Tools.

In addition to providing some background on the capabilities of the Webmaster Tools API, this document provides examples for interacting with the API using the Java client library. If you're interested in understanding more about the underlying protocol that the library uses to interact with the Webmaster Tools Data API, see the Developer's Guide.

Contents

  1. Audience
  2. Getting started
    1. Creating a Webmaster Tools account
    2. Running the sample code
  3. Authenticating to the Webmaster Tools service
    1. AuthSub proxy authentication
    2. ClientLogin username/password authentication
  4. Managing sites
    1. Retrieving a list of sites
    2. Adding a site
    3. Verifying site ownership
    4. Deleting sites
    5. Updating site settings
  5. Managing site settings
    1. Changing site settings
  6. Understanding keywords
    1. Requesting the Keywords feed
  7. Managing Sitemaps
    1. Retrieving a list of submitted Sitemaps
    2. Adding Sitemaps
    3. Deleting Sitemaps
  8. Managing Message Center messages
    1. Retrieving the Messages feed
    2. Marking a message as read or unread
    3. Deleting messages

Audience

This document is intended for programmers who want to write Java client applications that can interact with Webmaster Tools. It provides a series of examples of basic Data API interactions using the Java client library.

For Webmaster Tools Data API reference information, see the reference guide.

This document assumes that you understand the general ideas behind the Google Data APIs protocol.

Getting started

The Java client library requires Java 1.5. After downloading the client library, you'll find the classes you need to get started in the java/lib/gdataclient-1.0.jar file and the Webmaster Tools API.

Creating a Webmaster Tools account

You may want to sign up for a Webmaster Tools account for testing purposes. Webmaster Tools uses Google Accounts, so if you already have a Google account, you're all set.

Running the sample code

A working sample client, containing the sample code shown in this document, is available in the Java client library distribution under the directory gdata/java/sample/webmastertools/WebmasterToolsClient.java. Build and execution instructions are included in the same directory in the README.txt file.

The sample client performs several operations to demonstrate the use of the Webmaster Tools Data API.

To compile your code, such as in the samples in this document, you'll need to use the following import statements:

import com.google.gdata.client.webmastertools.WebmasterToolsService;
import com.google.gdata.data.OutOfLineContent;
import com.google.gdata.data.webmastertools.CrawlRate;
import com.google.gdata.data.webmastertools.DomainPreference;
import com.google.gdata.data.webmastertools.SitemapsEntry;
import com.google.gdata.data.webmastertools.SitemapsFeed;
import com.google.gdata.data.webmastertools.SitemapsRegularEntry;
import com.google.gdata.data.webmastertools.SitesEntry;
import com.google.gdata.data.webmastertools.SitesFeed;
import com.google.gdata.data.webmastertools.VerificationMethod;
import com.google.gdata.util.AuthenticationException;
import com.google.gdata.util.ServiceException;
import com.google.gdata.client.sample.util.SimpleCommandLineParser;
import java.io.IOException;
import java.net.MalformedURLException;
import java.net.URL;
import java.net.URLEncoder;

Authenticating to the Webmaster Tools service

You can only access private feeds using the Webmaster Tools Data API. If you want to modify your Webmaster Tools account then your client needs to authenticate before requesting private feeds. It can authenticate using either of two approaches: ClientLogin username/password authentication or AuthSub proxy authentication.

For a more detailed discussion of these authentication systems, see the Google Account Authentication document and the Google Data APIs Authentication Overview.

AuthSub proxy authentication

AuthSub proxy authentication is used by web applications that need to authenticate their users to Google accounts. The website operator and the client code don't have access to the username and password for the Webmaster Tools user; instead, the client obtains special AuthSub tokens that allow the client to act on a particular user's behalf. See the AuthSub documentation for more detailed information.

When a user first visits your application, they have not yet been authenticated. In this case, you need to display some information and a link directing the user to a Google page to authenticate your request for access to their (service object)s. The Java client library provides a function to generate the Google page's URL from the AuthSubUtil class. The code below retrieves the URL of the AuthSubRequest page:

String next = "http://www.example.com/welcome.html";
String scope = "http://www.google.com/webmasters/tools/feeds";
boolean secure = false;
boolean session = true;
String authSubLogin = AuthSubUtil.getRequestUrl(next, scope, secure, session);

The AuthSubRequest URL might look like this:

https://www.google.com/accounts/AuthSubRequest?scope=http%3A%2F%2Fwww.google.com%2Fwebmasters%2Ftools%2Ffeeds
   %2F&session=1&secure=0&next=http%3A%2F%2Fwww.example.com%2Fwelcome.html

The following query parameters are included in the AuthSubRequest URL:

scope
Indicates that the application is requesting a token to access Webmaster Tools feeds.
session
Indicates whether the token returned can be exchanged for a multi-use (session) token.
secure
Indicates whether the client is requesting a secure token.
next
The URL of the page that Google should redirect the user to after authentication.

After the user logs in, the AuthSub system redirects them to the URL you specified in the next query parameter of the AuthSubRequest URL. The AuthSub system appends an authentication token to that URL, as the value of the token query parameter. For example:

http://www.example.com/welcome.html?token=yourAuthToken

This token value represents a single-use AuthSub token. In this example, since session = true was specified, this token can be exchanged for an AuthSub session token by calling the AuthSubSessionToken service with the single-use token in an Authorization header, as follows, where urlFromAuthSub is the URL that AuthSub appended the token to:

String token = AuthSubUtil.getTokenFromReply(urlFromAuthSub);
String sessionToken = AuthSubUtil.exchangeForSessionToken(token, null);

Your application can then use the session token value in subsequent interactions with Webmaster Tools. The client library automatically sends the token along with requests.

ClientLogin username/password authentication

Use ClientLogin authentication if your client is a standalone, single-user "installed" client (such as a desktop application). Just call the setUserCredentials method on your WebmasterToolsService object and all subsequent interactions with Webmaster Tools will be authenticated:

WebmasterToolsService myService = new WebmasterToolsService("exampleCo-exampleApp-1");
myService.setUserCredentials("user@domain.com", "secretPassword");

In the snippet above we pass one parameter to the WebmasterToolsService constructor. The parameter is the name of our application in the form <companyName>-<applicationName>-<versionNumber>.

For more information about ClientLogin authentication, including sample requests and responses, see the Account Authentication for Installed Applications documentation.

Managing sites

Retrieving a list of sites

The Webmaster Tools Data API provides a feed that lists the sites for a particular user. The following sample code uses an authenticated WebmasterToolsService object to retrieve the metafeed and then prints each site's domain.

public static void printUserSites(WebmasterToolsService myService)
    throws ServiceException, IOException {
  try {
    System.out.println("Printing user's sites:");
    
    // Request the feed
    URL feedUrl = getSitesFeedUrl();
    SitesFeed sitesResultFeed = myService.getFeed(feedUrl, SitesFeed.class);
  
    // Print the results
    for (SitesEntry entry : sitesResultFeed.getEntries()) {
      System.out.println("\t" + entry.getTitle().getPlainText());
    }
  } catch (MalformedURLException e) {
    throw new IOException("URL for sites feed is malformed.");
  }
}

Adding a site

You can use the Java client library to add new sites to your account. First, create a SitesEntry object to represent the item. Then you can set the URL of the SitesEntry. Finally, use the WebmasterToolsService object to add the site. Here's an example of how to add a new site:

public static SitesEntry insertSite(WebmasterToolsService myService, 
    String siteUrl) throws IOException, ServiceException {
  SitesEntry entry = new SitesEntry();
  OutOfLineContent content = new OutOfLineContent();
  content.setUri(siteUrl);
  entry.setContent(content);
  System.out.println("Site: " + siteUrl + " now being added.");
  return myService.insert(getSitesFeedUrl(), entry);
}

The insert method used above takes the service's post URL as a parameter. Then the method returns the entry as it was stored by Webmaster Tools. The entry returned includes various elements added by Webmaster Tools, such as site information.

If your request fails for some reason, Webmaster Tools may return a different status code. For information about the status codes, see the protocol reference document.

Verifying site ownership

Webmaster Tools will provide key information only to users who have verified their ownership of the site(s). There are two methods of verification: using an HTML page, or using a meta tag. The HTML page file name and the meta tag value are unique to each site and are included in the Sites feed.

To verify using an HTML page, you will need to create an HTML page and upload it to the highest-level directory on your server. The file must have the filename specified in the Sites feed: 

<wt:verification-method type="htmlpage" in-use="false">456456-google.html</wt:verification-method>

To verify using a meta tag, you'll need to update your home page with the meta tag specified in the Sites feed: 

<wt:verification-method type="metatag" in-use="false"><meta name="verify-v1" content="a2Ai"/></wt:verification-method>

You can obtain your verification methods by calling the listVerificationValues on your SitesEntry. 

public static void listVerificationValues(WebmasterToolsService myService, 
    String siteUrl) throws IOException, ServiceException {
  // Request the entry
  String siteId = URLEncoder.encode(siteUrl, "UTF-8");
  URL feedUrl = new URL(getSitesFeedUrl() + siteId);
  SitesEntry entry = myService.getEntry(feedUrl, SitesEntry.class);
      
  // Print verification options
  for (VerificationMethod method : entry.getVerificationMethods()) {
    System.out.println("Verification method: " + method.getMethodType());
    if (method.getMethodType() == VerificationMethod.MethodType.METATAG) {
      System.out.println("Meta verification tag value: " + method.getValue());
    } else if (
      method.getMethodType() == VerificationMethod.MethodType.HTMLPAGE) {
      System.out.println("HTML verification page is: " + method.getValue());
    }
  }
}

Once you've created the file or updated your home page, you'll need to submit a verification request.

public static SitesEntry verifySite(WebmasterToolsService myService,
    String siteUrl) throws IOException, ServiceException {
  VerificationMethod method = new VerificationMethod();
  method.setMethodType(VerificationMethod.MethodType.HTMLPAGE);
  // Or method.setMethodType(VerificationMethod.MethodType.METATAG);
  method.setInUse(true);
  
  // Create the new SitesEntry to be verified
  SitesEntry entry = new SitesEntry();
  entry.addVerificationMethod(method);
  String siteId = URLEncoder.encode(siteUrl, "UTF-8");
  System.out.println("Now verifying site: " + siteUrl);
  URL updateUrl = new URL(getSitesFeedUrl() + siteId);
  return myService.update(updateUrl, entry);
}

Deleting sites

To delete a site, call the delete method on your SitesEntry object:

public static void deleteSite(WebmasterToolsService myService, String siteUrl)
    throws IOException, ServiceException {
  String siteId = URLEncoder.encode(siteUrl, "UTF-8");
  URL feedUrl = new URL(getSitesFeedUrl() + siteId);
  SitesEntry entry = myService.getEntry(feedUrl, SitesEntry.class);
  System.out.println("Now deleting site: " + siteUrl);
  entry.delete();
}

Managing site settings

Before you can see or update the current site settings for a particular site, you must verify your site. The site settings information is included in the sites feed and can be accessed using the methods in SitesEntry.

public static void printSiteSettings(WebmasterToolsService myService)
    throws IOException, ServiceException {
  try {
    System.out.println("Printing site settings:");

    // Get the sites feed
    URL feedUrl = getSitesFeedUrl();
    SitesFeed sitesResultFeed = myService.getFeed(feedUrl, SitesFeed.class);
    for (SitesEntry entry : sitesResultFeed.getEntries()) {
      // Print site name and settings
      System.out.println("\t" + entry.getTitle().getPlainText());
      System.out.println("\t\tGeographic location:"
        + entry.getGeolocation());
      System.out.println("\t\tDesired Crawl Rate:"
        + entry.getCrawlRate());
      System.out.println("\t\tPreferred Domain Association:"
        + entry.getPreferredDomain());
      System.out.println("\t\tEnhanced Image Search:"
        + entry.getEnhancedImageSearch());
    }
  } catch (MalformedURLException e) {
    throw new IOException("URL for site URL is malformed.");
  }
}

Changing site settings

To change the site settings, you must create a new SitesEntry for each desired update.

The entry returned is the same one you sent, but it also includes the new setting. You can only update one setting at a time. Entries with multiple updates, as well as unsuccessful or unauthorized updates, will receive an error. 

public static SitesEntry updateSiteSettings(WebmasterToolsService myService,
    String siteUrl) throws IOException, ServiceException {
  System.out.println("Site: " + siteUrl);
  String siteId = URLEncoder.encode(siteUrl, "UTF-8");
  URL updateUrl = new URL(getSitesFeedUrl() + siteId);

  try {
    // Update geographic location
    System.out.println("Updating geographic location...");
    SitesEntry entryUpdate = new SitesEntry();
    entryUpdate.setGeolocation(SAMPLE_LOCATION);
    myService.update(updateUrl, entryUpdate);

    // Update desired crawl rate
    System.out.println("Updating desired crawl rate...");
    entryUpdate = new SitesEntry();
    entryUpdate.setCrawlRate(SAMPLE_RATE);
    myService.update(updateUrl, entryUpdate);

    // Update preferred domain
    System.out.println("Updating preferred domain...");
    entryUpdate = new SitesEntry();
    entryUpdate.setPreferredDomain(SAMPLE_PREFERRED_DOMAIN);
    myService.update(updateUrl, entryUpdate);

    // Update enhanced image search and return the last updated entry
    System.out.println("Updating enhanced image search...");
    entryUpdate = new SitesEntry();
    entryUpdate.setEnhancedImageSearch(SAMPLE_ENHANCED_IMAGE_SEARCH);
    return myService.update(updateUrl, entryUpdate);
  } catch (ServiceException e) {
    System.out.println("Please make sure that the site to update "
      + "contains a trailing forward slash");
    throw(e);
  }
}

Understanding keywords

Retrieving the Keywords feed

To retrieve a list of keywords Google found on your site or in the anchor text of links to your site, call the same getFeed method used to retrieve the metafeed, but this time send the Keywords feed URL: 

public void printKeywords(String siteId) {
  // Get the feed URL
  URL feedUrl;
  try {
    if (!siteId.endsWith("/")) {
      siteId = siteId.concat("/");
    }
    String encodedSiteId = URLEncoder.encode(siteId, "UTF-8");
    feedUrl = new URL("http://www.google.com/webmasters/tools/feeds/"
        + encodedSiteId + "/keywords/");
  } catch (UnsupportedEncodingException e) {
    System.out.println("Encoding not supported");
    return;
  } catch (MalformedURLException e) {
    System.out.println("Malformed URL");
    return;
  }

  // Retrieve the keywords feed from the URL
  KeywordsFeed feed;
  try {
    feed = service.getFeed(feedUrl, KeywordsFeed.class);
  } catch (ServiceException e) {
    System.out.println("Service error");
    return;
  } catch (IOException e) {
    System.out.println("IO Error");
    return;
  }

  // Print the Keywords feed
  System.out.println("Keywords: ");
  for (Keyword keyword : feed.getKeywords()) {
    System.out.print("\tSource: " + keyword.getSource());
    System.out.println("\tKeyword: " + keyword.getValue());
  }
  System.out.println("Number of keywords: " + feed.getKeywords().size());
}

Managing Sitemaps

Retrieving a list of submitted Sitemaps

To retrieve all the Sitemaps for a site, call the same getFeed method used to retrieve the metafeed, but this time send the Sitemap feed URL:

public static void printUserSitemaps(WebmasterToolsService myService,
    String siteUrl) throws ServiceException, IOException {
  try {
    // Request the feed
    String siteId = URLEncoder.encode(siteUrl, "UTF-8");
    URL feedUrl = new
        URL("http://www.google.com/webmasters/tools/feeds/" + siteId +
        "/sitemaps/");
    SitemapsFeed resultFeed = myService.getFeed(feedUrl,
        SitemapsFeed.class, null);

    // Print the results
    for (int i = 0; i < resultFeed.getEntries().size(); i++) {
      SitemapsEntry entry = resultFeed.getEntries().get(i);
      System.out.println("\t" + entry.getTitle().getPlainText());
    }
  } catch (MalformedURLException e) {
    throw new IOException("URL for site URL is malformed.");
  }
}

Adding Sitemaps

Once you have created a Sitemap in the approved format, you can submit it to Google. You can use the Java client library to add new Sitemaps to your account. First, create a SitemapsEntry object to represent the Sitemap. Then you can set the url of the Sitemap. Finally, use the WebmasterToolsService object to add the Sitemap. Here's an example of how to add a new Sitemap:

    // Create the Sitemap entry to submit
    SitemapsRegularEntry entry = new SitemapsRegularEntry();

    // sitemapUrl is the URL of the Sitemap to be submitted
    entry.setId(sitemapUrl);
    entry.setSitemapType("WEB");
    String sitemapId = URLEncoder.encode(sitemapUrl, "UTF-8");

    // Submit the Sitemap  
    URL postURL = new URL("http://www.google.com/webmasters/tools/feeds/" + sitemapId + "/sitemaps");
    return myService.insert(postUrl, entry);

Deleting Sitemaps

To delete a Sitemap call the delete method on your SitemapsRegularEntry object (or SitemapsNewsEntry or SitemapsMobileEntry):

public static void deleteSitemap(WebmasterToolsService myService,
    String siteUrl, String sitemapUrl) throws IOException,
    ServiceException {
  String siteId = URLEncoder.encode(siteUrl, "UTF-8");
  String sitemapId = URLEncoder.encode(sitemapUrl, "UTF-8");
  URL feedUrl = new
      URL("http://www.google.com/webmasters/tools/feeds/" + siteId +
      "/sitemaps/" + sitemapId);
  SitemapsRegularEntry entry = myService.getEntry(feedUrl,
  SitemapsRegularEntry.class);
  entry.delete();
}

Back to top

Managing Message Center messages

Retrieving the Messages feed

The Webmaster Tools Data API provides a feed that lists messages sent by Google to the Message Center in Webmaster Tools. These messages contain important information about sites in the user's Webmaster Tools account. The following sample code uses an authenticated WebmasterToolsService object to retrieve the feed. 

public static MessagesFeed getMessagesFeed(WebmasterToolsService myService,
    String userLanguage) throws ServiceException, IOException {
  // Construct feed URL
  URL feedUrl;
  if (userLanguage == null) {
    feedUrl = new URL(MESSAGES_FEED_URI);
  } else {
    feedUrl = new URL(MESSAGES_FEED_URI + "?hl=" + userLanguage);
  }

  // Request feed from the service
  return myService.getFeed(feedUrl, MessagesFeed.class);
}

The following sample code prints and formats the retrieved messages on the screen, including the status (read or unread), date, and subject of each message.

private static void printMessagesFeed(MessagesFeed feed) {
  System.out.println(feed.getTitle().getPlainText());
  for (int i = 0; i < feed.getEntries().size(); i++) {
    MessageEntry entry = feed.getEntries().get(i);
    System.out.print(i + ":\t");
    if (entry.getRead()) {
      System.out.print("   \t");
    } else {
      System.out.print("new\t");
    }
    System.out.print(entry.getDate().toUiString() + "\t");
    System.out.println(entry.getSubject());
  }
}

Marking a message as read or unread

You can use the Java client library to mark the messages as read or unread.

  1. Create a MessageEntry instance with the updated information.
  2. Obtain the message URL (for messages, this is http://www.google.com/webmasters/tools/feeds/messages).
  3. Use the WebmasterToolsService object to update the message.

Use the following code:

public static URL getMessageUrl(String messageId, String userLanguage)
    throws UnsupportedEncodingException, MalformedURLException {
  // Get correctly encoded message Id
  if (!messageId.endsWith("/")) {
    messageId = messageId.concat("/");
  }
  String encodedMessageId = URLEncoder.encode(messageId, "UTF-8");

  // Construct message URL
  if (userLanguage == null) {
    return new URL(MESSAGES_FEED_URI + encodedMessageId);
  } else {
    return new URL(MESSAGES_FEED_URI + encodedMessageId + "?hl="
        + userLanguage);
  }
}

To update a message, using the message Id:

public static void markMessageAsRead(WebmasterToolsService service,
    String messageId) throws IOException, ServiceException {
  // Create update entry
  MessageEntry updateEntry = new MessageEntry();
  updateEntry.setRead(true);

  // Request update from service
  service.update(getMessageUrl(messageId, null), updateEntry);
}

Deleting messages

To delete a message, call the delete method on the previously retrieved MessageEntry instance:

public static void deleteMessage(WebmasterToolsService service,
    String messageId) throws IOException, ServiceException {
  // Retrieve entry and set to delete
  MessageEntry deleteEntry = service.getEntry(
      getMessageUrl(messageId, null), MessageEntry.class);
  deleteEntry.delete();
}