Living Vicariously: Using Proxy Servers with the Google Data API Client Libraries

Jeff Fisher, Google Data APIs team
June 2007

Introduction: Why proxy?

A proxy server is a computer (or service on a computer) that makes requests for a number of client computers on their behalf, typically to external resources. This article is concerned with HTTP proxy servers as HTTP is the protocol used to access the public APIs for Google's web services. By extension, HTTPS or SSL proxies are also of interest when making HTTP requests that contain sensitive information such as private user data or passwords. Many large companies today use HTTP proxies to control what websites or information employees can view on the internet. Public libraries and schools have also been known to implement proxies for this purpose. There are also a number of publicly available proxy servers that can be used to anonymously access web content.

Potential issues faced when using a proxy server depend on what software is being used and how it is configured. A proxy is considered to be "transparent" if it does not alter the request from the client or the response from the server in any way other than is necessary for proxy identification and authentication. However, a large number of proxy servers do alter either the request or the response in ways that a developer should be aware of. In particular, certain proxies will alter the content-type of the response or strip out HTTP keep-alive headers from being sent to the outside server hosting the resource.

So why would a developer want to use an HTTP or SSL proxy? Generally, there are two reasons for this: it is required by some corporate infrastructure, or, the developer wishes to debug an application that uses a web service. The first reason is entirely unavoidable if the rules for the network the developer is working on prohibit non-proxied web or SSL connections to external websites. The latter reason is reported frequently in our support forums by developers trying to troubleshoot issues when dealing with a Google web service. There are special-purpose "debugging" proxies such as Fiddler and Charles that are geared towards this exact situation. For more information on this use of a proxy server, you may want to read our article On the Wire: Tools for API Developers.

For some applications, adding in proxy server support can be difficult. Fortunately, most of the client libraries for the Google Data API can be made to work with an HTTP proxy server after some slight code modifications. This article is intended to serve as a starting point for a developer who would like to use a proxy server for the web requests made by their application.

Java

Using an HTTP proxy with the Java client library is easy thanks to Sun's use of system properties to manage connection settings.

For example, if your corporate proxy server were running on "my.proxy.domain.com", on port 3128, you could add the following to your code before creating a service object for Google Calendar, Google Spreadsheets, etc.

System.setProperty("http.proxyHost", "my.proxy.domain.com");
System.setProperty("http.proxyPort", "3128");

Alternatively, this can be done on the command line when starting your servlet environment:

java -Dhttp.proxyHost=my.proxy.domain.com -Dhttp.proxyPort=3128

With more recent versions of the JSSE package, this can be extended to SSL proxies as well. If the same proxy server in the previous example was running an SSL proxy on port 3129, the necessary code would be:

System.setProperty("https.proxyHost", "my.proxy.domain.com");
System.setProperty("https.proxyPort", "3129");

This can also be done from the command line in the same fashion as with the HTTP proxy.

Sometimes, you might need to provide credentials to a proxy server in order to use it. Usually, they are submitted using a base64 hash included in an HTTP header, as follows:

String encoded = new String(Base64.encodeBase64(new String("username:password").getBytes()));
String base64encodedCredentials = "Basic " + encoded;
myService.getRequestFactory().setPrivateHeader("Proxy-Authorization", base64encodedCredentials);

Note that the above code uses the Apache Commons Codec package in order to do the base64 encoding necessary. You'll have to import the org.apache.commons.codec.binary.Base64 class in order to run the above code.

.NET

To use an HTTP proxy with the .NET client library is not as trivial as with the Java client, but it can be accomplished in a similar way when creating the service object for a particular product.

For example, we might want to use a proxy to interact with the Google Calendar service:

using System.Net;

CalendarService service = new CalendarService("CalendarSampleApp");
query.Uri = new Uri(calendarURI);
GDataRequestFactory requestFactory = (GDataRequestFactory) service.RequestFactory;
IWebProxy iProxy = WebRequest.DefaultWebProxy;
WebProxy myProxy = new WebProxy(iProxy.GetProxy(query.Uri));
// potentially, setup credentials on the proxy here
myProxy.Credentials = CredentialCache.DefaultCredentials;
myProxy.UseDefaultCredentials = true;
requestFactory.Proxy = myProxy;

This should detect the necessary proxy from your Internet connection settings--a nice feature of the .NET library. However, if you do not trust it to discover the proxy properly, you can also set it by changing the code to:

using System.Net;

CalendarService service = new CalendarService("CalendarSampleApp");
GDataRequestFactory requestFactory = (GDataRequestFactory) service.RequestFactory;
WebProxy myProxy = new WebProxy("http://my.proxy.example.com:3128/",true);
// potentially, setup credentials on the proxy here
myProxy.Credentials = CredentialCache.DefaultCredentials;
myProxy.UseDefaultCredentials = true;
requestFactory.Proxy = myProxy;

Conclusion

This article has discussed how to have some of the Google Data API client libraries work with an HTTP proxy server. Developers working behind a proxy server mandated by network policy can still use these libraries. Developers can also employ a proxy server to help debug their code by having the proxy server record the contents of HTTP requests and responses being sent to and from a Google web service. There are more advanced use cases of a proxy server and other client libraries not covered by this tutorial. Developers needing additional help are encouraged to participate in our public support groups linked below.

For additional information on the client libraries used in this article, visit the following pages:

Other Resources: