This page introduces the basic concepts behind Programmable Search Engine configuration files.
- What's in a Programmable Search Engine
- How the Components Work Together
- Creating a Search Engine
- Editing the Programmable Search Engine Files
- Choosing the Right Format
If the Control Panel does not give you the level of customization that you need, consider using the Programmable Search XML or TSV format, which gives you more control, flexibility, and access to more powerful features.
To use the Custom Search JSON API, start by creating a basic search engine using the Programmable Search Engine Control Panel. Once you've created your search engine, you can download your annotations and context XML files from the Advanced tab of the Control Panel.
Extensible Markup Language or XML is a general-purpose markup language. It is text with tags that you can read. For example, the Programmable Search XML format includes the following tags:
<Context> </Context> and
As with any XML file, your Programmable Search Engine specifications must follow XML syntax (
<element attribute="value">content</element>) and be well-formed. XML has the following rules:
- XML requires you to precede your top-level tags with an XML declaration (
<?xml version="1.0"?>), but the Custom Search JSON API doesn't require it.
- All your elements must have an opening tag (
<tag>) and a closing tag (
- All your tags must be properly nested. You cannot have XML code that looks like:
<sandwich><filling> peanut butter</sandwich></filling>. Instead, it should be like:
<sandwich><filling> peanut butter</filling></sandwich>.
- XML is case-sensitive, so carefully follow the capitalization and spelling of the tags in the instructions.
- All attribute values must be enclosed in double quotation marks (
- All attributes must be defined in the opening tag (
<element attribute="value">), not the closing tag (
You can write notes for yourself using comment tags (
<!-- your comment here -->), and Programmable Search Engine will not parse that line of text as XML code. Apart from writing reminders or description, you can use comments to temporarily put some XML code out of commission (perhaps because you want to experiment with certain effects or you want to troubleshoot issues). However, these comments are not preserved in the files that you download from the Control Panel. If you want to keep the comments, you should keep a copy of your commented XML files even after you upload them to the Control Panel.
You can use a simple text editor to create and edit XML files. Just save the text file with the file extension
.xml (for example,
The Programmable Search XML format is not hard to follow, but if you feel uncomfortable using it, you can use the Programmable Search Engine TSV (tab-separated values) format. As the name implies, a TSV file is a plain-text file that includes lines of fields (strings of characters) that are separated from each other by single tab stops. You can use a simple text editor or a spreadsheet editor to create and edit TSV files. Just save the text file with the file extension
.tsv (for example,
What's in a Programmable Search Engine
A Programmable Search Engine has two main components, each of which is controlled by an XML file:
- Context: The context XML file describes the basic features of a search engine. It specifies the global settings of the search engine, such as whether Image Search or promotions are enabled. In the Control Panel, these settings are defined in the Basics, Refinements, Look and feel, Admin accounts, and Make money tabs. Each search engine has its own context file. More information about the context XML file. For more information about selecting the most appropriate file format for your search engine, see Choosing the Right Format for Your Search Engine.
- Annotations: The annotations XML file lists the webpages or websites you want your search engine to cover, and indicates any preferences you have about how these sites should be ranked in your search results. Each site and its associated information is called an annotation. More information about the annotations XML file.
You can create an annotations file for each context file or you can have a single annotations file shared by all your search engines.
We don't recommend that you create either of these files from scratch. Instead, download them from the Advanced tab of the Control Panel. When you download annotations from the Control Panel, you will get a single annotations file that combines all the annotations from different search engines in your account.
In addition to these main components, a search engine can also have the following auxiliary files:
- Promotions: The promotions XML file lists a series of custom results that are triggered by a pre-defined set of query terms. When a user types a search that exactly matches one of your query terms, the promotion appears at the top of the page. You can use promotions to directly answer the queries of your users, lead them to important information, or point them to webpages that are not at the top of the results page yet are especially relevant. In the Control Panel, promotions are defined in the Promotions tab. More information about promotions.
- Synonyms: The synonyms XML file expands the queries of your users to include variants of the search term. For example, if your user searches for "simian," the search engine also searches for "monkey" and "ape." In the Control Panel, synonyms are defined in the Synonyms tab. More information about synonyms.
How the Components Work Together
The context XML file doesn't specify the annotations file to use, and the annotations XML makes no reference to the context file. Programmable Search Engine uses labels to associate context and annotations. The context XML file includes labels that identify the search engine, and each annotation listed in the annotations XML is tagged with one or more labels identifying the search engine(s) to which it belongs.If you change the name of the label in the context file, you have to change all the annotations that have been tagged with that label.
Although you can upload multiple annotations files, when you download them through the Control Panel, Programmable Search Engine merges all your annotations files into a single annotations file. Having a single annotations file for multiple search engines (with their own separate context files) simplifies your work and eliminates replication. It enables you to list sites only once, yet have the flexibility to customize the same site for various search engines. For example, one search engine could restrict its search to some sites, another could eliminate those sites, and yet another could promote those sites.
Here's an example of a context.xml file containing labels identifying the search engine to which it applies:
<BackgroundLabels> <Label name="_cse_hwbuiarvsbo" mode="FILTER"/> <Label name="_cse_exclude_hwbuiarvsbo" mode="ELIMINATE"/> <BackgroundLabels>
Here's an example of an annotations file showing how each site (annotation) is associated with a label:
<Annotation about="code.google.com/*" score="1"> <Label name="_cse_hwbuiarvsbo"/> </Annotation>
Creating Advanced Programmable Search Engines
Creating advanced engines involves the following steps:
- Determine the format that is appropriate for your needs.
- Define the specifications for your search engine.
- Tell Programmable Search Engine which sites to search.
- Tell Programmable Search Engine how to rank the search results.
Editing the Programmable Search Engine Files
To work on an XML file, download the XML specification from Advanced tab of the Control Panel. Don't start a file from scratch. Do the following:
- Download the context file or annotations file from the Advanced tab of the Control Panel. Click the Download button in the appropriate section.
You can choose to download the files to your hard drive or view them in another browser window or tab window.
- Use your browser to save the XML file, or copy the XML text from the webpage and paste it to your favorite text editor.
Use a text editor that can handle UNIX-style line endings (WordPad, Emacs, and TextMate works; NotePad doesn't). It does not matter what you name the file, so long as you save it with the file extension
- Make a backup copy of the downloaded file in case your edited version does not work as expected, and you have to revert to the previous version.
If you do not make a copy and the version that you edited does not work properly, you will need to debug your file or recreate your search engine all over again. Not fun.
- Edit the XML file and save it. Make sure that your text editor is saving the file as a Unicode text document and not some other file format.
- Upload the file under the correct section in the Advanced tab.
Choosing the Right Format
Before you start creating your Programmable Search Engine, determine which format best suits your needs. You don't want to select a format that is more powerful and complex than what you need, nor do you want to use one that you will quickly outgrow.
Use the following table to pick the appropriate format.
|To create||Use||Because||Limitations||More information|
|One or few search engines with a small number of sites||Control Panel||You can quickly create your Programmable Search Engine by filling out text boxes instead of creating files with a text editor and uploading the files.||The Control Panel is mostly useful for familiarizing yourself with Programmable Search Engine and creating search engines with few sites.
If you want to really customize your search engine or add a great number of sites, you might find the following quite limiting:
|Complex search engines that use lots of sites, use feeds, or are programmatically created||Context file and annotations files||
The Programmable Search Engine files give you a greater level of control over your search engines, and make the tasks of defining and managing sites a lot easier.
Even though you plan to create your search engine using context and annotations files, it's still a good idea to familiarize yourself with the Control Panel. The Preview tab lets you instantly view the results of your experimentation.
|The more you customize your search engine, the more complex it becomes. You have to learn the Programmable Search Elements and attributes, which are not hard to pick up, but they do require you to invest some time.
You will have to read the rest of the developer guide, which is not the most exciting reading material, unfortunately.
|Context: Defining a Search Engine Specifications and Annotations: Selecting Sites|