Conversational Actions were deprecated on June 13, 2023. For more information, see Conversational Actions sunset.

Helper intents (Dialogflow)

Helpers tell the Assistant to momentarily take over the conversation to obtain common data such as a user's full name, a date and time, or a delivery address. When you request a helper, the Assistant presents a standard, consistent UI to users to obtain this information, so you don't have to design your own.

Usage overview

The general process for using a helper with Dialogflow and Actions SDK is described below for Dialogflow and the Actions SDK. See the specific helper sections for more information about each helper.

Dialogflow

Node.js

Call conv.ask() with the appropriate helper object. When you call a helper function, the client library sends a response to the Assistant that contains the corresponding helper intent. Based on the intent, the Assistant knows to carry out the dialog for the corresponding helper.
Declare a Dialogflow intent that specifies an event that corresponds to one of the helper intents. See the helper intents section for a list of supported events. This intent doesn't need to have any User says phrases, because it's always triggered when the event is fired (when the Assistant is done carrying out the helper).
When the Assistant returns the result of the helper in the subsequent request to your fulfillment, the corresponding Dialogflow intent is triggered, and you handle the intent normally.

JSON

Specify the helper's intent in the possibleIntents object when responding to the Assistant. When the Assistant receives the response, it knows that it should carry out the dialog for the helper. See helper intents for information on what intents you can request to be fulfilled.
Declare a Dialogflow intent that specifies an event that corresponds to one of the helper intents. See the helper intents section for a list of supported events. This intent doesn't need to have any User says phrases, because it's always triggered when the event is fired.
When the Assistant returns the result of the helper in the subsequent request to your fulfillment, parse the request and the data you need.

Actions SDK

Node.js

Call ask with the appropriate parameters. A helper function asks the Assistant to fulfill one of the intents described in helper intents. When you call a helper function, the client library sends a response to the Assistant that contains one of these intents. Based on the intent, the Assistant knows to carry out the dialog for the corresponding helper.
When the Assistant returns the result of the helper in the subsequent request to your fulfillment, you receive the corresponding intent in the request. This lets you detect that a helper has returned a result. Use the corresponding getter function for the helper to obtain the data you need.

JSON

Specify the helper's intent in the possibleIntents object when responding to the Assistant. When the Assistant receives the response, it knows that it should carry out the dialog for the helper. See helper intents for information on what intents you can request to be fulfilled.
When the Assistant returns the result of the helper in the subsequent request to your fulfillment, parse the request and the data you need.

Helper intents

The following table describes the supported intents that you can request the Assistant to fulfill. If you're using Dialogflow, you also need to create a Dialogflow intent that specifies the corresponding event for the helper intent.

Intent name	Dialogflow Event name	Usage
`actions.intent.PERMISSION`	`actions_intent_PERMISSION`	Obtain the user's full name, coarse location, or precise location, or all 3.
`actions.intent.OPTION`	`actions_intent_OPTION`	Receive the selected item from a list or carousel UI. Or, if the user does not select from the list or carousel UI, receive speech or text input that matches with the key in the list or carousel UI.
`actions.intent.DATETIME`	`actions_intent_DATETIME`	Obtain a date and time input from the user.
`actions.intent.SIGN_IN`	`actions_intent_SIGN_IN`	Requests an account linking flow to link a user's account.
`actions.intent.PLACE`	`actions_intent_PLACE`	Obtain an address or saved location from the user.
`actions.intent.CONFIRMATION`	`actions_intent_CONFIRMATION`	Obtain a confirmation from the user (for example, an answer to a yes or no question).

The following sections describe the available helpers and the associated intent that you must request to use the helper.

User information

You can obtain the following user information with this helper:

Display name
Given name
Family name
Coarse device location (zip code and city)
Precise device location (coordinates and street address)

Calling the helper

The following code example shows how you can call the helper using the client library. The JSON snippets show the corresponding webhook response.

Node.js

app.intent('Permission', (conv) => {
  const permissions = ['NAME'];
  let context = 'To address you by name';
  // Location permissions only work for verified users
  // https://developers.google.com/actions/assistant/guest-users
  if (conv.user.verification === 'VERIFIED') {
    // Could use DEVICE_COARSE_LOCATION instead for city, zip code
    permissions.push('DEVICE_PRECISE_LOCATION');
    context += ' and know your location';
  }
  const options = {
    context,
    permissions,
  };
  conv.ask(new Permission(options));
});index.js

Java

@ForIntent("Permission")
public ActionResponse getPermission(ActionRequest request) {
  ResponseBuilder responseBuilder = getResponseBuilder(request);
  String[] permissions = new String[] {ConstantsKt.PERMISSION_NAME};
  String context = "To address you by name";
  // Location permissions only work for verified users
  // https://developers.google.com/actions/assistant/guest-users
  if (request.getUser().getUserVerificationStatus().equals("VERIFIED")) {
    // Could use PERMISSION_DEVICE_COARSE_LOCATION instead for city, zip code
    permissions =
        new String[] {
          ConstantsKt.PERMISSION_NAME, ConstantsKt.PERMISSION_DEVICE_PRECISE_LOCATION
        };
  }
  responseBuilder
      .add("PLACEHOLDER")
      .add(new Permission().setPermissions(permissions).setContext(context));

  return responseBuilder.build();
}MyActionsApp.java

Node.js

const permissions = ['NAME'];
let context = 'To address you by name';
// Location permissions only work for verified users
// https://developers.google.com/actions/assistant/guest-users
if (conv.user.verification === 'VERIFIED') {
  // Could use DEVICE_COARSE_LOCATION instead for city, zip code
  permissions.push('DEVICE_PRECISE_LOCATION');
  context += ' and know your location';
}
const options = {
  context,
  permissions,
};
conv.ask(new Permission(options));index.js

Java

ResponseBuilder responseBuilder = getResponseBuilder(request);
String[] permissions = new String[] {ConstantsKt.PERMISSION_NAME};
String context = "To address you by name";
// Location permissions only work for verified users
// https://developers.google.com/actions/assistant/guest-users
if (request.getUser().getUserVerificationStatus().equals("VERIFIED")) {
  // Could use PERMISSION_DEVICE_COARSE_LOCATION instead for city, zip code
  permissions =
      new String[] {
        ConstantsKt.PERMISSION_NAME, ConstantsKt.PERMISSION_DEVICE_PRECISE_LOCATION
      };
}
responseBuilder
    .add("PLACEHOLDER")
    .add(new Permission().setPermissions(permissions).setContext(context));

return responseBuilder.build();MyActionsApp.java

JSON

Note that the JSON below describes a webhook request.

{
  "payload": {
    "google": {
      "expectUserResponse": true,
      "systemIntent": {
        "intent": "actions.intent.PERMISSION",
        "data": {
          "@type": "type.googleapis.com/google.actions.v2.PermissionValueSpec",
          "optContext": "To address you by name and know your location",
          "permissions": [
            "NAME",
            "DEVICE_PRECISE_LOCATION"
          ]
        }
      }
    }
  }
}

JSON

Note that the JSON below describes a webhook request.

{
  "expectUserResponse": true,
  "expectedInputs": [
    {
      "possibleIntents": [
        {
          "intent": "actions.intent.PERMISSION",
          "inputValueData": {
            "@type": "type.googleapis.com/google.actions.v2.PermissionValueSpec",
            "optContext": "To address you by name and know your location",
            "permissions": [
              "NAME",
              "DEVICE_PRECISE_LOCATION"
            ]
          }
        }
      ]
    }
  ]
}

Getting the results of the helper

The following code example shows how to access the result of the helper using the client library. The JSON snippets represent the request, containing the result of the helper that your fulfillment will receive.

Node.js

app.intent('Permission Handler', (conv, params, confirmationGranted) => {
  // Also, can access latitude and longitude
  // const { latitude, longitude } = location.coordinates;
  const {location} = conv.device;
  const {name} = conv.user;
  if (confirmationGranted && name && location) {
    conv.ask(`Okay ${name.display}, I see you're at ` +
      `${location.formattedAddress}`);
  } else {
    conv.ask(`Looks like I can't get your information.`);
  }
  conv.ask(`Would you like to try another helper?`);
  conv.ask(new Suggestions([
    'Confirmation',
    'DateTime',
    'Place',
  ]));
});index.js

Java

@ForIntent("Permission Handler")
public ActionResponse handlePermission(ActionRequest request) {
  ResponseBuilder responseBuilder = getResponseBuilder(request);
  Location location = request.getDevice().getLocation();
  String name = request.getUser().getProfile().getDisplayName();

  if (request.isPermissionGranted()) {
    responseBuilder.add("Okay " + name + ", I see you're at " + location.getFormattedAddress());
  } else {
    responseBuilder.add("Looks like I can't get your information");
  }
  responseBuilder
      .add("Would you like to try another helper?")
      .addSuggestions(new String[] {"Confirmation", "DateTime", "Place"});

  return responseBuilder.build();
}MyActionsApp.java

Node.js

app.intent('actions.intent.PERMISSION', (conv, confirmationGranted) => {
  // Also, can access latitude and longitude
  // const { latitude, longitude } = location.coordinates;
  const {location} = conv.device;
  const {name} = conv.user;
  if (confirmationGranted && name && location) {
    conv.ask(`Okay ${name.display}, I see you're at ` +
      `${location.formattedAddress}`);
  } else {
    conv.ask(`Looks like I can't get your information.`);
  }
  conv.ask(`Would you like to try another helper?`);
  conv.ask(new Suggestions([
    'Confirmation',
    'DateTime',
    'Place',
  ]));
});index.js

Java

@ForIntent("actions.intent.PERMISSION")
public ActionResponse handlePermission(ActionRequest request) {
  ResponseBuilder responseBuilder = getResponseBuilder(request);
  Location location = request.getDevice().getLocation();
  String name = request.getUser().getProfile().getDisplayName();

  if (request.isPermissionGranted()) {
    responseBuilder.add("Okay " + name + ", I see you're at " + location.getFormattedAddress());
  } else {
    responseBuilder.add("Looks like I can't get your information");
  }
  responseBuilder
      .add("Would you like to try another helper?")
      .addSuggestions(new String[] {"Confirmation", "DateTime", "Place"});

  return responseBuilder.build();
}MyActionsApp.java

JSON

Note that the JSON below describes a webhook request.

{
  "responseId": "f26883c9-0283-4827-85bf-533f4442b4f9-712767ed",
  "queryResult": {
    "queryText": "actions_intent_PERMISSION",
    "parameters": {},
    "allRequiredParamsPresent": true,
    "fulfillmentText": "Webhook failed for intent: Permission Handler",
    "fulfillmentMessages": [
      {
        "text": {
          "text": [
            "Webhook failed for intent: Permission Handler"
          ]
        }
      }
    ],
    "outputContexts": [
      {
        "name": "projects/df-helpers-kohler/agent/sessions/ABwppHGu7m_m6bTsKuLb6noyG_o-9BMSf8J9cfY8dpJIPN_hHT79qfqRrZPtyx515K-sl9SU8Jy3auH6pLmy99Y/contexts/actions_capability_account_linking"
      },
      {
        "name": "projects/df-helpers-kohler/agent/sessions/ABwppHGu7m_m6bTsKuLb6noyG_o-9BMSf8J9cfY8dpJIPN_hHT79qfqRrZPtyx515K-sl9SU8Jy3auH6pLmy99Y/contexts/actions_capability_screen_output"
      },
      {
        "name": "projects/df-helpers-kohler/agent/sessions/ABwppHGu7m_m6bTsKuLb6noyG_o-9BMSf8J9cfY8dpJIPN_hHT79qfqRrZPtyx515K-sl9SU8Jy3auH6pLmy99Y/contexts/actions_capability_web_browser"
      },
      {
        "name": "projects/df-helpers-kohler/agent/sessions/ABwppHGu7m_m6bTsKuLb6noyG_o-9BMSf8J9cfY8dpJIPN_hHT79qfqRrZPtyx515K-sl9SU8Jy3auH6pLmy99Y/contexts/actions_capability_audio_output"
      },
      {
        "name": "projects/df-helpers-kohler/agent/sessions/ABwppHGu7m_m6bTsKuLb6noyG_o-9BMSf8J9cfY8dpJIPN_hHT79qfqRrZPtyx515K-sl9SU8Jy3auH6pLmy99Y/contexts/actions_capability_media_response_audio"
      },
      {
        "name": "projects/df-helpers-kohler/agent/sessions/ABwppHGu7m_m6bTsKuLb6noyG_o-9BMSf8J9cfY8dpJIPN_hHT79qfqRrZPtyx515K-sl9SU8Jy3auH6pLmy99Y/contexts/google_assistant_input_type_keyboard"
      },
      {
        "name": "projects/df-helpers-kohler/agent/sessions/ABwppHGu7m_m6bTsKuLb6noyG_o-9BMSf8J9cfY8dpJIPN_hHT79qfqRrZPtyx515K-sl9SU8Jy3auH6pLmy99Y/contexts/actions_intent_permission",
        "parameters": {
          "PERMISSION": true,
          "text": "yes"
        }
      }
    ],
    "intent": {
      "name": "projects/df-helpers-kohler/agent/intents/6475b0ef-dd52-4afa-a1b9-7a19e7e93a80",
      "displayName": "Permission Handler"
    },
    "intentDetectionConfidence": 1,
    "languageCode": "en"
  },
  "originalDetectIntentRequest": {
    "source": "google",
    "version": "2",
    "payload": {
      "user": {
        "profile": {
          "displayName": "Sachit Mishra",
          "givenName": "Sachit",
          "familyName": "Mishra"
        },
        "permissions": [
          "NAME",
          "DEVICE_PRECISE_LOCATION"
        ],
        "locale": "en-US",
        "userVerificationStatus": "VERIFIED"
      },
      "conversation": {
        "conversationId": "ABwppHGu7m_m6bTsKuLb6noyG_o-9BMSf8J9cfY8dpJIPN_hHT79qfqRrZPtyx515K-sl9SU8Jy3auH6pLmy99Y",
        "type": "ACTIVE",
        "conversationToken": "[]"
      },
      "inputs": [
        {
          "intent": "actions.intent.PERMISSION",
          "rawInputs": [
            {
              "inputType": "KEYBOARD",
              "query": "yes"
            }
          ],
          "arguments": [
            {
              "name": "PERMISSION",
              "boolValue": true,
              "textValue": "true"
            },
            {
              "name": "text",
              "rawText": "yes",
              "textValue": "yes"
            }
          ]
        }
      ],
      "surface": {
        "capabilities": [
          {
            "name": "actions.capability.ACCOUNT_LINKING"
          },
          {
            "name": "actions.capability.SCREEN_OUTPUT"
          },
          {
            "name": "actions.capability.WEB_BROWSER"
          },
          {
            "name": "actions.capability.AUDIO_OUTPUT"
          },
          {
            "name": "actions.capability.MEDIA_RESPONSE_AUDIO"
          }
        ]
      },
      "device": {
        "location": {
          "coordinates": {
            "latitude": 37.42241550000001,
            "longitude": -122.0840805
          },
          "formattedAddress": "Google Building 40 1600 Amphitheatre Parkway, Mountain View, California 94043",
          "zipCode": "94043",
          "city": "Mountain View"
        }
      },
      "availableSurfaces": [
        {
          "capabilities": [
            {
              "name": "actions.capability.WEB_BROWSER"
            },
            {
              "name": "actions.capability.SCREEN_OUTPUT"
            },
            {
              "name": "actions.capability.AUDIO_OUTPUT"
            }
          ]
        }
      ]
    }
  },
  "session": "projects/df-helpers-kohler/agent/sessions/ABwppHGu7m_m6bTsKuLb6noyG_o-9BMSf8J9cfY8dpJIPN_hHT79qfqRrZPtyx515K-sl9SU8Jy3auH6pLmy99Y"
}

JSON

Note that the JSON below describes a webhook request.

{
  "user": {
    "profile": {
      "displayName": "Sachit Mishra",
      "givenName": "Sachit",
      "familyName": "Mishra"
    },
    "permissions": [
      "DEVICE_PRECISE_LOCATION",
      "NAME"
    ],
    "locale": "en-US",
    "userVerificationStatus": "VERIFIED"
  },
  "conversation": {
    "conversationId": "ABwppHEcidO2jguO4hS8maDeMVx8aasSCcBWZHrF3EmOMIaWRP9qF0BcACistiIoeyiSNTW6KD_tqN_U-xAIOXhPvA",
    "type": "NEW"
  },
  "inputs": [
    {
      "intent": "actions.intent.PERMISSION",
      "rawInputs": [
        {
          "inputType": "KEYBOARD",
          "query": "yes"
        }
      ],
      "arguments": [
        {
          "name": "PERMISSION",
          "boolValue": true,
          "textValue": "true"
        },
        {
          "name": "text",
          "rawText": "yes",
          "textValue": "yes"
        }
      ]
    }
  ],
  "surface": {
    "capabilities": [
      {
        "name": "actions.capability.AUDIO_OUTPUT"
      },
      {
        "name": "actions.capability.MEDIA_RESPONSE_AUDIO"
      },
      {
        "name": "actions.capability.ACCOUNT_LINKING"
      },
      {
        "name": "actions.capability.SCREEN_OUTPUT"
      },
      {
        "name": "actions.capability.WEB_BROWSER"
      }
    ]
  },
  "device": {
    "location": {
      "coordinates": {
        "latitude": 37.4197238,
        "longitude": -122.08212759999999
      },
      "formattedAddress": "Google Building 45 1585 Charleston Road, Mountain View, California 94043",
      "zipCode": "94043",
      "city": "Mountain View"
    }
  },
  "availableSurfaces": [
    {
      "capabilities": [
        {
          "name": "actions.capability.SCREEN_OUTPUT"
        },
        {
          "name": "actions.capability.WEB_BROWSER"
        },
        {
          "name": "actions.capability.AUDIO_OUTPUT"
        }
      ]
    }
  ]
}

The snippet above shows how to check if the user granted you the information and then access the data.

Once you obtain the user's information, we recommend that you persist this information, so you don't have to ask again. You can use user storage to store user information across conversations. Check out our Name Psychic sample in Node.js and Java to see UserStorage in action.

List and Carousel Option

Calling the helper

The following code example shows how you can call the helper using the client library. The JSON snippets show the corresponding webhook response.

Node.js

app.intent('List', (conv) => {
  if (!conv.screen) {
    conv.ask('Sorry, try this on a screen device or select the ' +
      'phone surface in the simulator.');
    return;
  }

  conv.ask('This is a list example.');
  // Create a list
  conv.ask(new List({
    title: 'List Title',
    items: {
      // Add the first item to the list
      'SELECTION_KEY_ONE': {
        synonyms: [
          'synonym 1',
          'synonym 2',
          'synonym 3',
        ],
        title: 'Title of First List Item',
        description: 'This is a description of a list item.',
        image: new Image({
          url: 'https://storage.googleapis.com/actionsresources/logo_assistant_2x_64dp.png',
          alt: 'Image alternate text',
        }),
      },
      // Add the second item to the list
      'SELECTION_KEY_GOOGLE_HOME': {
        synonyms: [
          'Google Home Assistant',
          'Assistant on the Google Home',
      ],
        title: 'Google Home',
        description: 'Google Home is a voice-activated speaker powered by ' +
          'the Google Assistant.',
        image: new Image({
          url: 'https://storage.googleapis.com/actionsresources/logo_assistant_2x_64dp.png',
          alt: 'Google Home',
        }),
      },
      // Add the third item to the list
      'SELECTION_KEY_GOOGLE_PIXEL': {
        synonyms: [
          'Google Pixel XL',
          'Pixel',
          'Pixel XL',
        ],
        title: 'Google Pixel',
        description: 'Pixel. Phone by Google.',
        image: new Image({
          url: 'https://storage.googleapis.com/actionsresources/logo_assistant_2x_64dp.png',
          alt: 'Google Pixel',
        }),
      },
    },
  }));
});index.js

Java

@ForIntent("List")
public ActionResponse list(ActionRequest request) {
  ResponseBuilder responseBuilder = getResponseBuilder(request);
  if (!request.hasCapability(Capability.SCREEN_OUTPUT.getValue())) {
    return responseBuilder
        .add("Sorry, try ths on a screen device or select the phone surface in the simulator.")
        .add("Which response would you like to see next?")
        .build();
  }

  responseBuilder
      .add("This is a list example.")
      .add(
          new SelectionList()
              .setTitle("List Title")
              .setItems(
                  Arrays.asList(
                      new ListSelectListItem()
                          .setTitle("Title of First List Item")
                          .setDescription("This is a description of a list item.")
                          .setImage(
                              new Image()
                                  .setUrl(
                                      "https://storage.googleapis.com/actionsresources/logo_assistant_2x_64dp.png")
                                  .setAccessibilityText("Image alternate text"))
                          .setOptionInfo(
                              new OptionInfo()
                                  .setSynonyms(
                                      Arrays.asList("synonym 1", "synonym 2", "synonym 3"))
                                  .setKey("SELECTION_KEY_ONE")),
                      new ListSelectListItem()
                          .setTitle("Google Home")
                          .setDescription(
                              "Google Home is a voice-activated speaker powered by the Google Assistant.")
                          .setImage(
                              new Image()
                                  .setUrl(
                                      "https://storage.googleapis.com/actionsresources/logo_assistant_2x_64dp.png")
                                  .setAccessibilityText("Google Home"))
                          .setOptionInfo(
                              new OptionInfo()
                                  .setSynonyms(
                                      Arrays.asList(
                                          "Google Home Assistant",
                                          "Assistant on the Google Home"))
                                  .setKey("SELECTION_KEY_GOOGLE_HOME")),
                      new ListSelectListItem()
                          .setTitle("Google Pixel")
                          .setDescription("Pixel. Phone by Google.")
                          .setImage(
                              new Image()
                                  .setUrl(
                                      "https://storage.googleapis.com/actionsresources/logo_assistant_2x_64dp.png")
                                  .setAccessibilityText("Google Pixel"))
                          .setOptionInfo(
                              new OptionInfo()
                                  .setSynonyms(
                                      Arrays.asList("Google Pixel XL", "Pixel", "Pixel XL"))
                                  .setKey("SELECTION_KEY_GOOGLE_PIXEL")))));
  return responseBuilder.build();
}MyActionsApp.java

Node.js

if (!conv.screen) {
  conv.ask('Sorry, try this on a screen device or select the ' +
    'phone surface in the simulator.');
  return;
}

conv.ask('This is a list example.');
// Create a list
conv.ask(new List({
  title: 'List Title',
  items: {
    // Add the first item to the list
    'SELECTION_KEY_ONE': {
      synonyms: [
        'synonym 1',
        'synonym 2',
        'synonym 3',
      ],
      title: 'Title of First List Item',
      description: 'This is a description of a list item.',
      image: new Image({
        url: 'https://storage.googleapis.com/actionsresources/logo_assistant_2x_64dp.png',
        alt: 'Image alternate text',
      }),
    },
    // Add the second item to the list
    'SELECTION_KEY_GOOGLE_HOME': {
      synonyms: [
        'Google Home Assistant',
        'Assistant on the Google Home',
    ],
      title: 'Google Home',
      description: 'Google Home is a voice-activated speaker powered by ' +
        'the Google Assistant.',
      image: new Image({
        url: 'https://storage.googleapis.com/actionsresources/logo_assistant_2x_64dp.png',
        alt: 'Google Home',
      }),
    },
    // Add the third item to the list
    'SELECTION_KEY_GOOGLE_PIXEL': {
      synonyms: [
        'Google Pixel XL',
        'Pixel',
        'Pixel XL',
      ],
      title: 'Google Pixel',
      description: 'Pixel. Phone by Google.',
      image: new Image({
        url: 'https://storage.googleapis.com/actionsresources/logo_assistant_2x_64dp.png',
        alt: 'Google Pixel',
      }),
    },
  },
}));index.js

Java

ResponseBuilder responseBuilder = getResponseBuilder(request);
if (!request.hasCapability(Capability.SCREEN_OUTPUT.getValue())) {
  return responseBuilder
      .add("Sorry, try ths on a screen device or select the phone surface in the simulator.")
      .add("Which response would you like to see next?")
      .build();
}

responseBuilder
    .add("This is a list example.")
    .add(
        new SelectionList()
            .setTitle("List Title")
            .setItems(
                Arrays.asList(
                    new ListSelectListItem()
                        .setTitle("Title of First List Item")
                        .setDescription("This is a description of a list item.")
                        .setImage(
                            new Image()
                                .setUrl(
                                    "https://storage.googleapis.com/actionsresources/logo_assistant_2x_64dp.png")
                                .setAccessibilityText("Image alternate text"))
                        .setOptionInfo(
                            new OptionInfo()
                                .setSynonyms(
                                    Arrays.asList("synonym 1", "synonym 2", "synonym 3"))
                                .setKey("SELECTION_KEY_ONE")),
                    new ListSelectListItem()
                        .setTitle("Google Home")
                        .setDescription(
                            "Google Home is a voice-activated speaker powered by the Google Assistant.")
                        .setImage(
                            new Image()
                                .setUrl(
                                    "https://storage.googleapis.com/actionsresources/logo_assistant_2x_64dp.png")
                                .setAccessibilityText("Google Home"))
                        .setOptionInfo(
                            new OptionInfo()
                                .setSynonyms(
                                    Arrays.asList(
                                        "Google Home Assistant",
                                        "Assistant on the Google Home"))
                                .setKey("SELECTION_KEY_GOOGLE_HOME")),
                    new ListSelectListItem()
                        .setTitle("Google Pixel")
                        .setDescription("Pixel. Phone by Google.")
                        .setImage(
                            new Image()
                                .setUrl(
                                    "https://storage.googleapis.com/actionsresources/logo_assistant_2x_64dp.png")
                                .setAccessibilityText("Google Pixel"))
                        .setOptionInfo(
                            new OptionInfo()
                                .setSynonyms(
                                    Arrays.asList("Google Pixel XL", "Pixel", "Pixel XL"))
                                .setKey("SELECTION_KEY_GOOGLE_PIXEL")))));
return responseBuilder.build();MyActionsApp.java

JSON

Note that the JSON below describes a webhook response.

{
  "payload": {
    "google": {
      "expectUserResponse": true,
      "systemIntent": {
        "intent": "actions.intent.OPTION",
        "data": {
          "@type": "type.googleapis.com/google.actions.v2.OptionValueSpec",
          "listSelect": {
            "title": "List Title",
            "items": [
              {
                "optionInfo": {
                  "key": "SELECTION_KEY_ONE",
                  "synonyms": [
                    "synonym 1",
                    "synonym 2",
                    "synonym 3"
                  ]
                },
                "description": "This is a description of a list item.",
                "image": {
                  "url": "https://storage.googleapis.com/actionsresources/logo_assistant_2x_64dp.png",
                  "accessibilityText": "Image alternate text"
                },
                "title": "Title of First List Item"
              },
              {
                "optionInfo": {
                  "key": "SELECTION_KEY_GOOGLE_HOME",
                  "synonyms": [
                    "Google Home Assistant",
                    "Assistant on the Google Home"
                  ]
                },
                "description": "Google Home is a voice-activated speaker powered by the Google Assistant.",
                "image": {
                  "url": "https://storage.googleapis.com/actionsresources/logo_assistant_2x_64dp.png",
                  "accessibilityText": "Google Home"
                },
                "title": "Google Home"
              },
              {
                "optionInfo": {
                  "key": "SELECTION_KEY_GOOGLE_PIXEL",
                  "synonyms": [
                    "Google Pixel XL",
                    "Pixel",
                    "Pixel XL"
                  ]
                },
                "description": "Pixel. Phone by Google.",
                "image": {
                  "url": "https://storage.googleapis.com/actionsresources/logo_assistant_2x_64dp.png",
                  "accessibilityText": "Google Pixel"
                },
                "title": "Google Pixel"
              }
            ]
          }
        }
      },
      "richResponse": {
        "items": [
          {
            "simpleResponse": {
              "textToSpeech": "This is a list example."
            }
          }
        ]
      }
    }
  }
}

JSON

Note that the JSON below describes a webhook response.

{
  "expectUserResponse": true,
  "expectedInputs": [
    {
      "possibleIntents": [
        {
          "intent": "actions.intent.OPTION",
          "inputValueData": {
            "@type": "type.googleapis.com/google.actions.v2.OptionValueSpec",
            "listSelect": {
              "title": "List Title",
              "items": [
                {
                  "optionInfo": {
                    "key": "SELECTION_KEY_ONE",
                    "synonyms": [
                      "synonym 1",
                      "synonym 2",
                      "synonym 3"
                    ]
                  },
                  "description": "This is a description of a list item.",
                  "image": {
                    "url": "https://storage.googleapis.com/actionsresources/logo_assistant_2x_64dp.png",
                    "accessibilityText": "Image alternate text"
                  },
                  "title": "Title of First List Item"
                },
                {
                  "optionInfo": {
                    "key": "SELECTION_KEY_GOOGLE_HOME",
                    "synonyms": [
                      "Google Home Assistant",
                      "Assistant on the Google Home"
                    ]
                  },
                  "description": "Google Home is a voice-activated speaker powered by the Google Assistant.",
                  "image": {
                    "url": "https://storage.googleapis.com/actionsresources/logo_assistant_2x_64dp.png",
                    "accessibilityText": "Google Home"
                  },
                  "title": "Google Home"
                },
                {
                  "optionInfo": {
                    "key": "SELECTION_KEY_GOOGLE_PIXEL",
                    "synonyms": [
                      "Google Pixel XL",
                      "Pixel",
                      "Pixel XL"
                    ]
                  },
                  "description": "Pixel. Phone by Google.",
                  "image": {
                    "url": "https://storage.googleapis.com/actionsresources/logo_assistant_2x_64dp.png",
                    "accessibilityText": "Google Pixel"
                  },
                  "title": "Google Pixel"
                }
              ]
            }
          }
        }
      ],
      "inputPrompt": {
        "richInitialPrompt": {
          "items": [
            {
              "simpleResponse": {
                "textToSpeech": "This is a list example."
              }
            }
          ]
        }
      }
    }
  ]
}

You can display a list or carousel UI and obtain the selected option from the user with the actions.intent.OPTION intent.

Getting the results of the helper

The following code examples show how to access the result of the helper using the client library. The JSON snippets represent the request, containing the result of the helper that your fulfillment will receive.

The following snippets show how to check which option the user selected.

Node.js

app.intent('List - OPTION', (conv, params, option) => {
  const SELECTED_ITEM_RESPONSES = {
    'SELECTION_KEY_ONE': 'You selected the first item',
    'SELECTION_KEY_GOOGLE_HOME': 'You selected the Google Home!',
    'SELECTION_KEY_GOOGLE_PIXEL': 'You selected the Google Pixel!',
  };
  conv.ask(SELECTED_ITEM_RESPONSES[option]);
  conv.ask('Which response would you like to see next?');
});index.js

Java

@ForIntent("List - OPTION")
public ActionResponse listSelected(ActionRequest request) {
  ResponseBuilder responseBuilder = getResponseBuilder(request);
  String selectedItem = request.getSelectedOption();
  String response;

  if (selectedItem.equals("SELECTION_KEY_ONE")) {
    response = "You selected the first item";
  } else if (selectedItem.equals("SELECTION_KEY_GOOGLE_HOME")) {
    response = "You selected the Google Home!";
  } else if (selectedItem.equals("SELECTION_KEY_GOOGLE_PIXEL")) {
    response = "You selected the Google Pixel!";
  } else {
    response = "You did not select a valid item";
  }
  return responseBuilder.add(response).add("Which response would you like to see next?").build();
}MyActionsApp.java

Node.js

app.intent('actions.intent.OPTION', (conv, params, option) => {
  const SELECTED_ITEM_RESPONSES = {
    'SELECTION_KEY_ONE': 'You selected the first item',
    'SELECTION_KEY_GOOGLE_HOME': 'You selected the Google Home!',
    'SELECTION_KEY_GOOGLE_PIXEL': 'You selected the Google Pixel!',
  };
  conv.ask(SELECTED_ITEM_RESPONSES[option]);
  conv.ask('Which response would you like to see next?');
});index.js

Java

  @ForIntent("actions.intent.OPTION")
  public ActionResponse listSelected(ActionRequest request) {
    ResponseBuilder responseBuilder = getResponseBuilder(request);
    String selectedItem = request.getSelectedOption();
    String response;

    if (selectedItem.equals("SELECTION_KEY_ONE")) {
      response = "You selected the first item";
    } else if (selectedItem.equals("SELECTION_KEY_GOOGLE_HOME")) {
      response = "You selected the Google Home!";
    } else if (selectedItem.equals("SELECTION_KEY_GOOGLE_PIXEL")) {
      response = "You selected the Google Pixel!";
    } else {
      response = "You did not select a valid item";
    }
    return responseBuilder.add(response).add("Which response would you like to see next?").build();
  }

  public ActionResponse carousel(ActionRequest request) {
    ResponseBuilder responseBuilder = getResponseBuilder(request);
    if (!request.hasCapability(Capability.SCREEN_OUTPUT.getValue())) {
      return responseBuilder
          .add("Sorry, try ths on a screen device or select the phone surface in the simulator.")
          .add("Which response would you like to see next?")
          .build();
    }

    responseBuilder
        .add("This is a carousel example.")
        .add(
            new SelectionCarousel()
                .setItems(
                    Arrays.asList(
                        new CarouselSelectCarouselItem()
                            .setTitle("Title of First List Item")
                            .setDescription("This is a description of a list item.")
                            .setImage(
                                new Image()
                                    .setUrl(
                                        "https://storage.googleapis.com/actionsresources/logo_assistant_2x_64dp.png")
                                    .setAccessibilityText("Image alternate text"))
                            .setOptionInfo(
                                new OptionInfo()
                                    .setSynonyms(
                                        Arrays.asList("synonym 1", "synonym 2", "synonym 3"))
                                    .setKey("SELECTION_KEY_ONE")),
                        new CarouselSelectCarouselItem()
                            .setTitle("Google Home")
                            .setDescription(
                                "Google Home is a voice-activated speaker powered by the Google Assistant.")
                            .setImage(
                                new Image()
                                    .setUrl(
                                        "https://storage.googleapis.com/actionsresources/logo_assistant_2x_64dp.png")
                                    .setAccessibilityText("Google Home"))
                            .setOptionInfo(
                                new OptionInfo()
                                    .setSynonyms(
                                        Arrays.asList(
                                            "Google Home Assistant",
                                            "Assistant on the Google Home"))
                                    .setKey("SELECTION_KEY_GOOGLE_HOME")),
                        new CarouselSelectCarouselItem()
                            .setTitle("Google Pixel")
                            .setDescription("Pixel. Phone by Google.")
                            .setImage(
                                new Image()
                                    .setUrl(
                                        "https://storage.googleapis.com/actionsresources/logo_assistant_2x_64dp.png")
                                    .setAccessibilityText("Google Pixel"))
                            .setOptionInfo(
                                new OptionInfo()
                                    .setSynonyms(
                                        Arrays.asList("Google Pixel XL", "Pixel", "Pixel XL"))
                                    .setKey("SELECTION_KEY_GOOGLE_PIXEL")))));
    return responseBuilder.build();
  }
}
MyActionsApp.java

JSON

Note that the JSON below describes a webhook request.

{
  "responseId": "5d7732d1-d22d-4a0e-ad34-8bc0a7fde20c-21947381",
  "queryResult": {
    "queryText": "actions_intent_OPTION",
    "action": "List.List-custom",
    "parameters": {},
    "allRequiredParamsPresent": true,
    "fulfillmentText": "Webhook failed for intent: List - OPTION",
    "fulfillmentMessages": [
      {
        "text": {
          "text": [
            "Webhook failed for intent: List - OPTION"
          ]
        }
      }
    ],
    "outputContexts": [
      {
        "name": "projects/df-responses-kohler/agent/sessions/ABwppHHsebncupHK11oKhsCTgyH96GRNYH-xpeeMTqb-cvOxbd67QenbRlZM4bGAIB8_KXdTfI7-7lYVKN1ovAhCaA/contexts/actions_capability_screen_output"
      },
      {
        "name": "projects/df-responses-kohler/agent/sessions/ABwppHHsebncupHK11oKhsCTgyH96GRNYH-xpeeMTqb-cvOxbd67QenbRlZM4bGAIB8_KXdTfI7-7lYVKN1ovAhCaA/contexts/actions_capability_account_linking"
      },
      {
        "name": "projects/df-responses-kohler/agent/sessions/ABwppHHsebncupHK11oKhsCTgyH96GRNYH-xpeeMTqb-cvOxbd67QenbRlZM4bGAIB8_KXdTfI7-7lYVKN1ovAhCaA/contexts/actions_capability_media_response_audio"
      },
      {
        "name": "projects/df-responses-kohler/agent/sessions/ABwppHHsebncupHK11oKhsCTgyH96GRNYH-xpeeMTqb-cvOxbd67QenbRlZM4bGAIB8_KXdTfI7-7lYVKN1ovAhCaA/contexts/actions_capability_audio_output"
      },
      {
        "name": "projects/df-responses-kohler/agent/sessions/ABwppHHsebncupHK11oKhsCTgyH96GRNYH-xpeeMTqb-cvOxbd67QenbRlZM4bGAIB8_KXdTfI7-7lYVKN1ovAhCaA/contexts/actions_capability_web_browser"
      },
      {
        "name": "projects/df-responses-kohler/agent/sessions/ABwppHHsebncupHK11oKhsCTgyH96GRNYH-xpeeMTqb-cvOxbd67QenbRlZM4bGAIB8_KXdTfI7-7lYVKN1ovAhCaA/contexts/google_assistant_input_type_touch"
      },
      {
        "name": "projects/df-responses-kohler/agent/sessions/ABwppHHsebncupHK11oKhsCTgyH96GRNYH-xpeeMTqb-cvOxbd67QenbRlZM4bGAIB8_KXdTfI7-7lYVKN1ovAhCaA/contexts/list-followup",
        "lifespanCount": 1
      },
      {
        "name": "projects/df-responses-kohler/agent/sessions/ABwppHHsebncupHK11oKhsCTgyH96GRNYH-xpeeMTqb-cvOxbd67QenbRlZM4bGAIB8_KXdTfI7-7lYVKN1ovAhCaA/contexts/actions_intent_option",
        "parameters": {
          "OPTION": "SELECTION_KEY_GOOGLE_PIXEL",
          "text": "Google Pixel"
        }
      }
    ],
    "intent": {
      "name": "projects/df-responses-kohler/agent/intents/88904350-193e-4472-a2de-977eb5d9e26e",
      "displayName": "List - OPTION"
    },
    "intentDetectionConfidence": 1,
    "languageCode": "en"
  },
  "originalDetectIntentRequest": {
    "source": "google",
    "version": "2",
    "payload": {
      "user": {
        "locale": "en-US",
        "lastSeen": "2019-08-04T23:56:32Z",
        "userVerificationStatus": "VERIFIED"
      },
      "conversation": {
        "conversationId": "ABwppHHsebncupHK11oKhsCTgyH96GRNYH-xpeeMTqb-cvOxbd67QenbRlZM4bGAIB8_KXdTfI7-7lYVKN1ovAhCaA",
        "type": "ACTIVE",
        "conversationToken": "[\"list-followup\"]"
      },
      "inputs": [
        {
          "intent": "actions.intent.OPTION",
          "rawInputs": [
            {
              "inputType": "TOUCH",
              "query": "Google Pixel"
            }
          ],
          "arguments": [
            {
              "name": "OPTION",
              "textValue": "SELECTION_KEY_GOOGLE_PIXEL"
            },
            {
              "name": "text",
              "rawText": "Google Pixel",
              "textValue": "Google Pixel"
            }
          ]
        }
      ],
      "surface": {
        "capabilities": [
          {
            "name": "actions.capability.SCREEN_OUTPUT"
          },
          {
            "name": "actions.capability.ACCOUNT_LINKING"
          },
          {
            "name": "actions.capability.MEDIA_RESPONSE_AUDIO"
          },
          {
            "name": "actions.capability.AUDIO_OUTPUT"
          },
          {
            "name": "actions.capability.WEB_BROWSER"
          }
        ]
      },
      "isInSandbox": true,
      "availableSurfaces": [
        {
          "capabilities": [
            {
              "name": "actions.capability.WEB_BROWSER"
            },
            {
              "name": "actions.capability.SCREEN_OUTPUT"
            },
            {
              "name": "actions.capability.AUDIO_OUTPUT"
            }
          ]
        }
      ],
      "requestType": "SIMULATOR"
    }
  },
  "session": "projects/df-responses-kohler/agent/sessions/ABwppHHsebncupHK11oKhsCTgyH96GRNYH-xpeeMTqb-cvOxbd67QenbRlZM4bGAIB8_KXdTfI7-7lYVKN1ovAhCaA"
}

JSON

Note that the JSON below describes a webhook request.

{
  "user": {
    "locale": "en-US",
    "lastSeen": "2019-08-06T07:37:53Z",
    "userVerificationStatus": "VERIFIED"
  },
  "conversation": {
    "conversationId": "ABwppHGcqunXh1M6IE0lu2sVqXdpJfdpC5FWMkMSXQskK1nzb4IkSUSRqQzoEr0Ly0z_G3mwyZlk5rFtd1w",
    "type": "NEW"
  },
  "inputs": [
    {
      "intent": "actions.intent.OPTION",
      "rawInputs": [
        {
          "inputType": "TOUCH",
          "query": "Google Home"
        }
      ],
      "arguments": [
        {
          "name": "OPTION",
          "textValue": "SELECTION_KEY_GOOGLE_HOME"
        },
        {
          "name": "text",
          "rawText": "Google Home",
          "textValue": "Google Home"
        }
      ]
    }
  ],
  "surface": {
    "capabilities": [
      {
        "name": "actions.capability.AUDIO_OUTPUT"
      },
      {
        "name": "actions.capability.MEDIA_RESPONSE_AUDIO"
      },
      {
        "name": "actions.capability.ACCOUNT_LINKING"
      },
      {
        "name": "actions.capability.SCREEN_OUTPUT"
      },
      {
        "name": "actions.capability.WEB_BROWSER"
      }
    ]
  },
  "isInSandbox": true,
  "availableSurfaces": [
    {
      "capabilities": [
        {
          "name": "actions.capability.WEB_BROWSER"
        },
        {
          "name": "actions.capability.AUDIO_OUTPUT"
        },
        {
          "name": "actions.capability.SCREEN_OUTPUT"
        }
      ]
    }
  ],
  "requestType": "SIMULATOR"
}

Date and Time

You can obtain a date and time from users by requesting fulfillment of the actions.intent.DATETIME intent.

Calling the helper

The following code example shows how you can call the helper using the client library. The JSON snippets show the corresponding webhook response.

You can specify custom prompts when asking the user for a date and time using the options object when creating the DateTime permission.

Node.js

app.intent('Date Time', (conv) => {
  const options = {
    prompts: {
      initial: 'When would you like to schedule the appointment?',
      date: 'What day was that?',
      time: 'What time works for you?',
    },
  };
  conv.ask(new DateTime(options));
});index.js

Java

@ForIntent("Date Time")
public ActionResponse getDateTime(ActionRequest request) {
  ResponseBuilder responseBuilder = getResponseBuilder(request);

  responseBuilder
      .add("PLACEHOLDER")
      .add(
          new DateTimePrompt()
              .setDateTimePrompt("When would you like to schedule the appointment?")
              .setDatePrompt("What day was that?")
              .setTimePrompt("What time works for you?"));
  return responseBuilder.build();
}MyActionsApp.java

Node.js

const options = {
  prompts: {
    initial: 'When would you like to schedule the appointment?',
    date: 'What day was that?',
    time: 'What time works for you?',
  },
};
conv.ask(new DateTime(options));index.js

Java

ResponseBuilder responseBuilder = getResponseBuilder(request);

responseBuilder
    .add("PLACEHOLDER")
    .add(
        new DateTimePrompt()
            .setDateTimePrompt("When would you like to schedule the appointment?")
            .setDatePrompt("What day was that?")
            .setTimePrompt("What time works for you?"));
return responseBuilder.build();MyActionsApp.java

JSON

Note that the JSON below describes a webhook request.

{
  "payload": {
    "google": {
      "expectUserResponse": true,
      "systemIntent": {
        "intent": "actions.intent.DATETIME",
        "data": {
          "@type": "type.googleapis.com/google.actions.v2.DateTimeValueSpec",
          "dialogSpec": {
            "requestDatetimeText": "When would you like to schedule the appointment?",
            "requestDateText": "What day was that?",
            "requestTimeText": "What time works for you?"
          }
        }
      }
    }
  }
}

JSON

Note that the JSON below describes a webhook request.

{
  "expectUserResponse": true,
  "expectedInputs": [
    {
      "possibleIntents": [
        {
          "intent": "actions.intent.DATETIME",
          "inputValueData": {
            "@type": "type.googleapis.com/google.actions.v2.DateTimeValueSpec",
            "dialogSpec": {
              "requestDatetimeText": "When would you like to schedule the appointment?",
              "requestDateText": "What day was that?",
              "requestTimeText": "What time works for you?"
            }
          }
        }
      ]
    }
  ]
}

Getting the results of the helper

The snippet below shows how to check whether the user has granted access and how to access the data.

Node.js

app.intent('Date Time Handler', (conv, params, datetime) => {
  const {month, day} = datetime.date;
  const {hours, minutes} = datetime.time;
  conv.ask(`<speak>` +
      `Great, we will see you on ` +
      `<say-as interpret-as="date" format="dm">${day}-${month}</say-as>` +
      `<say-as interpret-as="time" format="hms12" detail="2">${hours}` +
      `:${minutes || '00'}</say-as>` +
      `</speak>`);
  conv.ask('Would you like to try another helper?');
  conv.ask(new Suggestions([
    'Confirmation',
    'Permission',
    'Place',
  ]));
});index.js

Java

@ForIntent("Date Time Handler")
public ActionResponse handleDateTime(ActionRequest request) {
  ResponseBuilder responseBuilder = getResponseBuilder(request);

  DateTime dateTimeValue = request.getDateTime();
  Integer day = dateTimeValue.getDate().getDay();
  Integer month = dateTimeValue.getDate().getMonth();
  Integer hours = dateTimeValue.getTime().getHours();
  Integer minutes = dateTimeValue.getTime().getMinutes();
  String minutesStr = (minutes != null) ? String.valueOf(minutes) : "00";
  responseBuilder.add(
      "<speak>"
          + "Great, we will see you on "
          + "<say-as interpret-as=\"date\" format=\"dm\">"
          + day
          + "-"
          + month
          + "</say-as>"
          + "<say-as interpret-as=\"time\" format=\"hms12\" detail=\"2\">"
          + hours
          + ":"
          + minutesStr
          + "</say-as>"
          + "</speak>");
  responseBuilder
      .add("Would you like to try another helper?")
      .addSuggestions(new String[] {"Confirmation", "Permission", "Place"});
  return responseBuilder.build();
}MyActionsApp.java

Node.js

app.intent('actions.intent.DATETIME', (conv) => {
  const datetime = conv.arguments.get('DATETIME');
  const {month, day} = datetime.date;
  const {hours, minutes} = datetime.time;
  conv.ask(`<speak>` +
      `Great, we will see you on ` +
      `<say-as interpret-as="date" format="dm">${day}-${month}</say-as>` +
      `<say-as interpret-as="time" format="hms12" detail="2">${hours}` +
      `:${minutes || '00'}</say-as>` +
      `</speak>`);
  conv.ask('Would you like to try another helper?');
  conv.ask(new Suggestions([
    'Confirmation',
    'Permission',
    'Place',
  ]));
});index.js

Java

@ForIntent("actions.intent.DATETIME")
public ActionResponse datetimeHandler(ActionRequest request) {
  ResponseBuilder responseBuilder = getResponseBuilder(request);

  DateTime dateTimeValue = request.getDateTime();
  Integer day = dateTimeValue.getDate().getDay();
  Integer month = dateTimeValue.getDate().getMonth();
  Integer hours = dateTimeValue.getTime().getHours();
  Integer minutes = dateTimeValue.getTime().getMinutes();
  String minutesStr = (minutes != null) ? String.valueOf(minutes) : "00";
  responseBuilder.add(
      "<speak>"
          + "Great, we will see you on "
          + "<say-as interpret-as=\"date\" format=\"dm\">"
          + day
          + "-"
          + month
          + "</say-as>"
          + "<say-as interpret-as=\"time\" format=\"hms12\" detail=\"2\">"
          + hours
          + ":"
          + minutesStr
          + "</say-as>"
          + "</speak>");
  responseBuilder
      .add("Would you like to try another helper?")
      .addSuggestions(new String[] {"Confirmation", "Permission", "Place"});
  return responseBuilder.build();
}MyActionsApp.java

JSON

Note that the JSON below describes a webhook request.

{
  "responseId": "2b0a039d-b033-4256-bd8a-ebb1f0ad90b4-712767ed",
  "queryResult": {
    "queryText": "actions_intent_DATETIME",
    "action": "DateTime.DateTime-custom",
    "parameters": {},
    "allRequiredParamsPresent": true,
    "fulfillmentText": "Webhook failed for intent: Date Time Handler",
    "fulfillmentMessages": [
      {
        "text": {
          "text": [
            "Webhook failed for intent: Date Time Handler"
          ]
        }
      }
    ],
    "outputContexts": [
      {
        "name": "projects/df-helpers-kohler/agent/sessions/ABwppHHnU7RSIOU994H8vd2wLYXbnoIAs3rk-4zUeiRAaX3aJ0dp68Y50MgvddE2stYuI6Hlu-MfWO75Gz76EeY/contexts/actions_capability_audio_output"
      },
      {
        "name": "projects/df-helpers-kohler/agent/sessions/ABwppHHnU7RSIOU994H8vd2wLYXbnoIAs3rk-4zUeiRAaX3aJ0dp68Y50MgvddE2stYuI6Hlu-MfWO75Gz76EeY/contexts/actions_capability_screen_output"
      },
      {
        "name": "projects/df-helpers-kohler/agent/sessions/ABwppHHnU7RSIOU994H8vd2wLYXbnoIAs3rk-4zUeiRAaX3aJ0dp68Y50MgvddE2stYuI6Hlu-MfWO75Gz76EeY/contexts/actions_capability_media_response_audio"
      },
      {
        "name": "projects/df-helpers-kohler/agent/sessions/ABwppHHnU7RSIOU994H8vd2wLYXbnoIAs3rk-4zUeiRAaX3aJ0dp68Y50MgvddE2stYuI6Hlu-MfWO75Gz76EeY/contexts/actions_capability_account_linking"
      },
      {
        "name": "projects/df-helpers-kohler/agent/sessions/ABwppHHnU7RSIOU994H8vd2wLYXbnoIAs3rk-4zUeiRAaX3aJ0dp68Y50MgvddE2stYuI6Hlu-MfWO75Gz76EeY/contexts/actions_capability_web_browser"
      },
      {
        "name": "projects/df-helpers-kohler/agent/sessions/ABwppHHnU7RSIOU994H8vd2wLYXbnoIAs3rk-4zUeiRAaX3aJ0dp68Y50MgvddE2stYuI6Hlu-MfWO75Gz76EeY/contexts/google_assistant_input_type_keyboard"
      },
      {
        "name": "projects/df-helpers-kohler/agent/sessions/ABwppHHnU7RSIOU994H8vd2wLYXbnoIAs3rk-4zUeiRAaX3aJ0dp68Y50MgvddE2stYuI6Hlu-MfWO75Gz76EeY/contexts/datetime-followup",
        "lifespanCount": 1
      },
      {
        "name": "projects/df-helpers-kohler/agent/sessions/ABwppHHnU7RSIOU994H8vd2wLYXbnoIAs3rk-4zUeiRAaX3aJ0dp68Y50MgvddE2stYuI6Hlu-MfWO75Gz76EeY/contexts/actions_intent_datetime",
        "parameters": {
          "DATETIME": "2019-08-14T10:30:00",
          "text": "1030am"
        }
      }
    ],
    "intent": {
      "name": "projects/df-helpers-kohler/agent/intents/3c145f18-32fe-40aa-842e-b99ab28c3f2a",
      "displayName": "Date Time Handler"
    },
    "intentDetectionConfidence": 1,
    "languageCode": "en"
  },
  "originalDetectIntentRequest": {
    "source": "google",
    "version": "2",
    "payload": {
      "user": {
        "locale": "en-US",
        "userVerificationStatus": "VERIFIED"
      },
      "conversation": {
        "conversationId": "ABwppHHnU7RSIOU994H8vd2wLYXbnoIAs3rk-4zUeiRAaX3aJ0dp68Y50MgvddE2stYuI6Hlu-MfWO75Gz76EeY",
        "type": "ACTIVE",
        "conversationToken": "[\"datetime-followup\"]"
      },
      "inputs": [
        {
          "intent": "actions.intent.DATETIME",
          "rawInputs": [
            {
              "inputType": "KEYBOARD",
              "query": "1030am"
            }
          ],
          "arguments": [
            {
              "name": "DATETIME",
              "datetimeValue": {
                "date": {
                  "year": 2019,
                  "month": 8,
                  "day": 14
                },
                "time": {
                  "hours": 10,
                  "minutes": 30
                }
              }
            },
            {
              "name": "text",
              "rawText": "1030am",
              "textValue": "1030am"
            }
          ]
        }
      ],
      "surface": {
        "capabilities": [
          {
            "name": "actions.capability.AUDIO_OUTPUT"
          },
          {
            "name": "actions.capability.SCREEN_OUTPUT"
          },
          {
            "name": "actions.capability.MEDIA_RESPONSE_AUDIO"
          },
          {
            "name": "actions.capability.ACCOUNT_LINKING"
          },
          {
            "name": "actions.capability.WEB_BROWSER"
          }
        ]
      },
      "availableSurfaces": [
        {
          "capabilities": [
            {
              "name": "actions.capability.AUDIO_OUTPUT"
            },
            {
              "name": "actions.capability.SCREEN_OUTPUT"
            },
            {
              "name": "actions.capability.WEB_BROWSER"
            }
          ]
        }
      ]
    }
  },
  "session": "projects/df-helpers-kohler/agent/sessions/ABwppHHnU7RSIOU994H8vd2wLYXbnoIAs3rk-4zUeiRAaX3aJ0dp68Y50MgvddE2stYuI6Hlu-MfWO75Gz76EeY"
}

JSON

Note that the JSON below describes a webhook request.

{
  "user": {
    "profile": {
      "displayName": "Sachit Mishra",
      "givenName": "Sachit",
      "familyName": "Mishra"
    },
    "permissions": [
      "DEVICE_PRECISE_LOCATION",
      "NAME"
    ],
    "locale": "en-US",
    "userVerificationStatus": "VERIFIED"
  },
  "conversation": {
    "conversationId": "ABwppHEcidO2jguO4hS8maDeMVx8aasSCcBWZHrF3EmOMIaWRP9qF0BcACistiIoeyiSNTW6KD_tqN_U-xAIOXhPvA",
    "type": "NEW"
  },
  "inputs": [
    {
      "intent": "actions.intent.DATETIME",
      "rawInputs": [
        {
          "inputType": "KEYBOARD",
          "query": "430pm"
        }
      ],
      "arguments": [
        {
          "name": "DATETIME",
          "datetimeValue": {
            "date": {
              "year": 2019,
              "month": 8,
              "day": 18
            },
            "time": {
              "hours": 16,
              "minutes": 30
            }
          }
        },
        {
          "name": "text",
          "rawText": "430pm",
          "textValue": "430pm"
        }
      ]
    }
  ],
  "surface": {
    "capabilities": [
      {
        "name": "actions.capability.ACCOUNT_LINKING"
      },
      {
        "name": "actions.capability.WEB_BROWSER"
      },
      {
        "name": "actions.capability.AUDIO_OUTPUT"
      },
      {
        "name": "actions.capability.MEDIA_RESPONSE_AUDIO"
      },
      {
        "name": "actions.capability.SCREEN_OUTPUT"
      }
    ]
  },
  "device": {
    "location": {
      "coordinates": {
        "latitude": 37.4197238,
        "longitude": -122.08212759999999
      },
      "formattedAddress": "Google Building 45 1585 Charleston Road, Mountain View, California 94043",
      "zipCode": "94043",
      "city": "Mountain View"
    }
  },
  "availableSurfaces": [
    {
      "capabilities": [
        {
          "name": "actions.capability.AUDIO_OUTPUT"
        },
        {
          "name": "actions.capability.SCREEN_OUTPUT"
        },
        {
          "name": "actions.capability.WEB_BROWSER"
        }
      ]
    }
  ]
}

Account Sign-in

You can have users sign-in to their accounts that are associated with your service by requesting fulfillment of the actions.intent.SIGN_IN intent. Users cannot sign-in over voice through OAuth.

Dialogflow: In order to get the results of the sign-in helper when using Dialogflow, the intent handling the sign-in needs to have the actions_intent_SIGN_IN event.

Calling the helper

The following code example shows how you can call the helper using the client library. The JSON snippets show the corresponding webhook response.

Node.js

const {SignIn} = require('actions-on-google');
app.intent('ask_for_sign_in_detail', (conv) => {
  conv.ask(new SignIn());
});

Java

ResponseBuilder responseBuilder = getResponseBuilder(request);
return responseBuilder.add("This is the SignIn helper intent").add(new SignIn()).build();

JSON

{
  "payload": {
    "google": {
      "expectUserResponse": true,
      "systemIntent": {
        "intent": "actions.intent.SIGN_IN",
        "data": {
          "@type": "type.googleapis.com/google.actions.v2.SignInValueSpec"
        }
      }
    }
  }
}

JSON

{
  "expectUserResponse": true,
  "expectedInputs": [
    {
      "possibleIntents": [
        {
          "intent": "actions.intent.SIGN_IN",
          "inputValueData": {
            "@type": "type.googleapis.com/google.actions.v2.SignInValueSpec"
          }
        }
      ]
    }
  ]
}

Getting the results of the helper

The snippet below shows how to check whether the user has granted access and how to access the data.

Node.js

app.intent('ask_for_sign_in_confirmation', (conv, params, signin) => {
  if (signin.status !== 'OK') {
    return conv.ask('You need to sign in before using the app.');
  }
  // const access = conv.user.access.token;
  // possibly do something with access token
  return conv.ask('Great! Thanks for signing in.');
});

Java

ResponseBuilder responseBuilder = getResponseBuilder(request);
boolean signedIn = request.isSignInGranted();
return responseBuilder
    .add(signedIn ? "Great! Thanks for signin in." : "You must sign in before using the app.")
    .build();

JSON

{
  "responseId": "",
  "queryResult": {
    "queryText": "",
    "action": "",
    "parameters": {},
    "allRequiredParamsPresent": true,
    "fulfillmentText": "",
    "fulfillmentMessages": [],
    "outputContexts": [],
    "intent": {
      "name": "ask_for_sign_in_confirmation",
      "displayName": "ask_for_sign_in_confirmation"
    },
    "intentDetectionConfidence": 1,
    "diagnosticInfo": {},
    "languageCode": ""
  },
  "originalDetectIntentRequest": {
    "source": "google",
    "version": "2",
    "payload": {
      "isInSandbox": true,
      "surface": {
        "capabilities": [
          {
            "name": "actions.capability.SCREEN_OUTPUT"
          },
          {
            "name": "actions.capability.AUDIO_OUTPUT"
          },
          {
            "name": "actions.capability.MEDIA_RESPONSE_AUDIO"
          },
          {
            "name": "actions.capability.WEB_BROWSER"
          }
        ]
      },
      "inputs": [
        {
          "rawInputs": [],
          "intent": "",
          "arguments": [
            {
              "name": "SIGN_IN",
              "extension": {
                "@type": "type.googleapis.com/google.actions.v2.SignInValue",
                "status": "OK"
              }
            }
          ]
        }
      ],
      "user": {},
      "conversation": {},
      "availableSurfaces": [
        {
          "capabilities": [
            {
              "name": "actions.capability.SCREEN_OUTPUT"
            },
            {
              "name": "actions.capability.AUDIO_OUTPUT"
            },
            {
              "name": "actions.capability.MEDIA_RESPONSE_AUDIO"
            },
            {
              "name": "actions.capability.WEB_BROWSER"
            }
          ]
        }
      ]
    }
  },
  "session": ""
}

JSON

{
  "user": {},
  "device": {},
  "surface": {
    "capabilities": [
      {
        "name": "actions.capability.SCREEN_OUTPUT"
      },
      {
        "name": "actions.capability.AUDIO_OUTPUT"
      },
      {
        "name": "actions.capability.MEDIA_RESPONSE_AUDIO"
      },
      {
        "name": "actions.capability.WEB_BROWSER"
      }
    ]
  },
  "conversation": {},
  "inputs": [
    {
      "rawInputs": [],
      "intent": "ask_for_sign_in_confirmation",
      "arguments": [
        {
          "name": "SIGN_IN",
          "extension": {
            "@type": "type.googleapis.com/google.actions.v2.SignInValue",
            "status": "OK"
          }
        }
      ]
    }
  ],
  "availableSurfaces": [
    {
      "capabilities": [
        {
          "name": "actions.capability.SCREEN_OUTPUT"
        },
        {
          "name": "actions.capability.AUDIO_OUTPUT"
        },
        {
          "name": "actions.capability.MEDIA_RESPONSE_AUDIO"
        },
        {
          "name": "actions.capability.WEB_BROWSER"
        }
      ]
    }
  ]
}

Place and Location

You can obtain a location from users by requesting fulfillment of the actions.intent.PLACE intent. This helper is used to prompt the user for addresses and other locations, including any home/work/contact locations that they've saved with Google.

Saved locations will only return the address, not the associated mapping (e.g. "123 Main St" as opposed to "HOME = 123 Main St").

Calling the helper

The following code example shows how you can call the helper using the client library. The JSON snippets show the corresponding webhook response.

Node.js

app.intent('Place', (conv) => {
  const options = {
    context: 'To find a location',
    prompt: 'Where would you like to go?',
  };
  conv.ask(new Place(options));
});index.js

Java

@ForIntent("Place")
public ActionResponse getPlace(ActionRequest request) {
  ResponseBuilder responseBuilder = getResponseBuilder(request);
  responseBuilder
      .add("PLACEHOLDER")
      .add(
          new Place()
              .setRequestPrompt("Where would you like to go?")
              .setPermissionContext("To find  a location"));
  return responseBuilder.build();
}MyActionsApp.java

Node.js

const options = {
  context: 'To find a location',
  prompt: 'Where would you like to go?',
};
conv.ask(new Place(options));index.js

Java

ResponseBuilder responseBuilder = getResponseBuilder(request);
responseBuilder
    .add("PLACEHOLDER")
    .add(
        new Place()
            .setRequestPrompt("Where would you like to go?")
            .setPermissionContext("To find  a location"));
return responseBuilder.build();MyActionsApp.java

JSON

Note that the JSON below describes a webhook request.

{
  "payload": {
    "google": {
      "expectUserResponse": true,
      "systemIntent": {
        "intent": "actions.intent.PLACE",
        "data": {
          "@type": "type.googleapis.com/google.actions.v2.PlaceValueSpec",
          "dialogSpec": {
            "extension": {
              "@type": "type.googleapis.com/google.actions.v2.PlaceValueSpec.PlaceDialogSpec",
              "permissionContext": "To find a location",
              "requestPrompt": "Where would you like to go?"
            }
          }
        }
      }
    }
  }
}

JSON

Note that the JSON below describes a webhook request.

{
  "expectUserResponse": true,
  "expectedInputs": [
    {
      "possibleIntents": [
        {
          "intent": "actions.intent.PLACE",
          "inputValueData": {
            "@type": "type.googleapis.com/google.actions.v2.PlaceValueSpec",
            "dialogSpec": {
              "extension": {
                "@type": "type.googleapis.com/google.actions.v2.PlaceValueSpec.PlaceDialogSpec",
                "permissionContext": "To find a location",
                "requestPrompt": "Where would you like to go?"
              }
            }
          }
        }
      ]
    }
  ]
}

Getting the results of the helper

The snippet below shows how to check whether the user has granted access and how to access the data.

Node.js

app.intent('Place Handler', (conv, params, place) => {
  if (!place) {
    conv.ask(`Sorry, I couldn't get a location from you.`);
  } else {
    // Place also contains formattedAddress and coordinates
    const {name} = place;
    conv.ask(`${name} sounds like a great place to go!`);
  }
  conv.ask('Would you like to try another helper?');
  conv.ask(new Suggestions([
    'Confirmation',
    'DateTime',
    'Permission',
  ]));
});index.js

Java

@ForIntent("Place Handler")
public ActionResponse handlePlace(ActionRequest request) {
  ResponseBuilder responseBuilder = getResponseBuilder(request);
  Location location = request.getPlace();
  if (location == null) {
    responseBuilder.add("Sorry, I couldn't get a location from you.");
  } else {
    responseBuilder.add(location.getName() + " sounds like a great place to go!");
  }
  responseBuilder
      .add("Would you like to try another helper?")
      .addSuggestions(new String[] {"Confirmation", "DateTime", "Permission"});
  return responseBuilder.build();
}MyActionsApp.java

Node.js

app.intent('actions.intent.PLACE', (conv) => {
  const place = conv.arguments.get('PLACE');
  if (!place) {
    conv.ask(`Sorry, I couldn't get a location from you.`);
  } else {
    // Place also contains formattedAddress and coordinates
    const {name} = place;
    conv.ask(`${name} sounds like a great place to go!`);
  }
  conv.ask('Would you like to try another helper?');
  conv.ask(new Suggestions([
    'Confirmation',
    'DateTime',
    'Permission',
  ]));
});index.js

Java

@ForIntent("actions.intent.PLACE")
public ActionResponse handlePlace(ActionRequest request) {
  ResponseBuilder responseBuilder = getResponseBuilder(request);
  Location location = request.getPlace();
  if (location == null) {
    responseBuilder.add("Sorry, I couldn't get a location from you.");
  } else {
    responseBuilder.add(location.getName() + " sounds like a great place to go!");
  }
  responseBuilder
      .add("Would you like to try another helper?")
      .addSuggestions(new String[] {"Confirmation", "DateTime", "Permission"});
  return responseBuilder.build();
}MyActionsApp.java

JSON

Note that the JSON below describes a webhook request.

{
  "responseId": "ce99fd40-b9ed-4151-bc99-f25db9eda8c5-712767ed",
  "queryResult": {
    "queryText": "actions_intent_PLACE",
    "action": "Place.Place-custom",
    "parameters": {},
    "allRequiredParamsPresent": true,
    "fulfillmentText": "Webhook failed for intent: Place Handler",
    "fulfillmentMessages": [
      {
        "text": {
          "text": [
            "Webhook failed for intent: Place Handler"
          ]
        }
      }
    ],
    "outputContexts": [
      {
        "name": "projects/df-helpers-kohler/agent/sessions/ABwppHEdLCo3vaqVs-pED_sPQDIUr_Ls5JYICyNks9vKbfUaIF8n4P7fYsKEYSiKvD3Vmnli9deu_uTB8s3nPeA/contexts/actions_capability_audio_output"
      },
      {
        "name": "projects/df-helpers-kohler/agent/sessions/ABwppHEdLCo3vaqVs-pED_sPQDIUr_Ls5JYICyNks9vKbfUaIF8n4P7fYsKEYSiKvD3Vmnli9deu_uTB8s3nPeA/contexts/actions_capability_screen_output"
      },
      {
        "name": "projects/df-helpers-kohler/agent/sessions/ABwppHEdLCo3vaqVs-pED_sPQDIUr_Ls5JYICyNks9vKbfUaIF8n4P7fYsKEYSiKvD3Vmnli9deu_uTB8s3nPeA/contexts/actions_capability_media_response_audio"
      },
      {
        "name": "projects/df-helpers-kohler/agent/sessions/ABwppHEdLCo3vaqVs-pED_sPQDIUr_Ls5JYICyNks9vKbfUaIF8n4P7fYsKEYSiKvD3Vmnli9deu_uTB8s3nPeA/contexts/actions_capability_account_linking"
      },
      {
        "name": "projects/df-helpers-kohler/agent/sessions/ABwppHEdLCo3vaqVs-pED_sPQDIUr_Ls5JYICyNks9vKbfUaIF8n4P7fYsKEYSiKvD3Vmnli9deu_uTB8s3nPeA/contexts/actions_capability_web_browser"
      },
      {
        "name": "projects/df-helpers-kohler/agent/sessions/ABwppHEdLCo3vaqVs-pED_sPQDIUr_Ls5JYICyNks9vKbfUaIF8n4P7fYsKEYSiKvD3Vmnli9deu_uTB8s3nPeA/contexts/google_assistant_input_type_keyboard"
      },
      {
        "name": "projects/df-helpers-kohler/agent/sessions/ABwppHEdLCo3vaqVs-pED_sPQDIUr_Ls5JYICyNks9vKbfUaIF8n4P7fYsKEYSiKvD3Vmnli9deu_uTB8s3nPeA/contexts/place-followup",
        "lifespanCount": 1
      },
      {
        "name": "projects/df-helpers-kohler/agent/sessions/ABwppHEdLCo3vaqVs-pED_sPQDIUr_Ls5JYICyNks9vKbfUaIF8n4P7fYsKEYSiKvD3Vmnli9deu_uTB8s3nPeA/contexts/actions_intent_place",
        "parameters": {
          "PLACE": "KFC, 2523 North Carson Street, Carson City, NV 89706, United States",
          "text": "yes"
        }
      }
    ],
    "intent": {
      "name": "projects/df-helpers-kohler/agent/intents/4cf9e154-e151-44db-ae79-84d8c5a707de",
      "displayName": "Place Handler"
    },
    "intentDetectionConfidence": 1,
    "languageCode": "en"
  },
  "originalDetectIntentRequest": {
    "source": "google",
    "version": "2",
    "payload": {
      "user": {
        "permissions": [
          "DEVICE_PRECISE_LOCATION"
        ],
        "locale": "en-US",
        "userVerificationStatus": "VERIFIED"
      },
      "conversation": {
        "conversationId": "ABwppHEdLCo3vaqVs-pED_sPQDIUr_Ls5JYICyNks9vKbfUaIF8n4P7fYsKEYSiKvD3Vmnli9deu_uTB8s3nPeA",
        "type": "ACTIVE",
        "conversationToken": "[\"place-followup\"]"
      },
      "inputs": [
        {
          "intent": "actions.intent.PLACE",
          "rawInputs": [
            {
              "inputType": "KEYBOARD",
              "query": "Query handled by Actions on Google"
            }
          ],
          "arguments": [
            {
              "name": "PLACE",
              "placeValue": {
                "coordinates": {
                  "latitude": 39.182895,
                  "longitude": -119.7679113
                },
                "formattedAddress": "KFC, 2523 North Carson Street, Carson City, NV 89706, United States",
                "name": "KFC",
                "placeId": "ChIJZfBpmMEKmYARFsfc574LgB8"
              }
            },
            {
              "name": "text",
              "rawText": "yes",
              "textValue": "yes"
            }
          ]
        }
      ],
      "surface": {
        "capabilities": [
          {
            "name": "actions.capability.AUDIO_OUTPUT"
          },
          {
            "name": "actions.capability.SCREEN_OUTPUT"
          },
          {
            "name": "actions.capability.MEDIA_RESPONSE_AUDIO"
          },
          {
            "name": "actions.capability.ACCOUNT_LINKING"
          },
          {
            "name": "actions.capability.WEB_BROWSER"
          }
        ]
      },
      "device": {
        "location": {
          "coordinates": {
            "latitude": 39.2814592,
            "longitude": -120.12421119999999
          },
          "formattedAddress": "Currant Road, Truckee, California 96161",
          "zipCode": "96161",
          "city": "Truckee"
        }
      },
      "availableSurfaces": [
        {
          "capabilities": [
            {
              "name": "actions.capability.WEB_BROWSER"
            },
            {
              "name": "actions.capability.SCREEN_OUTPUT"
            },
            {
              "name": "actions.capability.AUDIO_OUTPUT"
            }
          ]
        }
      ]
    }
  }
}

JSON

Note that the JSON below describes a webhook request.

{
  "user": {
    "permissions": [
      "DEVICE_PRECISE_LOCATION"
    ],
    "locale": "en-US",
    "userVerificationStatus": "VERIFIED"
  },
  "conversation": {
    "conversationId": "ABwppHEcidO2jguO4hS8maDeMVx8aasSCcBWZHrF3EmOMIaWRP9qF0BcACistiIoeyiSNTW6KD_tqN_U-xAIOXhPvA",
    "type": "NEW"
  },
  "inputs": [
    {
      "intent": "actions.intent.PLACE",
      "rawInputs": [
        {
          "inputType": "KEYBOARD",
          "query": "Query handled by Actions on Google"
        }
      ],
      "arguments": [
        {
          "name": "PLACE",
          "placeValue": {
            "coordinates": {
              "latitude": 37.4219999,
              "longitude": -122.0840575
            },
            "formattedAddress": "Googleplex, 1600 Amphitheatre Parkway, Mountain View, CA 94043, United States",
            "name": "Googleplex",
            "placeId": "ChIJj61dQgK6j4AR4GeTYWZsKWw"
          }
        },
        {
          "name": "text",
          "rawText": "googleplex",
          "textValue": "googleplex"
        }
      ]
    }
  ],
  "surface": {
    "capabilities": [
      {
        "name": "actions.capability.AUDIO_OUTPUT"
      },
      {
        "name": "actions.capability.MEDIA_RESPONSE_AUDIO"
      },
      {
        "name": "actions.capability.ACCOUNT_LINKING"
      },
      {
        "name": "actions.capability.SCREEN_OUTPUT"
      },
      {
        "name": "actions.capability.WEB_BROWSER"
      }
    ]
  },
  "device": {
    "location": {
      "coordinates": {
        "latitude": 37.4197238,
        "longitude": -122.08212759999999
      },
      "formattedAddress": "Google Building 45 1585 Charleston Road, Mountain View, California 94043",
      "zipCode": "94043",
      "city": "Mountain View"
    }
  },
  "availableSurfaces": [
    {
      "capabilities": [
        {
          "name": "actions.capability.SCREEN_OUTPUT"
        },
        {
          "name": "actions.capability.WEB_BROWSER"
        },
        {
          "name": "actions.capability.AUDIO_OUTPUT"
        }
      ]
    }
  ]
}

Confirmation

You can ask a generic confirmation from the user (yes/no question) and get the resulting answer. The grammar for "yes" and "no" naturally expands to things like "Yea" or "Nope", making it usable in many situations.

Calling the helper

The following code example shows how you can call the helper using the client library. The JSON snippets show the corresponding webhook response.

You can specify a custom prompt when asking the user for a confirmation.

Node.js

app.intent('Confirmation', (conv) => {
  conv.ask(new Confirmation('Can you confirm?'));
});index.js

Java

@ForIntent("Confirmation")
public ActionResponse getConfirmation(ActionRequest request) {
  ResponseBuilder responseBuilder = getResponseBuilder(request);
  responseBuilder
      .add("PLACEHOLDER")
      .add(new Confirmation().setConfirmationText("Can you confirm?"));
  return responseBuilder.build();
}MyActionsApp.java

Node.js

conv.ask(new Confirmation('Can you confirm?'));index.js

Java

ResponseBuilder responseBuilder = getResponseBuilder(request);
responseBuilder
    .add("PLACEHOLDER")
    .add(new Confirmation().setConfirmationText("Can you confirm?"));
return responseBuilder.build();MyActionsApp.java

JSON

Note that the JSON below describes a webhook request.

{
  "payload": {
    "google": {
      "expectUserResponse": true,
      "systemIntent": {
        "intent": "actions.intent.CONFIRMATION",
        "data": {
          "@type": "type.googleapis.com/google.actions.v2.ConfirmationValueSpec",
          "dialogSpec": {
            "requestConfirmationText": "Can you confirm?"
          }
        }
      }
    }
  }
}

JSON

Note that the JSON below describes a webhook request.

{
  "expectUserResponse": true,
  "expectedInputs": [
    {
      "possibleIntents": [
        {
          "intent": "actions.intent.CONFIRMATION",
          "inputValueData": {
            "@type": "type.googleapis.com/google.actions.v2.ConfirmationValueSpec",
            "dialogSpec": {
              "requestConfirmationText": "Can you confirm?"
            }
          }
        }
      ]
    }
  ]
}

Getting the results of the helper

After the user responds to the helper, you receive a request to your fulfillment and can check whether the user confirmed or not.

Node.js

app.intent('Confirmation Handler', (conv, params, confirmationGranted) => {
  conv.ask(confirmationGranted
    ? 'Thank you for confirming'
    : 'No problem, you have not confirmed');
  conv.ask('Would you like to try another helper?');
  conv.ask(new Suggestions([
    'DateTime',
    'Permission',
    'Place',
  ]));
});index.js

Java

@ForIntent("Confirmation Handler")
public ActionResponse handleConfirmation(ActionRequest request) {
  ResponseBuilder responseBuilder = getResponseBuilder(request);
  boolean userConfirmation = request.getUserConfirmation();
  responseBuilder
      .add(userConfirmation ? "Thank you for confirming" : "No problem, you have not confirmed")
      .addSuggestions(new String[] {"Place", "DateTime", "Permission"});
  return responseBuilder.build();
}MyActionsApp.java

Node.js

app.intent('actions.intent.CONFIRMATION', (conv, confirmationGranted) => {
  conv.ask(confirmationGranted
    ? 'Thank you for confirming'
    : 'No problem, you have not confirmed');
  conv.ask('Would you like to try another helper?');
  conv.ask(new Suggestions([
    'DateTime',
    'Permission',
    'Place',
  ]));
});index.js

Java

@ForIntent("actions.intent.CONFIRMATION")
public ActionResponse handleConfirmation(ActionRequest request) {
  ResponseBuilder responseBuilder = getResponseBuilder(request);
  boolean userConfirmation = request.getUserConfirmation();
  responseBuilder
      .add(userConfirmation ? "Thank you for confirming" : "No problem, you have not confirmed")
      .addSuggestions(new String[] {"Place", "DateTime", "Permission"});
  return responseBuilder.build();
}MyActionsApp.java

JSON

Note that the JSON below describes a webhook request.

{
  "responseId": "26c3dd46-321c-4ab5-9f6c-a445f6d5de01-712767ed",
  "queryResult": {
    "queryText": "actions_intent_CONFIRMATION",
    "parameters": {},
    "allRequiredParamsPresent": true,
    "fulfillmentText": "Webhook failed for intent: Confirmation Handler",
    "fulfillmentMessages": [
      {
        "text": {
          "text": [
            "Webhook failed for intent: Confirmation Handler"
          ]
        }
      }
    ],
    "outputContexts": [
      {
        "name": "projects/df-helpers-kohler/agent/sessions/ABwppHEdLCo3vaqVs-pED_sPQDIUr_Ls5JYICyNks9vKbfUaIF8n4P7fYsKEYSiKvD3Vmnli9deu_uTB8s3nPeA/contexts/actions_capability_account_linking"
      },
      {
        "name": "projects/df-helpers-kohler/agent/sessions/ABwppHEdLCo3vaqVs-pED_sPQDIUr_Ls5JYICyNks9vKbfUaIF8n4P7fYsKEYSiKvD3Vmnli9deu_uTB8s3nPeA/contexts/actions_capability_media_response_audio"
      },
      {
        "name": "projects/df-helpers-kohler/agent/sessions/ABwppHEdLCo3vaqVs-pED_sPQDIUr_Ls5JYICyNks9vKbfUaIF8n4P7fYsKEYSiKvD3Vmnli9deu_uTB8s3nPeA/contexts/actions_capability_audio_output"
      },
      {
        "name": "projects/df-helpers-kohler/agent/sessions/ABwppHEdLCo3vaqVs-pED_sPQDIUr_Ls5JYICyNks9vKbfUaIF8n4P7fYsKEYSiKvD3Vmnli9deu_uTB8s3nPeA/contexts/actions_capability_web_browser"
      },
      {
        "name": "projects/df-helpers-kohler/agent/sessions/ABwppHEdLCo3vaqVs-pED_sPQDIUr_Ls5JYICyNks9vKbfUaIF8n4P7fYsKEYSiKvD3Vmnli9deu_uTB8s3nPeA/contexts/actions_capability_screen_output"
      },
      {
        "name": "projects/df-helpers-kohler/agent/sessions/ABwppHEdLCo3vaqVs-pED_sPQDIUr_Ls5JYICyNks9vKbfUaIF8n4P7fYsKEYSiKvD3Vmnli9deu_uTB8s3nPeA/contexts/google_assistant_input_type_keyboard"
      },
      {
        "name": "projects/df-helpers-kohler/agent/sessions/ABwppHEdLCo3vaqVs-pED_sPQDIUr_Ls5JYICyNks9vKbfUaIF8n4P7fYsKEYSiKvD3Vmnli9deu_uTB8s3nPeA/contexts/actions_intent_confirmation",
        "parameters": {
          "CONFIRMATION": true,
          "text": "yes"
        }
      }
    ],
    "intent": {
      "name": "projects/df-helpers-kohler/agent/intents/11cc7c8e-0710-42ea-9cd9-723aff03e97f",
      "displayName": "Confirmation Handler"
    },
    "intentDetectionConfidence": 1,
    "languageCode": "en"
  },
  "originalDetectIntentRequest": {
    "source": "google",
    "version": "2",
    "payload": {
      "user": {
        "locale": "en-US",
        "userVerificationStatus": "VERIFIED"
      },
      "conversation": {
        "conversationId": "ABwppHEdLCo3vaqVs-pED_sPQDIUr_Ls5JYICyNks9vKbfUaIF8n4P7fYsKEYSiKvD3Vmnli9deu_uTB8s3nPeA",
        "type": "ACTIVE",
        "conversationToken": "[]"
      },
      "inputs": [
        {
          "intent": "actions.intent.CONFIRMATION",
          "rawInputs": [
            {
              "inputType": "KEYBOARD",
              "query": "yes"
            }
          ],
          "arguments": [
            {
              "name": "CONFIRMATION",
              "boolValue": true
            },
            {
              "name": "text",
              "rawText": "yes",
              "textValue": "yes"
            }
          ]
        }
      ],
      "surface": {
        "capabilities": [
          {
            "name": "actions.capability.ACCOUNT_LINKING"
          },
          {
            "name": "actions.capability.MEDIA_RESPONSE_AUDIO"
          },
          {
            "name": "actions.capability.AUDIO_OUTPUT"
          },
          {
            "name": "actions.capability.WEB_BROWSER"
          },
          {
            "name": "actions.capability.SCREEN_OUTPUT"
          }
        ]
      },
      "availableSurfaces": [
        {
          "capabilities": [
            {
              "name": "actions.capability.AUDIO_OUTPUT"
            },
            {
              "name": "actions.capability.SCREEN_OUTPUT"
            },
            {
              "name": "actions.capability.WEB_BROWSER"
            }
          ]
        }
      ]
    }
  },
  "session": "projects/df-helpers-kohler/agent/sessions/ABwppHEdLCo3vaqVs-pED_sPQDIUr_Ls5JYICyNks9vKbfUaIF8n4P7fYsKEYSiKvD3Vmnli9deu_uTB8s3nPeA"
}

JSON

Note that the JSON below describes a webhook request.

{
  "user": {
    "locale": "en-US",
    "userVerificationStatus": "VERIFIED"
  },
  "conversation": {
    "conversationId": "ABwppHGY8XRtymCExS313Cc6Vp42zR0uQuGf6BwpSW0772Er-rxSTtFjs3KfGYu6jCDgHbA8IKVWjB8Gr9odc4yHhg",
    "type": "NEW"
  },
  "inputs": [
    {
      "intent": "actions.intent.CONFIRMATION",
      "rawInputs": [
        {
          "inputType": "KEYBOARD",
          "query": "yes"
        }
      ],
      "arguments": [
        {
          "name": "CONFIRMATION",
          "boolValue": true
        },
        {
          "name": "text",
          "rawText": "yes",
          "textValue": "yes"
        }
      ]
    }
  ],
  "surface": {
    "capabilities": [
      {
        "name": "actions.capability.MEDIA_RESPONSE_AUDIO"
      },
      {
        "name": "actions.capability.SCREEN_OUTPUT"
      },
      {
        "name": "actions.capability.WEB_BROWSER"
      },
      {
        "name": "actions.capability.ACCOUNT_LINKING"
      },
      {
        "name": "actions.capability.AUDIO_OUTPUT"
      }
    ]
  },
  "availableSurfaces": [
    {
      "capabilities": [
        {
          "name": "actions.capability.AUDIO_OUTPUT"
        },
        {
          "name": "actions.capability.SCREEN_OUTPUT"
        },
        {
          "name": "actions.capability.WEB_BROWSER"
        }
      ]
    }
  ]
}

Helper intents (Dialogflow)

Explore in Dialogflow

Usage overview

Dialogflow

Node.js

JSON

Actions SDK

Node.js

JSON

Helper intents

User information

Calling the helper

Node.js

Java

Node.js

Java

JSON

JSON

Getting the results of the helper

Node.js

Java

Node.js

Java

JSON

JSON

List and Carousel Option

Calling the helper

Node.js

Java

Node.js

Java

JSON

JSON

Getting the results of the helper

Node.js

Java

Node.js

Java

JSON

JSON

Date and Time

Calling the helper

Node.js

Java

Node.js

Java

JSON

JSON

Getting the results of the helper

Node.js

Java

Node.js

Java

JSON

JSON

Account Sign-in

Calling the helper

Node.js

Java

JSON

JSON

Getting the results of the helper

Node.js

Java

JSON

JSON

Place and Location

Calling the helper

Node.js

Java

Node.js

Java

JSON

JSON

Getting the results of the helper

Node.js

Java

Node.js

Java

JSON