Conversational Actions were deprecated on June 13, 2023. For more information, see Conversational Actions sunset.

Webhooks

To give you even more flexibility in building Actions, you can delegate logic to HTTPS web services (fulfillment). Your Actions can trigger webhooks that make requests to an HTTPS endpoint. Some examples of what you can do in fulfillment include:

Generating a dynamic prompt based on information provided by the user.
Placing an order in an external system and confirming success.
Validating slots with backend data.

**Figure 1.** Invocation intents and scenes can trigger webhooks.

Webhook triggers and handlers

Your Actions can trigger a webhook within invocation intents or scenes, which sends a request to your fulfillment endpoint. Your fulfillment contains webhook handlers that process the JSON payload in the request. You can trigger webhooks in the following situations:

After an invocation intent match
During a scene's on enter stage
After a condition evaluates to true in a scene's condition stage
During a scene's slot-filling stage
After an intent match occurs in a scene's input stage

When you trigger a webhook in your Actions, Google Assistant sends a request with a JSON payload to your fulfillment, which contains the name of the handler to use to process the event. Your fulfillment endpoint can route the event to the appropriate handler to carry out logic and return a corresponding response with a JSON payload.

Payloads

The following snippets show example requests that your Actions send to fulfillment and a response that your fulfillment sends back. See the reference documentation for more information.

Example Request

{
  "handler": {
    "name": "handler_name"
  },
  "intent": {
    "name": "actions.intent.MAIN",
    "params": {},
    "query": ""
  },
  "scene": {
    "name": "SceneName",
    "slotFillingStatus": "UNSPECIFIED",
    "slots": {}
  },
  "session": {
    "id": "example_session_id",
    "params": {},
    "typeOverrides": []
  },
  "user": {
    "locale": "en-US",
    "params": {
      "verificationStatus": "VERIFIED"
    }
  },
  "home": {
    "params": {}
  },
  "device": {
    "capabilities": [
      "SPEECH",
      "RICH_RESPONSE",
      "LONG_FORM_AUDIO"
    ]
  }
}

Example response

{
  "session": {
    "id": "example_session_id",
    "params": {}
  },
  "prompt": {
    "override": false,
    "firstSimple": {
      "speech": "Hello World.",
      "text": ""
    }
  },
  "scene": {
    "name": "SceneName",
    "slots": {},
    "next": {
      "name": "actions.scene.END_CONVERSATION"
    }
  }
}

Runtime interactions

The following sections describe common tasks that you can carry out in your webhook handlers.

Send prompts

You can create prompts with simple text, rich text, cards, and even full-blown HTML prompts backed by a web app with Interactive Canvas. The prompts documentation has complete information on how to create a prompt when handling a webhook event. The following snippets show a card prompt:

Node.js

app.handle('rich_response', conv => {
  conv.add('This is a card rich response.');
  conv.add(new Card({
    title: 'Card Title',
    subtitle: 'Card Subtitle',
    text: 'Card Content',
    image: new Image({
      url: 'https://developers.google.com/assistant/assistant_96.png',
      alt: 'Google Assistant logo'
    })
  }));
});

Response JSON

{
  "session": {
    "id": "example_session_id",
    "params": {}
  },
  "prompt": {
    "override": false,
    "content": {
      "card": {
        "title": "Card Title",
        "subtitle": "Card Subtitle",
        "text": "Card Content",
        "image": {
          "alt": "Google Assistant logo",
          "height": 0,
          "url": "https://developers.google.com/assistant/assistant_96.png",
          "width": 0
        }
      }
    },
    "firstSimple": {
      "speech": "This is a card rich response.",
      "text": ""
    }
  }
}

Read intent parameters

When the Assistant runtime matches an intent, it extracts any defined parameters. The original property was what the user provided as input and the resolved property is what the NLU resolved the input to based on the type specification.

Node.js

conv.intent.params['param_name'].original
conv.intent.params['param_name'].resolved

Request JSON

{
  "handler": {
    "name": "handler_name"
  },
  "intent": {
    "name": "intent_name",
    "params": {
      "slot_name": {
        "original": "1",
        "resolved": 1
      }
    },
    "query": ""
  },
  "scene": {
    "name": "SceneName",
    "slotFillingStatus": "UNSPECIFIED",
    "slots": {},
    "next": {
      "name": "actions.scene.END_CONVERSATION"
    }
  },
  "session": {
    "id": "session_id",
    "params": {},
    "typeOverrides": []
  },
  "user": {
    "locale": "en-US",
    "params": {
      "verificationStatus": "VERIFIED"
    }
  },
  "home": {
    "params": {}
  },
  "device": {
    "capabilities": [
      "SPEECH",
      "RICH_RESPONSE",
      "LONG_FORM_AUDIO"
    ]
  }
}

Read user locale

This value corresponds to the user's locale setting for Google Assistant.

Node.js

conv.user.locale

JSON

{
  "handler": {
    "name": "handler_name"
  },
  "intent": {
    "name": "actions.intent.MAIN",
    "params": {},
    "query": ""
  },
  "scene": {
    "name": "SceneName",
    "slotFillingStatus": "UNSPECIFIED",
    "slots": {}
  },
  "session": {
    "id": "session_id",
    "params": {},
    "typeOverrides": []
  },
  "user": {
    "locale": "en-US",
    "params": {
      "verificationStatus": "VERIFIED"
    }
  },
  "home": {
    "params": {}
  },
  "device": {
    "capabilities": [
      "SPEECH",
      "RICH_RESPONSE",
      "LONG_FORM_AUDIO"
    ]
  }
}

Read and write storage

See the storage documentation for complete information on how to use various storage features.

Node.js

//read
conv.session.params.key
conv.user.params.key
conv.home.params.key

// write
conv.session.params.key = value
conv.user.params.key = value
conv.home.params.key = value

Request JSON

{
  "handler": {
    "name": "handler_name"
  },
  "intent": {
    "name": "actions.intent.MAIN",
    "params": {},
    "query": ""
  },
  "scene": {
    "name": "SceneName",
    "slotFillingStatus": "UNSPECIFIED",
    "slots": {}
  },
  "session": {
    "id": "session_id",
    "params": {
      "key": "value"
    },
    "typeOverrides": [],
    "languageCode": ""
  },
  "user": {
    "locale": "en-US",
    "params": {
      "verificationStatus": "VERIFIED",
      "key": "value"
    }
  },
  "home": {
    "params": {
      "key": "value"
    }
  },
  "device": {
    "capabilities": [
      "SPEECH",
      "RICH_RESPONSE",
      "LONG_FORM_AUDIO"
    ]
  }
}

Response JSON

{
  "session": {
    "id": "session_id",
    "params": {
      "key": "value"
    }
  },
  "prompt": {
    "override": false,
    "firstSimple": {
      "speech": "Hello world.",
      "text": ""
    }
  },
  "user": {
    "locale": "en-US",
    "params": {
      "verificationStatus": "VERIFIED",
      "key": "value"
    }
  },
  "home": {
    "params": {
      "key": "value"
    }
  }
}

Check device capabilities

You can check a device's capabilities to deliver different experiences or conversation flows.

Node.js

const supportsRichResponse = conv.device.capabilities.includes("RICH_RESPONSE");
const supportsLongFormAudio = conv.device.capabilities.includes("LONG_FORM_AUDIO");
const supportsSpeech = conv.device.capabilities.includes("SPEECH");
const supportsInteractiveCanvas = conv.device.capabilities.includes("INTERACTIVE_CANVAS");

Request JSON

{
  "handler": {
    "name": "handler_name"
  },
  "intent": {
    "name": "actions.intent.MAIN",
    "params": {},
    "query": ""
  },
  "scene": {
    "name": "SceneName",
    "slotFillingStatus": "UNSPECIFIED",
    "slots": {}
  },
  "session": {
    "id": "session_id",
    "params": {},
    "typeOverrides": [],
    "languageCode": ""
  },
  "user": {
    "locale": "en-US",
    "params": {
      "verificationStatus": "VERIFIED"
    }
  },
  "home": {
    "params": {}
  },
  "device": {
    "capabilities": [
      "SPEECH",
      "RICH_RESPONSE",
      "LONG_FORM_AUDIO",
      "INTERACTIVE_CANVAS"
    ]
  }
}

For a full list of surface capabilities, see the Capability reference.

Runtime type overrides

Runtime types let you modify type specifications at runtime. You can use this feature to load data from other sources to populate a type's valid values. For example, you can use runtime type overrides to add dynamic options to a survey question or to add a daily item to a menu.

To use runtime types, you trigger a webhook from your Action that calls a handler in your fulfillment. From there, you can populate the session.typeOverrides parameter in a response back to your Action. Available modes include TYPE_MERGE to preserve existing type entries or TYPE_REPLACE to replace existing entries with the overrides.

Node.js

conv.session.typeOverrides = [{
    name: type_name,
    mode: 'TYPE_REPLACE',
    synonym: {
      entries: [
        {
          name: 'ITEM_1',
          synonyms: ['Item 1', 'First item']
        },
        {
          name: 'ITEM_2',
          synonyms: ['Item 2', 'Second item']
       },
       {
          name: 'ITEM_3',
          synonyms: ['Item 3', 'Third item']
        },
        {
          name: 'ITEM_4',
          synonyms: ['Item 4', 'Fourth item']
        },
    ]
  }
}];

Response JSON

{
  "session": {
    "id": "session_id",
    "params": {},
    "typeOverrides": [
      {
        "name": "type_name",
        "synonym": {
          "entries": [
            {
              "name": "ITEM_1",
              "synonyms": [
                "Item 1",
                "First item"
              ]
            },
            {
              "name": "ITEM_2",
              "synonyms": [
                "Item 2",
                "Second item"
              ]
            },
            {
              "name": "ITEM_3",
              "synonyms": [
                "Item 3",
                "Third item"
              ]
            },
            {
              "name": "ITEM_4",
              "synonyms": [
                "Item 4",
                "Fourth item"
              ]
            }
          ]
        },
        "typeOverrideMode": "TYPE_REPLACE"
      }
    ]
  },
  "prompt": {
    "override": false,
    "firstSimple": {
      "speech": "This is an example prompt.",
      "text": "This is an example prompt."
    }
  }
}

Provide speech biasing

Speech biasing lets you specify hints to the NLU to improve intent matching. You can specify up to 1000 entries.

Node.js

conv.expected.speech = ['value_1', 'value_2']
conv.expected.language = 'locale_string'

Response JSON

{
  "session": {
    "id": "session_id",
    "params": {}
  },
  "prompt": {
    "override": false,
    "firstSimple": {
      "speech": "This is an example prompt.",
      "text": "This is an example prompt."
    }
  },
  "expected": {
    "speech": "['value_1', 'value_2']",
    "language": "locale_string"
  }
}

Transition scenes

In addition to defining static transitions in your Actions project, you can cause scene transitions to occur at runtime.

Node.js

app.handle('transition_to_hidden_scene', conv => {
  // Dynamic transition
  conv.scene.next.name = "HiddenScene";
});

Response JSON

{
  "session": {
    "id": "session_id",
    "params": {}
  },
  "prompt": {
    "override": false,
    "firstSimple": {
      "speech": "This is an example prompt.",
      "text": ""
    }
  },
  "scene": {
    "name": "SceneName",
    "slots": {},
    "next": {
      "name": "HiddenScene"
    }
  }
}

Read scene slots

During slot filling, you can use fulfillment to validate the slot or check the status of slot filling (SlotFillingStatus).

Node.js

conv.scene.slotFillingStatus  // FINAL means all slots are filled
conv.scene.slots  // Object that contains all the slots
conv.scene.slots['slot_name'].<property_name> // Accessing a specific slot's properties

For example, suppose you want to extract the time zone from a response. In this example, the slot name is datetime1. To obtain the time zone, you would use:

conv.scene.slots['datetime1'].value.time_zone.id

Request JSON

{
  "handler": {
    "name": "handler_name"
  },
  "intent": {
    "name": "",
    "params": {
      "slot_name": {
        "original": "1",
        "resolved": 1
      }
    },
    "query": ""
  },
  "scene": {
    "name": "SceneName",
    "slotFillingStatus": "FINAL",
    "slots": {
      "slot_name": {
        "mode": "REQUIRED",
        "status": "SLOT_UNSPECIFIED",
        "updated": true,
        "value": 1
      }
    },
    "next": {
      "name": "actions.scene.END_CONVERSATION"
    }
  },
  "session": {
    "id": "session_id",
    "params": {
      "slot_name": 1
    },
    "typeOverrides": []
  },
  "user": {
    "locale": "en-US",
    "params": {
      "verificationStatus": "VERIFIED"
    }
  },
  "home": {
    "params": {}
  },
  "device": {
    "capabilities": [
      "SPEECH",
      "RICH_RESPONSE",
      "LONG_FORM_AUDIO"
    ]
  }
}

Invalidate scene slots

You can invalidate slots and make the user provide a new value.

Node.js

conv.scene.slots['slot_name'].status = 'INVALID'

Response JSON

{
  "session": {
    "id": "session_id",
    "params": {
      "slot_name": 1
    }
  },
  "prompt": {
    "override": false,
    "firstSimple": {
      "speech": "This is an example prompt.",
      "text": ""
    }
  },
  "scene": {
    "name": "SceneName",
    "slots": {
      "slot_name": {
        "mode": "REQUIRED",
        "status": "INVALID",
        "updated": true,
        "value": 1
      }
    },
    "next": {
      "name": "actions.scene.END_CONVERSATION"
    }
  }
}

Development options

Actions Builder provides an inline editor called the Cloud Functions editor, which lets you build and deploy a Cloud Function for Firebase directly in the console. You can also build and deploy fulfillment to your hosting of choice and register your HTTPS fulfillment endpoint as your webhook handler.

Inline editor

To develop with the Cloud Functions editor:

Create the file sdk/webhooks/ActionsOnGoogleFulfillment.yaml, and define the handlers for your Action and the inline cloud function used for fulfillment.
```
handlers:
- name: questionOnEnterFunc
- name: fruitSlotValidationFunc
inlineCloudFunction:
  executeFunction: ActionsOnGoogleFulfillment
    
```

Create the folder sdk/webhooks/ActionsOnGoogleFulfillment, and add an index.js file that implements the handlers previously defined and a package.json file that defines npm requirements for your code.

// index.js
const {conversation} = require('@assistant/conversation');
const functions = require('firebase-functions');

const app = conversation();

app.handle('questionOnEnterFunc', conv => {
  conv.add('questionOnEnterFunc triggered on webhook');
});

app.handle('fruitSlotValidationFunc', conv => {
  conv.add('fruitSlotValidationFunc triggered on webhook');
});

exports.ActionsOnGoogleFulfillment = functions.https.onRequest(app);

// package.json
{
  "name": "ActionsOnGoogleFulfillment",
  "version": "0.1.0",
  "description": "Actions on Google fulfillment",
  "main": "index.js",
  "dependencies": {
    "@assistant/conversation": "^3.0.0",
    "firebase-admin": "^5.4.3",
    "firebase-functions": "^0.7.1"
  }
}

External HTTPS endpoint

This section describes how to set up Cloud Functions for Firebase as a fulfillment service for your Conversational Action. However, you can deploy fulfillment to a hosting service of your choice.

Set up environment

We recommend the following project structure when you use Cloud functions for Firebase as a fulfillment service:

ProjectFolder        - Root folder for the project
  sdk                - Actions project configuration files
  functions          - Cloud functions for Firebase files

To set up your environment, follow these steps:

Download and install Node.js.
Set up and initialize the Firebase CLI. If the following command fails with an EACCES error, you may need to change npm permissions.
```
npm install -g firebase-tools
```
Authenticate the firebase tool with your Google account:
```
firebase login
```
Start the project directory where you saved your Actions project. You'll be asked to select which Firebase CLI features you want to setup for your Actions project. Choose Functions and other features you might want to use, like Firestore, then press Enter to confirm and continue:
```
$ cd <ACTIONS_PROJECT_DIRECTORY>
$ firebase init
```
Associate the Firebase tool with your Actions project by selecting it using the arrow keys to navigate the projects list:

Note: You can skip this step by selecting [don't setup a default project], but then you will need to do this association later using the command firebase use PROJECT_ID.

After choosing the project, the Firebase tool starts the Functions setup and asks you what language you want to use. Select using the arrow keys and press Enter to continue.

=== Functions Setup
A functions directory will be created in your project with a Node.js
package pre-configured. Functions can be deployed with firebase deploy.

? What language would you like to use to write Cloud Functions? (Use arrow keys)
> JavaScript
TypeScript

Choose if you want to use ESLint to catch probable bugs and enforce style by typing Y or N:
```
? Do you want to use ESLint to catch probable bugs and enforce style? (Y/n)
```
Get the project dependencies by typing Y to the prompt:
```
? Do you want to install dependencies with npm now? (Y/n)
```
Once the setup is completed, you'll see an output similar to the following:
```
✔  Firebase initialization complete!
```

Install the @assistant/conversation dependency:

$ cd <ACTIONS_PROJECT_DIRECTORY>/functions
$ npm install @assistant/conversation --save

Get the fulfillment dependencies and deploy the fulfillment function:

$ npm install
$ firebase deploy --only functions

The deployment takes a few minutes. Once completed, you'll see output similar to the following. You'll need the Function URL to enter in Dialogflow.

✔  Deploy complete!

Project Console: https://console.firebase.google.com/project/<PROJECT_ID>/overview
Function URL (<FUNCTION_NAME>): https://us-central1-<PROJECT_ID>.cloudfunctions.net/<FUNCTION_NAME>

Copy the fulfillment URL to use in the next section.

Register webhook handler

Create the file sdk/webhooks/ActionsOnGoogleFulfillment.yaml, and define the handlers for your Action and the URL for webhook requests.

httpsEndpoint:
  baseUrl: https://my.web.hook/ActionsOnGoogleFulfillment
  endpointApiVersion: 2
handlers:
- name: questionOnEnterFunc
- name: fruitSlotValidationFunc