Intent fulfillment

Once you have created a smart home Action, the next step is to add functionality in your fulfillment to process the smart home intents and return responses that Google Assistant recognizes.

This page describes the process for implementing intent fulfillment in your your smart home Action:

Identify the user
List devices and their capabilities
Respond to queries and commands
Handle unlinking events

Identify the user

Assistant makes requests to your smart home Action's fulfillment with the access token provided by your OAuth 2.0 server in the Authorization header.

POST /fulfillment HTTP/1.1
Host: smarthome.example.com
Content-Type: application/json
Authorization: Bearer ACCESS_TOKEN

Before responding to any requests, your fulfillment logic should verify that this token credential is valid and determine the associated user account. If the access token is invalid, your fulfillment should return an HTTP 401 Unauthorized error.

List devices and their capabilities

Assistant sends an action.devices.SYNC intent to your fulfillment to request the list of devices associated with the given user and their capabilities. Your fulfillment must return a unique ID for each user in the agentUserId field of the SYNC response. This ID must be an immutable value to represent the user to your cloud service. It is not recommended to provide email addresses or other attributes based on settings the user can change.

The devices field of your SYNC response contains all of the devices the user has authorized Assistant to access, the types and traits they support, and the attributes required to configure the trait's behavior for that specific device.

The SYNC intent is triggered during account linking or when a user manually resyncs their devices. If the users' list of devices, supported traits, or attribute values change, use Request Sync to trigger a new SYNC intent and report the updates to Google.

Request

{
    "requestId": "ff36a3cc-ec34-11e6-b1a0-64510650abcf",
    "inputs": [{
      "intent": "action.devices.SYNC"
    }]
}

JSON

{
  "requestId": "ff36a3cc-ec34-11e6-b1a0-64510650abcf",
  "payload": {
    "agentUserId": "1836.15267389",
    "devices": [
      {
        "id": "123",
        "type": "action.devices.types.OUTLET",
        "traits": [
          "action.devices.traits.OnOff"
        ],
        "name": {
          "defaultNames": [
            "My Outlet 1234"
          ],
          "name": "Night light",
          "nicknames": [
            "wall plug"
          ]
        },
        "willReportState": false,
        "roomHint": "kitchen",
        "deviceInfo": {
          "manufacturer": "lights-out-inc",
          "model": "hs1234",
          "hwVersion": "3.2",
          "swVersion": "11.4"
        },
        "otherDeviceIds": [
          {
            "deviceId": "local-device-id"
          }
        ],
        "customData": {
          "fooValue": 74,
          "barValue": true,
          "bazValue": "foo"
        }
      },
      {
        "id": "456",
        "type": "action.devices.types.LIGHT",
        "traits": [
          "action.devices.traits.OnOff",
          "action.devices.traits.Brightness",
          "action.devices.traits.ColorSetting"
        ],
        "name": {
          "defaultNames": [
            "lights out inc. bulb A19 color hyperglow"
          ],
          "name": "lamp1",
          "nicknames": [
            "reading lamp"
          ]
        },
        "willReportState": false,
        "roomHint": "office",
        "attributes": {
          "colorModel": "rgb",
          "colorTemperatureRange": {
            "temperatureMinK": 2000,
            "temperatureMaxK": 9000
          },
          "commandOnlyColorSetting": false
        },
        "deviceInfo": {
          "manufacturer": "lights out inc.",
          "model": "hg11",
          "hwVersion": "1.2",
          "swVersion": "5.4"
        },
        "customData": {
          "fooValue": 12,
          "barValue": false,
          "bazValue": "bar"
        }
      }
    ]
  }
}

Node.js

const {smarthome} = require('actions-on-google');
const app = smarthome();
// ...
app.onSync((body, headers) => {
  // TODO Get devices for user
  return {
    requestId: body.requestId,
    payload: {
      agentUserId: "1836.15267389",
      devices: [{
        id: "123",
        type: "action.devices.types.OUTLET",
        traits: [
          "action.devices.traits.OnOff"
        ],
        name: {
          defaultNames: ["My Outlet 1234"],
          name: "Night light",
          nicknames: ["wall plug"]
        },
        willReportState: false,
        roomHint: "kitchen",
        deviceInfo: {
          manufacturer: "lights-out-inc",
          model: "hs1234",
          hwVersion: "3.2",
          swVersion: "11.4"
        },
        otherDeviceIds: [{
          deviceId: "local-device-id"
        }],
        customData: {
          fooValue: 74,
          barValue: true,
          bazValue: "foo"
        }
      }, {
        id: "456",
        type: "action.devices.types.LIGHT",
        traits: [
          "action.devices.traits.OnOff",
          "action.devices.traits.Brightness",
          "action.devices.traits.ColorSetting"
        ],
        name: {
          defaultNames: ["lights out inc. bulb A19 color hyperglow"],
          name: "lamp1",
          nicknames: ["reading lamp"]
        },
        willReportState: false,
        roomHint: "office",
        attributes: {
          colorModel: 'rgb',
          colorTemperatureRange: {
            temperatureMinK: 2000,
            temperatureMaxK: 9000
          },
          commandOnlyColorSetting: false
        },
        deviceInfo: {
          manufacturer: "lights out inc.",
          model: "hg11",
          hwVersion: "1.2",
          swVersion: "5.4"
        },
        customData: {
          fooValue: 12,
          barValue: false,
          bazValue: "bar"
        }
      }]
    }
  };
});

Java

@NotNull
@Override
public SyncResponse onSync(@NotNull SyncRequest syncRequest, @Nullable Map<?, ?> map) {
  Payload payload = new Payload();
  payload.setAgentUserId("1836.15267389");
  payload.setDevices(
      new Device[] {
        new Device.Builder()
            .setId("123")
            .setType("action.devices.types.OUTLET")
            .addTrait("action.devices.traits.OnOff")
            .setName(
                Collections.singletonList("My Outlet 1234"),
                "Night light",
                Collections.singletonList("Wall plug"))
            .setWillReportState(true)
            .setDeviceInfo("lights-out-inc", "hs1234", "3.2", "11.4")
            .setCustomData(
                new JSONObject()
                    .put("fooValue", 74)
                    .put("barValue", true)
                    .put("bazValue", "foo"))
            .build(),
        new Device.Builder()
            .setId("456")
            .setType("action.devices.types.LIGHT")
            .addTrait("action.devices.traits.OnOff")
            .addTrait("action.devices.traits.Brightness")
            .addTrait("action.devices.traits.ColorTemperature")
            .addTrait("action.devices.traits.ColorSpectrum")
            .setName(
                Collections.singletonList("Lights Out Inc. bulb A19 color hyperglow"),
                "Lamp",
                Collections.singletonList("Reading lamp"))
            .setWillReportState(true)
            .setDeviceInfo("Lights Out Inc.", "hg11", "1.2", "5.4")
            .setCustomData(
                new JSONObject()
                    .put("fooValue", 12)
                    .put("barValue", false)
                    .put("bazValue", "bar"))
            .build(),
      });
  return new SyncResponse(syncRequest.getRequestId(), payload);
}

For additional reference, see the SYNC intent reference documentation.

Respond to queries and commands

When users interact with Assistant to query the current state of a device, your fulfillment receives an action.devices.QUERY intent containing a list of device IDs (as provided by your SYNC response). Your fulfillment receives an action.devices.EXECUTE intent when users send commands to Assistant to control your device.

Handle `QUERY` intents

Your QUERY response includes a full set of states for each of the traits supported by the requested devices.

Request

{
    "requestId": "ff36a3cc-ec34-11e6-b1a0-64510650abcf",
    "inputs": [{
      "intent": "action.devices.QUERY",
      "payload": {
        "devices": [{
          "id": "123",
          "customData": {
            "fooValue": 74,
            "barValue": true,
            "bazValue": "foo"
          }
        }, {
          "id": "456",
          "customData": {
            "fooValue": 12,
            "barValue": false,
            "bazValue": "bar"
          }
        }]
      }
    }]
}

JSON

{
  "requestId": "ff36a3cc-ec34-11e6-b1a0-64510650abcf",
  "payload": {
    "devices": {
      "123": {
        "on": true,
        "online": true
      },
      "456": {
        "on": true,
        "online": true,
        "brightness": 80,
        "color": {
          "name": "cerulean",
          "spectrumRGB": 31655
        }
      }
    }
  }
}

Node.js

const {smarthome} = require('actions-on-google');
const app = smarthome();
// ...
app.onQuery((body, headers) => {
  // TODO Get device state
  return {
    requestId: body.requestId,
    payload: {
      devices: {
        123: {
          on: true,
          online: true
        },
        456: {
          on: true,
          online: true,
          brightness: 80,
          color: {
            name: "cerulean",
            spectrumRGB: 31655
          }
        }
      }
    }
  };
});

Java

@NotNull
@Override
public QueryResponse onQuery(@NotNull QueryRequest queryRequest, @Nullable Map<?, ?> map) {
  QueryResponse.Payload payload = new QueryResponse.Payload();
  payload.setDevices(
      new HashMap<String, Map<String, Object>>() {
        {
          put(
              "123",
              new HashMap<String, Object>() {
                {
                  put("on", true);
                  put("online", true);
                }
              });
          put(
              "456",
              new HashMap<String, Object>() {
                {
                  put("on", true);
                  put("online", true);
                  put("brightness", 80);
                  put(
                      "color",
                      new HashMap<String, Object>() {
                        {
                          put("name", "cerulean");
                          put("spectrumRGB", 31655);
                        }
                      });
                }
              });
        }
      });

  return new QueryResponse(queryRequest.getRequestId(), payload);
}

For additional reference, see the QUERY intent reference documentation.

Handle `EXECUTE` intents

Similar to QUERY, a single intent can target multiple device IDs. A single EXECUTE intent may also contain multiple distinct commands given to a group of devices. For example, a triggered intent may set both brightness and color on a group of lights, or set multiple lights each to a different color. Your EXECUTE response should return the new state of the device after the execution.

Use Report State when the state of the users' device changes; for example, due to an EXECUTE intent or a local state change (such as manually flipping a light switch). This keeps Home Graph synchronized with your cloud service.

Request

{
    "requestId": "ff36a3cc-ec34-11e6-b1a0-64510650abcf",
    "inputs": [{
      "intent": "action.devices.EXECUTE",
      "payload": {
        "commands": [{
          "devices": [{
            "id": "123",
            "customData": {
              "fooValue": 74,
              "barValue": true,
              "bazValue": "sheepdip"
            }
          }, {
            "id": "456",
            "customData": {
              "fooValue": 36,
              "barValue": false,
              "bazValue": "moarsheep"
            }
          }],
          "execution": [{
            "command": "action.devices.commands.OnOff",
            "params": {
              "on": true
            }
          }]
        }]
      }
    }]
}

JSON

{
  "requestId": "ff36a3cc-ec34-11e6-b1a0-64510650abcf",
  "payload": {
    "commands": [
      {
        "ids": [
          "123"
        ],
        "status": "SUCCESS",
        "states": {
          "on": true,
          "online": true
        }
      },
      {
        "ids": [
          "456"
        ],
        "status": "ERROR",
        "errorCode": "deviceTurnedOff"
      }
    ]
  }
}

Node.js

const {smarthome} = require('actions-on-google');
const app = smarthome();
// ...
app.onExecute((body, headers) => {
  // TODO Send command to device
  return {
    requestId: body.requestId,
    payload: {
      commands: [{
        ids: ["123"],
        status: "SUCCESS",
        states: {
          on: true,
          online: true
        }
      }, {
        ids: ["456"],
        status: "ERROR",
        errorCode: "deviceTurnedOff"
      }]
    }
  };
});

Java

@NotNull
@Override
public ExecuteResponse onExecute(
    @NotNull ExecuteRequest executeRequest, @Nullable Map<?, ?> map) {
  ExecuteResponse.Payload payload = new ExecuteResponse.Payload();

  payload.setCommands(
      new Commands[] {
        new Commands(
            new String[] {"123"},
            "SUCCESS",
            new HashMap<String, Object>() {
              {
                put("on", true);
                put("online", true);
              }
            },
            null,
            null),
        new Commands(new String[] {"456"}, "ERROR", null, "deviceTurnedOff", null)
      });
  return new ExecuteResponse(executeRequest.getRequestId(), payload);
}

For additional reference, see the EXECUTE intent reference documentation.

Status responses

Your QUERY and EXECUTE responses include a status field to report the result of the request. Each status response can provide one of the following values:

SUCCESS: The request succeeded.
OFFLINE: Target device is offline or otherwise unreachable.
EXCEPTIONS: There is an issue or alert associated with the request.
ERROR: The request failed with the corresponding errorCode.

For ERROR and EXCEPTIONS, see Handle errors and exceptions and Errors and exceptions for more details.

Handle unlinking events

If the user unlinks your smart home Action from Assistant, your fulfillment receives an action.devices.DISCONNECT intent. This intent indicates that Assistant will not send any more intents for this user, and your cloud service should stop calling Home Graph APIs (Request Sync and Report State) for their devices.

Request

{
    "requestId": "ff36a3cc-ec34-11e6-b1a0-64510650abcf",
    "inputs": [{
      "intent": "action.devices.DISCONNECT",
    }]
}

JSON

{}

Node.js

const {smarthome} = require('actions-on-google');
const app = smarthome();
// ...
app.onDisconnect((body, headers) => {
  // TODO Disconnect user account from Google Assistant
  // You can return an empty body
  return {};
});

Java

@Override
public void onDisconnect(
    @NotNull DisconnectRequest disconnectRequest, @Nullable Map<?, ?> map) {
  // TODO Disconnect user account from Google Assistant
  // This function does not return anything
}

For additional reference, see the DISCONNECT intent reference documentation.

You have successfully implemented fulfillment for your smart home Action project. You are now ready to publish asynchronous changes using Request Sync and Report State.

Intent fulfillment

Identify the user

List devices and their capabilities

Respond to queries and commands

Handle QUERY intents

Handle EXECUTE intents

Status responses

Handle unlinking events

Handle `QUERY` intents

Handle `EXECUTE` intents