Your device may have special abilities not covered by the current set of traits -- such as a "blink my light" trait for a device that can blink its lights. You can define custom actions for your device that specify the commands sent to your device to trigger special abilities.
To define a custom device action, you need the following:
- A pattern to match against the user query
- A custom device action to associate with a matched query
- Text spoken back to the user if the device supports the action
- A command name that is sent back to your device, along with any parameters
You create the custom device action by putting this information into an action package. Action packages define the format for the Assistant responses. Unlike the Actions SDK, custom device actions are fulfilled locally; you do not specify an endpoint to process requests and provide responses. Custom device actions are not conversational in nature.
Create an Action Package
Using the following as an example, create a file (such as actions.json) that
defines a test command: blinking an LED.
Copy the file from the sample code you
downloaded in a previous step:
cd assistant-sdk-python/google-assistant-sdk/googlesamples/assistant/library/
cp ~/assistant-sdk-python/google-assistant-sdk/actions.json .
{
"manifest": {
"displayName": "Blinky light",
"invocationName": "Blinky light",
"category": "PRODUCTIVITY"
},
"actions": [
{
"name": "com.example.actions.BlinkLight",
"availability": {
"deviceClasses": [
{
"assistantSdkDevice": {}
}
]
},
"intent": {
"name": "com.example.intents.BlinkLight",
"parameters": [
{
"name": "number",
"type": "SchemaOrg_Number"
},
{
"name": "speed",
"type": "Speed"
}
],
"trigger": {
"queryPatterns": [
"blink ($Speed:speed)? $SchemaOrg_Number:number times",
"blink $SchemaOrg_Number:number times ($Speed:speed)?"
]
}
},
"fulfillment": {
"staticFulfillment": {
"templatedResponse": {
"items": [
{
"simpleResponse": {
"textToSpeech": "Blinking $number times"
}
},
{
"deviceExecution": {
"command": "com.example.commands.BlinkLight",
"params": {
"speed": "$speed",
"number": "$number"
}
}
}
]
}
}
}
}
],
"types": [
{
"name": "$Speed",
"entities": [
{
"key": "SLOWLY",
"synonyms": [
"slowly",
"slow"
]
},
{
"key": "NORMALLY",
"synonyms": [
"normally",
"regular"
]
},
{
"key": "QUICKLY",
"synonyms": [
"quickly",
"fast",
"quick"
]
}
]
}
]
}
The previous example uses the following information to define the custom device action:
- A pattern to match against the user query (blink N times)
- The custom device action to associate with a matched query (
com.example.actions.BlinkLight) for organizational purposes - Text spoken back to the user if the device supports the action (Blinking N times)
- A command name (
com.example.commands.BlinkLight) that is sent back to the device, along with any parameters (a number and possibly a description of the speed)
To define the query pattern, note the following:
- You can use schema.org-defined types in the query pattern.
- The
types [...]array defines the list of custom types (for example,$Speed). - You can use custom types in the query pattern. The user can speak any of the synonyms in your custom type to match the query pattern.
- When a synonym does match, the type instance (
speed) would return the normalized key (SLOWLY). There can be multiple entities in case, for example, there are different lights that support different speeds of blinking. - Parts of the request TTS pattern can be optional. For example, use
($Speed:speed)?in the query pattern to make this part optional. $type.raw(for example,$speed.raw) in the response TTS is replaced by the word(s) the user actually spoke.
For descriptions of many of these fields, see the ActionPackage documentation.
Deploy the Action Package
After you build your custom device action in an Action package, you can make the Action package accessible to the Assistant.
While you can perform the steps in this section on your device, it may be easier to do them on your development system. The following commands do not require a virtual environment to run.
Download the
gactionscommand line tool.Remove any existing credentials from the same directory as the
gactionstool.rm creds.data
Save your Action package to Google by using the
gactionsCLI. Replaceproject_idwith your Actions Console project ID../gactions update --action_package actions.json --project project_id
The first time you run this command you will be given a URL and be asked to sign in. Copy the URL and paste it into a browser (this can be done on any system). The page will ask you to sign in to your Google account. Sign into the Google account that created the project in a previous step.
After you approve the permission request from the API, a code will appear in your browser, such as "4/XXXX". Copy and paste this code into the terminal:
Enter the authorization code:
If authorization was successful, you will see a response similar to the following:
Your app for the Assistant for project my-devices-project was successfully updated with your actions.Deploy your Action package into test mode by using the
gactionsCLI. You must have saved your Action package to Google at least once before running this command. Test mode enables the Action package on your user account only../gactions test --action_package actions.json --project project_id
Currently, you cannot test the project using the Actions simulator.
To update the Action package, use the
gactions updatecommand.(Optional) You can create localized Action packages to support many different languages and locales at the same time in a single project.
Modify the sample
Do the steps in this section on the device.
nano hotword.py
Add a handler for your custom action. You can use the handler below if you want to use the sample Action Package above.
import time # Add this to the imports near the top of the file ... if event.type == EventType.ON_DEVICE_ACTION: for command, params in event.actions: print('Do command', command, 'with params', str(params)) # Add the following lines after the existing line above: if command == "com.example.commands.BlinkLight": number = int( params['number'] ) for i in range(int(number)): print('Device is blinking.') # GPIO.output(25, 1) time.sleep(1) # GPIO.output(25, 0) time.sleep(1)
Run the sample
Run the modified source code.
python hotword.py --device-model-id my-model
Try a query. For the example above, try the following:
Ok Google, blink 5 times.
Note that the query needs to match the query pattern in the Action Package.