Index
EmbeddedAssistant
(interface)AudioInConfig
(message)AudioInConfig.Encoding
(enum)AudioOut
(message)AudioOutConfig
(message)AudioOutConfig.Encoding
(enum)ConverseConfig
(message)ConverseRequest
(message)ConverseResponse
(message)ConverseResponse.EventType
(enum)ConverseResult
(message)ConverseResult.MicrophoneMode
(enum)ConverseState
(message)
EmbeddedAssistant
Service that implements Google Assistant API.
Converse | |
---|---|
Initiates or continues a conversation with the embedded assistant service. Each call performs one round-trip, sending an audio request to the service and receiving the audio response. Uses bidirectional streaming to receive results, such as the A conversation is one or more gRPC connections, each consisting of several streamed requests and responses. For example, the user says Add to my shopping list and the assistant responds What do you want to add?. The sequence of streamed requests and responses in the first gRPC message could be:
The user then says bagels and the assistant responds OK, I've added bagels to your shopping list. This is sent as another gRPC connection call to the
Although the precise order of responses is not guaranteed, sequential ConverseResponse.audio_out messages will always contain sequential portions of audio.
|
AudioInConfig
Specifies how to process the audio_in
data that will be provided in subsequent requests. For recommended settings, see the Google Assistant gRPC API best practices.
Fields | |
---|---|
encoding |
Required Encoding of audio data sent in all |
sample_rate_hertz |
Required Sample rate (in Hertz) of the audio data sent in all |
Encoding
Audio encoding of the data sent in the audio message. Audio must be one-channel (mono). The only language supported is "en-US".
Enums | |
---|---|
ENCODING_UNSPECIFIED |
Not specified. Will return result google.rpc.Code.INVALID_ARGUMENT . |
LINEAR16 |
Uncompressed 16-bit signed little-endian samples (Linear PCM). This encoding includes no header, only the raw audio bytes. |
FLAC |
FLAC (Free Lossless Audio Codec) is the recommended encoding because it is lossless--therefore recognition is not compromised--and requires only about half the bandwidth of LINEAR16 . This encoding includes the FLAC stream header followed by audio data. It supports 16-bit and 24-bit samples, however, not all fields in STREAMINFO are supported. |
AudioOut
The audio containing the assistant's response to the query. Sequential chunks of audio data are received in sequential ConverseResponse
messages.
Fields | |
---|---|
audio_data |
Output-only The audio data containing the assistant's response to the query. Sequential chunks of audio data are received in sequential |
AudioOutConfig
Specifies the desired format for the server to use when it returns audio_out
messages.
Fields | |
---|---|
encoding |
Required The encoding of audio data to be returned in all |
sample_rate_hertz |
Required The sample rate in Hertz of the audio data returned in |
volume_percentage |
Required Current volume setting of the device's audio output. Valid values are 1 to 100 (corresponding to 1% to 100%). |
Encoding
Audio encoding of the data returned in the audio message. All encodings are raw audio bytes with no header, except as indicated below.
Enums | |
---|---|
ENCODING_UNSPECIFIED |
Not specified. Will return result google.rpc.Code.INVALID_ARGUMENT . |
LINEAR16 |
Uncompressed 16-bit signed little-endian samples (Linear PCM). |
MP3 |
MP3 audio encoding. The sample rate is encoded in the payload. |
OPUS_IN_OGG |
Opus-encoded audio wrapped in an ogg container. The result will be a file which can be played natively on Android and in some browsers (such as Chrome). The quality of the encoding is considerably higher than MP3 while using the same bitrate. The sample rate is encoded in the payload. |
ConverseConfig
Specifies how to process the ConverseRequest
messages.
Fields | |
---|---|
audio_in_config |
Required Specifies how to process the subsequent incoming audio. |
audio_out_config |
Required Specifies how to format the audio that will be returned. |
converse_state |
Required Represents the current dialog state. |
ConverseRequest
The top-level message sent by the client. Clients must send at least two, and typically numerous ConverseRequest
messages. The first message must contain a config
message and must not contain audio_in
data. All subsequent messages must contain audio_in
data and must not contain a config
message.
Fields | ||
---|---|---|
Union field converse_request . Exactly one of these fields must be specified in each ConverseRequest . converse_request can be only one of the following: |
||
config |
The |
|
audio_in |
The audio data to be recognized. Sequential chunks of audio data are sent in sequential |
ConverseResponse
The top-level message received by the client. A series of one or more ConverseResponse
messages are streamed back to the client.
Fields | ||
---|---|---|
Union field converse_response . Exactly one of these fields will be populated in each ConverseResponse . converse_response can be only one of the following: |
||
error |
Output-only If set, returns a |
|
event_type |
Output-only Indicates the type of event. |
|
audio_out |
Output-only The audio containing the assistant's response to the query. |
|
result |
Output-only The semantic result for the user's spoken query. |
EventType
Indicates the type of event.
Enums | |
---|---|
EVENT_TYPE_UNSPECIFIED |
No event specified. |
END_OF_UTTERANCE |
This event indicates that the server has detected the end of the user's speech utterance and expects no additional speech. Therefore, the server will not process additional audio (although it may subsequently return additional results). The client should stop sending additional audio data, half-close the gRPC connection, and wait for any additional results until the server closes the gRPC connection. |
ConverseResult
The semantic result for the user's spoken query.
Fields | |
---|---|
spoken_request_text |
Output-only The recognized transcript of what the user said. |
spoken_response_text |
Output-only The text of the assistant's spoken response. This is only returned for an IFTTT action. |
conversation_state |
Output-only State information for subsequent |
microphone_mode |
Output-only Specifies the mode of the microphone after this |
volume_percentage |
Output-only Updated volume level. The value will be 0 or omitted (indicating no change) unless a voice command such as "Increase the volume" or "Set volume level 4" was recognized, in which case the value will be between 1 and 100 (corresponding to the new volume level of 1% to 100%). Typically, a client should use this volume level when playing the |
MicrophoneMode
Possible states of the microphone after a Converse
RPC completes.
Enums | |
---|---|
MICROPHONE_MODE_UNSPECIFIED |
No mode specified. |
CLOSE_MICROPHONE |
The service is not expecting a follow-on question from the user. The microphone should remain off until the user re-activates it. |
DIALOG_FOLLOW_ON |
The service is expecting a follow-on question from the user. The microphone should be re-opened when the AudioOut playback completes (by starting a new Converse RPC call to send the new audio). |
ConverseState
Provides information about the current dialog state.
Fields | |
---|---|
conversation_state |
Required The |