Use thinking mode

Thinking mode prompts the model to output its reasoning process before generating a final response. Thinking mode improves the model's function-calling accuracy and works well for math, problem-solving, and other complex reasoning tasks.

When thinking mode is enabled, the model output is divided into two parts:

1. Thought process: The internal reasoning or planning steps the model takes.
2. Response: The final answer generated for the user.

Prerequisites

Thinking mode works on devices running Gemini Nano V4 and higher. You can test thinking mode through the Developer Preview program on any AICore-enabled device.

Enable thinking mode

To use thinking mode, you must opt in by setting the enableThinking flag to true in your generateContentRequest builder function:

val request = generateContentRequest {
    text("Solve this complex riddle: ...")
    enableThinking = true // Enable thinking mode for Gemini Nano V4
}

Access thoughts

The ML Kit SDK provides different ways to access the thoughts that precede a model's response, depending on whether you are using a non-streaming or streaming implementation.

Non-streaming mode

In standard non-streaming calls, the reasoning process is returned as part of the full response. You can access this process using the thoughtProcess property—which contains a list of Candidate objects—on the GenerateContentResponse object.

try {
    val response = generativeModel.generateContent(request)
    
    // 1. Access the main response
    val answer = response.candidates.firstOrNull()?.text
    println("Final Answer: $answer")
    
    // 2. Access the separated thought process
    println("Thought Process:")
    response.thoughtProcess.forEach { thoughtCandidate ->
        print(thoughtCandidate.text)
    }
} catch (e: GenAiException) {
    // Handle SDK-specific exceptions
}

Streaming mode

To support real-time feedback, the StreamingCallback interface has been updated to include the onNewThought method. This enables your app to handle streaming thought chunks as they are generated by the model.

val callback = object : StreamingCallback {
    override fun onNewText(additionalText: String) {
        // Called for regular response chunks
        print(additionalText)
    }

    override fun onNewThought(additionalThought: String) {
        // Called for thought chunks before final text arrives
        // Update a separate reasoning UI or status indicator
        print("[Thinking: $additionalThought]")
    }
}

try {
    generativeModel.generateContent(request, callback)
} catch (e: GenAiException) {
    // Handle exception
}

Best practices

Here are some best practices when using thinking mode:

• UI indicators: Because reasoning steps can take time to generate, displaying the thoughts or a "thinking" indicator in streaming mode can significantly improve perceived performance by showing the user the model is active.
• Selective implementation: If you don't want to expose raw reasoning to end-users, you can ignore the thoughtProcess field or omit the onNewThought implementation. The onNewThought method has a default empty implementation to help ensure backward compatibility.

For more tips, see Thinking mode in Gemma. Note that multi-turn conversations aren't currently supported in AICore.