Record and play back an AR session on Android

The Recording & Playback API enables you to record video and AR data once within a given environment and use that content to replace a live camera session.

Prerequisites

Make sure that you understand fundamental AR concepts and how to configure an ARCore session before proceeding.

Compatibility with other ARCore APIs

Due to the way session data is processed, ARCore APIs may produce different results during playback than observed during recording. They may also produce different results during subsequent playback sessions. For example, the number of detected trackables, the precise timing of their detection, and their poses over time may be different during playback.

Compatibility with Shared Camera

Sessions that use Shared Camera can be recorded. However, playback for these sessions using Shared Camera mode is currently unavailable.

Compatibility with Cloud Anchors

You can host and resolve Cloud Anchors while recording or playing back a session.

Recording

Start, stop, and check the status of an ARCore session recording.

Record an ARCore session

To record an ARCore session, configure the session and provide an MP4 URI for the recording. Call session.startRecording() before the first call to session.resume(). Recording automatically starts when the session resumes. To automatically stop recording when the session is paused, call RecordingConfig.setAutoStopOnPause(). To record a partial session, call session.startRecording() while the session is running.

Java

// Configure the ARCore session.
Session session = new Session(context);
Uri destination = Uri.fromFile(new File(context.getFilesDir(), "recording.mp4"));
RecordingConfig recordingConfig =
        new RecordingConfig(session)
        .setMp4DatasetUri(destination)
        .setAutoStopOnPause(true);
try {
  // Prepare the session for recording, but do not start recording yet.
  session.startRecording(recordingConfig);
} catch (RecordingFailedException e) {
  Log.e(TAG, "Failed to start recording", e);
}

// Resume the ARCore session to start recording.
session.resume();

Kotlin

// Configure the ARCore session.
val session = Session(context)
val destination = Uri.fromFile(File(context.getFilesDir(), "recording.mp4"))
val recordingConfig = RecordingConfig(session)
  .setMp4DatasetUri(destination)
  .setAutoStopOnPause(true)
session.startRecording(recordingConfig)

// Resume the ARCore session to start recording.
session.resume()

Stop a session recording

To stop recording without pausing the currently running AR session, call session.stopRecording().

Java

try {
  session.stopRecording();  // Stop recording.
} catch (RecordingFailedException e) {
  Log.e(TAG, "Failed to stop recording", e);
}

Kotlin

session.stopRecording()

Check recording status

session.getRecordingStatus() can be used at any time to determine the current RecordingStatus.

Java

// Use any time to determine current RecordingStatus.
if (session.getRecordingStatus() == RecordingStatus.OK) {
  // Update the UI to show that the session is currently being recorded.
}

Kotlin

// Use any time to determine current RecordingStatus.
if (session.recordingStatus == RecordingStatus.OK) {
  // Update the UI to show that the session is currently being recorded.
}

Playback

Play back previously recorded AR sessions. Sessions play back in real time, and session playback or speed cannot be adjusted.

Play back a previously recorded session

To play back a previously recorded session, call session.setPlaybackDatasetUri() before the first call to session.resume().

Once playback has started due to the first call to session.resume(), pausing the session by calling session.pause() will suspend processing of all camera image frames and any other recorded sensor data in the dataset. Camera image frames and sensor frame data that is discarded in this way will not be reprocessed when the session is again resumed by calling session.resume(). AR tracking for the session will generally suffer due to the gap in processed data.

Java

// Configure the ARCore session.
Session session = new Session(context);

// Specify the previously recorded MP4 file.
Uri recordingUri = Uri.fromFile(new File(context.getFilesDir(), "recording.mp4"));
session.setPlaybackDatasetUri(recordingUri);
…

// Start playback from the beginning of the dataset.
session.resume();
…

// Pause the AR session, but silently continue MP4 playback. Camera frames
// and other recorded sensor data is discarded while the session is paused.
session.pause();
…

// Resume the AR session. Camera frames and other sensor data from the MP4
// that was silently played back while the session was paused is not
// processed by ARCore.
session.resume();

Kotlin

// Configure the ARCore session.
val session = Session(context)

// Specify the previously recorded MP4 file.
val recordingUri = Uri.fromFile(File(context.filesDir, "recording.mp4"))
session.playbackDatasetUri = recordingUri
…

// Start playback from the beginning of the dataset.
session.resume()
…

// Pause the AR session, but silently continue MP4 playback. Camera frames
// and other recorded sensor data is discarded while the session is paused.
session.pause()
…

// Resume the AR session. Camera frames and other sensor data from the MP4
// that was silently played back while the session was paused is not
// processed by ARCore.
session.resume()

Restart playback from the beginning

To restart a playback from the beginning of the dataset, pause the session and call session.setPlaybackDatasetUri(), specifying the same MP4 recording before resuming the session.

Java

session.pause();
// Pause and specify the SAME dataset:
session.setPlaybackDatasetUri(previousRecordingUri);
session.resume();  // Playback starts from the BEGINNING of the dataset.

Kotlin

session.pause()
// Pause and specify the SAME dataset:
session.playbackDatasetUri = previousRecordingUri
session.resume()  // Playback starts from the BEGINNING of the dataset.

Play back a different session

To play back a different dataset, pause the session and specify the new dataset before resuming the session.

Java

// Switch to a different dataset.
session.pause();   // Pause the playback of the first dataset.
// Specify a different dataset to use.
session.setPlaybackDatasetUri(newRecordingUri);
session.resume();  // Start playback from the beginning of the new dataset.

Kotlin

// Switch to a different dataset.
session.pause()   // Pause the playback of the first dataset.
// Specify a different dataset to use.
session.playbackDatasetUri = newRecordingUri
session.resume()  // Start playback from the beginning of the new dataset.

Check playback status

Use session.getPlaybackStatus() at any time to determine the current PlaybackStatus.

Java

// Use any time to determine current PlaybackStatus.
if (session.getPlaybackStatus() != PlaybackStatus.OK) {
  // Update the UI to show that the session playback has finished.
}

Kotlin

// Use any time to determine current PlaybackStatus.
if (session.playbackStatus != PlaybackStatus.OK) {
  // Update the UI to show that the session playback has finished.
}

What’s next