ARCore is back at Google I/O on May 14! Register now.

Instant Placement developer guide for Android NDK

Learn how to use the Instant Placement API in your own apps.

Prerequisites

Make sure that you understand fundamental AR concepts and how to configure an ARCore session before proceeding.

Configure a new session with Instant Placement

In a new ARCore session, enable Instant Placement mode.

// Create a session config.
ArConfig* ar_config = NULL;
ArConfig_create(ar_session, &ar_config);

// Enable Instant Placement mode.
ArConfig_setInstantPlacementMode(ar_session, ar_config,
                                 AR_INSTANT_PLACEMENT_MODE_LOCAL_Y_UP);
CHECK(ArSession_configure(ar_session, ar_config) == AR_SUCCESS);

// Release config resources.
ArConfig_destroy(ar_config);

Place an object

In a new ARCore session, perform an Instant Placement hit-test with ArFrame_hitTestInstantPlacement. Then create a new ArAnchor using the ArInstantPlacementPoint pose from the hit result's ARTrackable.

ArFrame* ar_frame = NULL;
if (ArSession_update(ar_session, ar_frame) != AR_SUCCESS) {
  // Get the latest frame.
  LOGE("ArSession_update error");
  return;
}

// Place an object on tap.
// Use the estimated distance from the user's device to the closest
// available surface, based on expected user interaction and behavior.
float approximate_distance_meters = 2.0f;

ArHitResultList* hit_result_list = NULL;
ArHitResultList_create(ar_session, &hit_result_list);
CHECK(hit_result_list);

// Returns a single result if the hit test was successful.
ArFrame_hitTestInstantPlacement(ar_session, ar_frame, x, y,
                                approximate_distance_meters, hit_result_list);

int32_t hit_result_list_size = 0;
ArHitResultList_getSize(ar_session, hit_result_list, &hit_result_list_size);
if (hit_result_list_size > 0) {
  ArHitResult* ar_hit_result = NULL;
  ArHitResult_create(ar_session, &ar_hit_result);
  CHECK(ar_hit_result);
  ArHitResultList_getItem(ar_session, hit_result_list, 0, ar_hit_result);
  if (ar_hit_result == NULL) {
    LOGE("ArHitResultList_getItem error");
    return;
  }

  ArTrackable* ar_trackable = NULL;
  ArHitResult_acquireTrackable(ar_session, ar_hit_result, &ar_trackable);
  if (ar_trackable == NULL) {
    LOGE("ArHitResultList_acquireTrackable error");
    return;
  }
  ArTrackableType ar_trackable_type = AR_TRACKABLE_NOT_VALID;
  ArTrackable_getType(ar_session, ar_trackable, &ar_trackable_type);

  if (ar_trackable_type == AR_TRACKABLE_INSTANT_PLACEMENT_POINT) {
    ArInstantPlacementPoint* point = (ArInstantPlacementPoint*)ar_trackable;

    // Gets the pose of the Instant Placement point.
    ArPose* ar_pose = NULL;
    ArPose_create(ar_session, NULL, &ar_pose);
    CHECK(ar_pose);
    ArInstantPlacementPoint_getPose(ar_session, point, ar_pose);

    // Attaches an anchor to the Instant Placement point.
    ArAnchor* anchor = NULL;
    ArStatus status = ArTrackable_acquireNewAnchor(ar_session, ar_trackable,
                                                   ar_pose, &anchor);
    ArPose_destroy(ar_pose);
    // Render content at the anchor.
    // ...
  }

  ArTrackable_release(ar_trackable);
}

Instant Placement supports screen space tracking with approximate distance, automatically switching to full tracking once the Instant Placement point is anchored in the real world. Retrieve the current pose with ArInstantPlacementPoint_getPose(). Get the current tracking method with ArInstantPlacementPoint_getTrackingMethod().

Although ARCore can perform Instant Placement hit-tests against surfaces of any orientation, hit results will always return a pose with +Y up, against the direction of gravity. On horizontal surfaces, hit-tests return accurate positions much faster.

Monitor the Instant Placement point tracking method

If ARCore has an accurate 3D pose for the ArInstantPlacementPoint returned by ArFrame_hitTestInstantPlacement, it starts with tracking method AR_INSTANT_PLACEMENT_POINT_TRACKING_METHOD_FULL_TRACKING. Otherwise, it will start with tracking method AR_INSTANT_PLACEMENT_POINT_TRACKING_METHOD_SCREENSPACE_WITH_APPROXIMATE_DISTANCE and transition to AR_INSTANT_PLACEMENT_POINT_TRACKING_METHOD_FULL_TRACKING once ARCore has an accurate 3D pose. Once the tracking method is AR_INSTANT_PLACEMENT_POINT_TRACKING_METHOD_FULL_TRACKING, it will not revert to AR_INSTANT_PLACEMENT_POINT_TRACKING_METHOD_SCREENSPACE_WITH_APPROXIMATE_DISTANCE.

Smooth the tracking method transition

When the tracking method changes from AR_INSTANT_PLACEMENT_POINT_TRACKING_METHOD_SCREENSPACE_WITH_APPROXIMATE_DISTANCE in one frame to AR_INSTANT_PLACEMENT_POINT_TRACKING_METHOD_FULL_TRACKING in the next frame, the pose jumps from its initial location based on the provided approximate distance to a new location at an accurate distance. This instantaneous change in pose changes the apparent scale of any objects that are anchored to the ArInstantPlacementPoint. That is, an object suddenly appears larger or smaller than it was in the previous frame.

Follow these steps to avoid the visual jump due to the sudden change in apparent object scale:

Keep track of the pose and tracking method of the ArInstantPlacementPoint in each frame.
Wait for the tracking method to change to AR_INSTANT_PLACEMENT_POINT_TRACKING_METHOD_FULL_TRACKING.
Use the pose from the previous frame and the pose in the current frame to determine the object's distance to the device in both frames.
Calculate the apparent change in scale due to the change in distance from the camera.
Adjust the scale of the object to counteract the perceived change in scale, so that the object does not appear to visually change in size.
Optionally, smoothly adjust the scale of the object back to its original value over several frames.

class WrappedInstantPlacement {
  ArInstantPlacementPoint* point;
  ArInstantPlacementPointTrackingMethod previous_tracking_method;
  float previous_distance_to_camera;
  float scale_factor = 1.0f;

 public:
  WrappedInstantPlacement(ArInstantPlacementPoint* point,
                          TrackingMethod previous_tracking_method,
                          float previous_distance_to_camera) {
    this.point = point;
    this.previous_tracking_method = previous_tracking_method;
    this.previous_distance_to_camera = previous_distance_to_camera;
  }
};

std::vector<WrappedInstantPlacement> wrapped_points_;

After creating the Instant Placement point, modify OnTouched() to wrap it.

if (ar_trackable_type == AR_TRACKABLE_INSTANT_PLACEMENT_POINT) {
  ArInstantPlacementPoint* point = (ArInstantPlacementPoint*)ar_trackable;
  ArInstantPlacementPointTrackingMethod tracking_method;
  ArInstantPlacementPoint_getTrackingMethod(ar_session, point,
                                            &tracking_method);
  ArCamera* ar_camera = nullptr;
  ArFrame_acquireCamera(ar_session, ar_frame, &ar_camera);
  CHECK(ar_camera);
  wrapped_points_.push_back(WrappedInstantPlacement(
      point, tracking_method, Distance(point, ar_camera)));
}

When the tracking method of the Instant Placement point transitions from AR_INSTANT_PLACEMENT_POINT_TRACKING_METHOD_SCREENSPACE_WITH_APPROXIMATE_DISTANCE to AR_INSTANT_PLACEMENT_POINT_TRACKING_METHOD_FULL_TRACKING, use the distance saved to counteract the apparent scale change.

void OnUpdate() {
  for (auto& wrapped_point : wrapped_points_) {
    ArInstantPlacementPoint* point = wrapped_point.point;

    ArTrackingState tracking_state = AR_TRACKING_STATE_STOPPED;
    ArTrackable_getTrackingState(ar_session, (ArTrackable)point,
                                 &tracking_state);

    if (tracking_state == AR_TRACKING_STATE_STOPPED) {
      wrapped_points_.remove(wrapped_point);
      continue;
    }
    if (tracking_state == AR_TRACKING_STATE_PAUSED) {
      continue;
    }

    ArInstantPlacementPointTrackingMethod tracking_method;
    ArInstantPlacementPoint_getTrackingMethod(ar_session, point,
                                              &tracking_method);
    ArCamera* ar_camera = nullptr;
    ArFrame_acquireCamera(ar_session, ar_frame, &ar_camera);
    CHECK(ar_camera);
    const float distance_to_camera = Distance(point, ar_camera);
    if (tracking_method ==
        AR_INSTANT_PLACEMENT_POINT_TRACKING_METHOD_SCREENSPACE_WITH_APPROXIMATE_DISTANCE) {
      // Continue to use the estimated depth and pose. Record the distance to
      // the camera for use in the next frame if the transition to full
      // tracking happens.
      wrapped_point.previous_distance_to_camera = distance_to_camera;
    } else if (
        tracking_method ==
            AR_INSTANT_PLACEMENT_POINT_TRACKING_METHOD_FULL_TRACKING &&
        wrapped_point.previous_tracking_method ==
            AR_INSTANT_PLACEMENT_POINT_TRACKING_METHOD_SCREENSPACE_WITH_APPROXIMATE_DISTANCE) {
      // Change from the estimated pose to the accurate pose. Adjust the
      // object scale to counteract the apparent change due to pose jump.
      wrapped_point.scale_factor =
          distance_to_camera / wrapped_point.previous_distance_to_camera;
      // Apply the scale factor to the model.
      // ...
      previous_tracking_method =
          AR_INSTANT_PLACEMENT_POINT_TRACKING_METHOD_FULL_TRACKING;
    }
  }
}

Performance considerations

When Instant Placement is enabled, ARCore consumes additional CPU cycles. If performance is a concern, consider disabling Instant Placement after the user has successfully placed their object and the tracking method of all Instant Placement points have changed to AR_INSTANT_PLACEMENT_POINT_TRACKING_METHOD_FULL_TRACKING.

When Instant Placement is disabled, use ArFrame_hitTest instead of ArFrame_hitTestInstantPlacement.

ArConfig* ar_config = NULL;
ArConfig_create(ar_session, &ar_config);
// Disable Instant Placement.
ArConfig_setInstantPlacementMode(ar_session, ar_config,
                                 AR_INSTANT_PLACEMENT_MODE_DISABLED);
CHECK(ArSession_configure(ar_session, ar_config) == AR_SUCCESS);
ArConfig_destroy(ar_config);