The ARCore SDK for iOS interfaces with ARKit to provide Cloud Anchor capabilities, letting you share anchors between iOS and Android devices in the same environment.
Learn how to use the ARCore API, or ARCore Cloud Anchor service, in your own apps.
Prerequisites
- Xcode version 13.0 or later
- Cocoapods 1.4.0 or later if using Cocoapods
- An ARKit-compatible Apple device running iOS 11.0 or later (deployment target of iOS 10.0 or later required)
If you are new to Cloud Anchors:
Make sure that you understand the process used to host and resolve a Cloud Anchor.
Read through the quickstart for system requirements, setup, and installation instructions.
Check out one of the Cloud Anchor samples
Enable Cloud Anchors in your app
To use the Cloud Anchors API, you have to create a
GARSessionConfiguration
and set the cloudAnchorMode property for it, as described in
Configure an ARCore session in iOS. Use
setConfiguration:error: (GARSession)
to set the configuration.
You must also enable the ARCore API for your Google Cloud Platform project.
Host and resolve anchors
You can host and resolve cloud anchors with the ARCore API. The API also includes delegate methods that provide callbacks on completed requests.
Host an anchor
Hosting an ARAnchor puts the anchor in a common coordinate system for any given
physical space.
A host request sends visual data to a Google server, which maps the ARAnchor’s
position in a coordinate system that represents the current physical space. This
request returns a GARAnchor without an ID. A successful host request assigns
the returned GARAnchor a unique ID in a later GARFrame.
- (void)addAnchorWithTransform:(matrix_float4x4)transform {
self.arAnchor = [[ARAnchor alloc] initWithTransform:transform];
[self.sceneView.session addAnchor:self.arAnchor];
self.garAnchor = [self.gSession hostCloudAnchor:self.arAnchor error:nil];
[self enterState:HelloARStateHosting];
}
Resolve an anchor
Resolving an ARAnchor allows Android and iOS devices in a given physical space
to add previously hosted anchors to new scenes.
A resolve request sends a Google server a cloud anchor ID along with visual data
from the current frame. This request returns a GARAnchor without a valid
transform or cloud anchor ID. The server will attempt to match this visual data
with the imagery of where currently hosted Cloud Anchors are mapped. A
successful host request assigns the GARanchor a valid transform and a unique
ID in a later GARFrame.
- (void)resolveAnchorWithIdentifier:(NSString *)identifier {
self.garAnchor = [self.gSession resolveCloudAnchorWithIdentifier:identifier error:nil];
}
// Pass the ARFRame to the ARCore session every time there is a frame update.
// This returns a GARFrame that contains a list of updated anchors. If your
// anchor's pose or tracking state changed, your anchor will be on the list.
- (void)cloudAnchorManager:(CloudAnchorManager *)manager didUpdateFrame:(GARFrame *)garFrame {
for (GARAnchor *garAnchor in garFrame.updatedAnchors) {
if ([garAnchor isEqual:self.garAnchor] && self.resolvedAnchorNode) {
self.resolvedAnchorNode.simdTransform = garAnchor.transform;
self.resolvedAnchorNode.hidden = !garAnchor.hasValidTransform;
}
}
}
Delegate methods for host and resolve requests
Host and resolve requests have
GARSessionDelegate
methods that provide callbacks upon request success and failure.
Host
session:didHostAnchor:session:didFailToHostAnchor:
Resolve
session:didResolveAnchor:session:didFailToResolveAnchor:
-(void)session:(ARSession *)arSession didUpdateFrame:(ARFrame *)arFrame {
[...]
-(void)session:(GARSession *)garSession didHostAnchor:(GARAnchor *)garAnchor {
// successful host
}
-(void)session:(GARSession *)garSession didFailToHostAnchor:(GARAnchor *)garAnchor {
// failed host
}
-(void)session:(GARSession *)garSession didResolveAnchor:(GARAnchor *)garAnchor {
// successful resolve
}
-(void)session:(GARSession *)garSession didFailToResolveAnchor:(GARAnchor *)garAnchor {
// failed resolve
}
Optional GARSession polling pattern
If you are using Metal or need a polling option, and your app runs at a
minimum of 30 fps, use the following pattern to pass ARFrames to the
GARSession:
-(void)myOwnPersonalUpdateMethod {
ARFrame *arFrame = arSession.currentFrame;
NSError *error = nil;
GARFrame *garFrame = [garSession update:arFrame error:&error];
// your update code here
}
Host a Cloud Anchor with persistence
Prior to ARCore v1.20, Cloud
Anchors could only be resolved for up to 24 hours after they were first hosted.
With persistent Cloud Anchors, you can now use
hostCloudAnchor:error: to create a Cloud Anchor with a time to live
(TTL) between one and 365 days. You can also extend the lifetime of the anchor
after it is already hosted using the
Cloud Anchor Management API.
/**
* This creates a new Cloud Anchor with a given lifetime in days, using the transform
* of the provided anchor.
*
* The cloud state of the returned anchor will be set to GARCloudAnchorStateTaskInProgress and the
* initial transform will be set to the transform of the provided anchor. However, the returned
* anchor and the original anchor are independent of one another, and their two transforms
* may diverge over time.
*
* Hosting requires a working Internet connection and an active session where the tracking state
* is ARTrackingStateNormal. If it is unable to establish a connection to the ARCore Cloud Anchor
* service, ARCore will continue to silently retry in the background.
*
* @param anchor The ARAnchor with the desired transform to be used to create a hosted Cloud
* Anchor.
* @param TTLDays The anchor’s lifetime in days. Must be a positive number. The maximum
* allowed value is 1 if you are using an API key to authorize the Cloud Anchor API call.
* Otherwise, the maximum allowed value is 365.
* @param error Out parameter for an NSError. Possible errors include:
* GARSessionErrorCodeInvalidArgument - Invalid (nil) anchor or invalid TTL.
* GARSessionErrorCodeNotTracking - Bad current ARTrackingState.
* GARSessionErrorCodeResourceExhausted - ARCore tried to create too many Cloud Anchors.
* @return The new GARAnchor, or nil if there is an error.
*/
- (GARAnchor *_Nullable)hostCloudAnchor:(ARAnchor *)anchor
TTLDays:(NSInteger)TTLDays
error:(NSError **)error;
Authorization
Your app must be authorized to use the Cloud Anchors API. You may use signed JSON Web Token (JWT) or API key authorization.
Token (signed JWT) authorization
Use signed JSON Web Token (JWT) authorization to host a Cloud Anchor for up to 365 days. Use API key authorization to host a Cloud Anchor for up to one day.
Currently, the only supported token type is a signed JWT (that is, a JSON Web token signed by a Google Service account). See the official JWT website for an introduction to JWTs. To generate JSON Web Tokens for iOS, you must have an endpoint on your server that satisfies the following requirements:
Your own authorization mechanism must protect the endpoint.
The endpoint must generate a new token every time, such that:
- Each user gets a unique token.
- Tokens don’t immediately expire.
Create a service account and signing key
Follow these steps to create a Google Service account and signing key:
In the navigation menu of the Google Cloud Platform console, go to APIs & Services > Credentials.
Select the desired project, then click Create Credentials > Service account.
Under Service account details, type a name for the new account, then click Create.
On the Service account permissions page, go to the Select a role dropdown. Select Service Accounts > Service Account Token Creator, then click Continue.
On the Grant users access to this service account page, click Done. This takes you back to APIs & Services > Credentials.
On the Credentials page, scroll down to the Service Accounts section and click the name of the account you just created.
On the Service account details page, scroll down to the Keys section and select Add Key > Create new key.
Select JSON as the key type and click Create. This downloads a JSON file containing the private key to your machine. Store the downloaded JSON key file in a secure location.
Create tokens on your server
To create new tokens (JWTs) on your server, use the standard JWT libraries and the JSON file that you securely downloaded from your new service account.
Create tokens on your development machine
To generate JWTs on your development machine, use the following oauth2l command:
oauth2l fetch --jwt --json $KEYFILE $AUDIENCE --cache ""
Specifying an empty cache location using the --cache flag is necessary to
ensure that a different token is produced each time. Be sure to trim the
resulting string because extra spaces or newline characters will cause ARCore to
reject the token.
Sign the token
You must use the RS256 algorithm and the following claims to sign the JWT:
iss— The service account email address.sub— The service account email address.iat— The Unix time when the token was generated, in seconds.exp—iat+3600(1 hour). The Unix time when the token expires, in seconds.aud— The audience. The correct ‘audience’ for the ARCore API ishttps://arcore.googleapis.com/.
Non-standard claims are not required in the JWT payload, though you may find the
uid claim useful for identifying the corresponding user.
If you use a different approach to generate your JWTs, such as using a Google API in a Google-managed environment, make sure to sign your JWTs with the claims in this section. Above all, make sure that the audience is correct.
Pass a token into the ARCore session
First, construct a session using sessionWithError::
NSError *error = nil;
GARSession *session = [GARSession sessionWithError:&error];
When you obtain a token, pass it into the session using setAuthToken:.
/**
* Provide an auth token to authorize your app to use the ARCore Cloud Anchor API. If
* you used an API key to create the session, ARCore will ignore the token and log an error.
* Otherwise, it will use the most recent valid auth token that you passed in. Call this
* method each time you refresh your token.
*
* @param authToken The token to use when authorizing your call to the ARCore Cloud Anchor API. This
* must be a nonempty ASCII string with no spaces or control characters. ARCore
* will use this until you pass in another token. Currently, JWTs are the only
* supported token types.
*/
- (void)setAuthToken:(NSString *)authToken;
Note the following when you pass a token into the session:
If you do not pass in a valid token before attempting to host or resolve an anchor, you will get authorization errors.
ARCore ignores tokens that contain spaces or special characters. ARCore also ignores all tokens if you create your session with a valid API key. If you previously used an API key and no longer need it, we recommend deleting it in the Google Developers Console and removing it from your app after migrating users to the newest version.
Tokens typically expire after one hour. If there is a possibility that your token may expire while in use, obtain a new token and pass it to the API.
API key authorization
Use API key authorization option to host a Cloud Anchor for up to one day.
Follow these steps to obtain and add an API key to your project:
See the Google Cloud Platform Console Help Center to obtain an API key.
In Xcode, pass the new API key to
sessionWithAPIKey:bundleIdentifier:error:to add it to your project:NSError *error = nil; GARSession *session = [GARSession sessionWithAPIKey:@"your-api-key" bundleIdentifier:nil error:&error];
Mapping quality
The mapping quality API estimates the quality of the visual feature points seen
by ARCore in the preceding few seconds and visible from the provided camera
transform. Cloud Anchors hosted using higher quality features will generally
result in easier and more accurately resolved Cloud Anchor transforms. If
feature map quality cannot be estimated for a given transform, ARCore will log a
warning message and return GARFeatureMapQualityInsufficient. This state
indicates that ARCore will likely have more difficulty resolving the Cloud
Anchor. Encourage the user to move the device, so that the desired position of
the Cloud Anchor to be hosted is viewed from different angles.
/**
* @param transform The camera transform to use to estimate the mapping quality.
* @param error Out parameter for an `NSError`. Possible errors:
* GARSessionErrorCodeNotTracking - Bad current ARTrackingState.
* @return The estimated quality of the visual feature points seen
* by ARCore in the preceding few seconds and visible from the provided camera
* transform.
*/
- (GARFeatureMapQuality)estimateFeatureMapQualityForHosting:(simd_float4x4)transform
error:(NSError **)error;
API quotas
The ARCore API has the following quotas for request bandwidth:
| Quota type | Maximum | Duration | Applies to |
|---|---|---|---|
| Number of anchors | Unlimited | N/A | Project |
| Anchor host requests | 30 | minute | IP address and project |
| Anchor resolve requests | 300 | minute | IP address and project |
Known issues and workarounds
There are a few known issues when working with the ARCore SDK for iOS.
Default scheme settings cause intermittent app crash
GPU Frame Capture and Metal API Validation scheme settings are enabled by default, which can sometimes cause the app to crash within the SDK.
Diagnose an app crash
Whenever you suspect that a crash has occurred, take a look at your stack trace.
If you see MTLDebugComputeCommandEncoder in the stack trace, it is likely due
to the default scheme settings.
Workaround
Go to Product > Scheme > Edit Scheme….
Open the Run tab.
Click Options to view your current settings.
Make sure that both GPU Frame Capture and Metal API Validation are disabled.
Build and run your app.
See the Cocoapods CHANGELOG for additional known issues.
Limitations
The ARCore SDK for iOS does not support the ARKit setWorldOrigin(relativeTransform:) method call.
Performance considerations
Memory usage increases when you enable the ARCore API. Expect the device’s battery usage to rise due to higher network usage and CPU utilization.