Augmented Faces developer guide for Unity

Learn how to use Augmented Faces in your own apps.

Prerequisites

Make sure that you understand fundamental AR concepts and how to configure an ARCore session before proceeding.

Prerequisites

• This guide assumes you have already installed and configured Unity. If not, see the Unity Quickstart for Android for installation and setup steps.

• Make sure you have assets for overlaying on a face.

Build and run the sample app

1. Create a new project in Unity based on the quickstart.

2. In Build Settings > Scenes in Build specify the AugmentedFaces scene from Assets/GoogleARCore/Examples/AugmentedFaces/Scenes/.

3. Make sure your device is connected to your machine and then click Build and Run. Unity builds your project into an Android APK, installs it on the device, and launches it.

   The app should open the front camera and immediately begin tracking your face in the camera feed. It should place images of fox ears over both sides of your forehead, and place a fox nose over your own nose.

Importing assets into Unity

To add your own assets such as textures and 3D models to a detected face in your app, first import the assets into Unity.

Overview of implementing Augmented Faces in your app

Configure your app to support Augmented Faces

Configuring your app by adding Augmented Faces to your ARCore session, setting up the scene, and enabling the front camera.

After you configure your app to support Augmented Faces, it will automatically create an augmented face mesh from the first detected face for each scene.

1. In Unity, create a new session configuration for the ARCore app (Assets > Create > Google ARCore > SessionConfig).

2. In the Inspector for the ARCore session configuration asset, set Augmented Face Mode to Mesh.

3. Create and configure the ARCore session to support Augmented Faces as follows:

   1. In the Assets folder, go to Prefab and drag ARCore Device to the scene.
   2. Replace the existing session configuration with the new one you previously imported.
   3. In the Inspector, set Device Camera Direction to Front.

Face mesh orientation

Note the orientation of the face mesh for Unity:

Attach a texture to the face mesh

When attaching a texture to the face mesh, the app overlays and binds the texture onto the identified vertices and surface of the face.

1. Import a texture into Unity.
2. Create a new empty GameObject.
3. In the GameObject, add a new ARCoreAugmentedFaceMeshFilter component from the ARFaces sample in the SDK. Pro Tip: In the project window in Unity, search for ARCoreAugmentedFaceMeshFilter to find it.
4. To ensure assets automatically bind to the augmented face mesh in the GameObject, select Auto Bind.
5. Add a mesh renderer.
6. Create a material that uses the texture you want to attach to the face.
7. Add the material to the renderer.

Attach a model to a region on the face

Follow these steps to place the root of the skeleton at the root of the center pose in the face. This also automatically updates the three regions to adhere to the face mesh.

1. Import the 3D model into Unity.
2. Add it to the scene.
3. Add the rig. In this case you should see the regions (for example, FOREHEAD_LEFT).
4. Click Add Component, and select the component of type AR Core Augmented Face Rig from the ARFaces sample in the SDK.
5. Select the Auto Bind option. This automatically binds the model to the facial regions.