Render your AR app using Vulkan on Android NDK (C)

When the ArTextureUpdateMode is set to AR_TEXTURE_UPDATE_MODE_EXPOSE_HARDWARE_BUFFER, ARCore will provide an Android hardware buffer when ArSession_update() is called. This hardware buffer can be bound to a Vulkan VkImage.

View the sample application

Vulkan rendering support is demonstrated in the hello_ar_vulkan_c sample app.

Enable the hardware buffer output mode

The configured ArTextureUpdateMode determines how ARCore will update the camera texture. When it is set to AR_TEXTURE_UPDATE_MODE_EXPOSE_HARDWARE_BUFFER, ARCore will provide the camera image through a AHardwareBuffer.

Configure the session to use AR_TEXTURE_UPDATE_MODE_EXPOSE_HARDWARE_BUFFER:

ArConfig* ar_config = NULL;
ArConfig_create(ar_session, &ar_config);
ArConfig_setTextureUpdateMode(ar_session, ar_config,
                              AR_TEXTURE_UPDATE_MODE_EXPOSE_HARDWARE_BUFFER);
CHECK(ArSession_configure(ar_session, ar_config) == AR_SUCCESS);
ArConfig_destroy(ar_config);

Obtain the hardware buffer

When AR_TEXTURE_UPDATE_MODE_EXPOSE_HARDWARE_BUFFER is enabled, use ArFrame_getHardwareBuffer() to get the hardware buffer:

void* native_hardware_buffer = NULL;
ArFrame_getHardwareBuffer(ar_session, ar_frame, &native_hardware_buffer);

if ((int64_t)native_hardware_buffer == 0) {
  // The hardware buffer isn't ready yet.
  return;
}

Use the hardware buffer during Vulkan rendering

See vulkan_handler.cc for an example of how to render an AR application using Vulkan.

Supported devices

Vulkan rendering support is only available on Android API levels 27 and above. Additionally, the device must support the VK_ANDROID_external_memory_android_hardware_buffer extension.

Require Vulkan in your app's manifest

Google Play uses <uses-feature> declared in your app manifest to filter your app from devices that don't meet its hardware and software feature requirements. Devices using Vulkan 1.0 might not support the required extension, but devices compatible with Vulkan 1.1 must have the required extension starting in Android 10 (API level 29).