SDK Runtime UI presentation APIs

The SDK Runtime allows ads SDKs run in a sandboxed environment, preventing them from being able to access a publisher's view hierarchy. To display ads, the platform exposes a SandboxedSdkProvider.getView API to the SDK to obtain an ad view, and packages it up as a SurfacePackage to be sent over IPC (inter-process communication) to the client application. This has several drawbacks, which are discussed below. This document will then present a proposed Jetpack library that is being built to address these challenges.

Rationale for augmenting the platform APIs

The framework APIs are designed for flexibility and leave the task of building a side channel between for UI presentation up to the app and the SDK. This side channel does the following:

  1. Lets the SDK manage multiple ad views during their lifetime and understand what happens to the ad UI once it's been created by the SDK.
  2. Decouples view creation and content binding. Using the side channel allows the SDK to return an object that corresponds to the ad request to the app (the content), which can be bound to the ad container whenever the app deems appropriate.
  3. Abstracts away the underlying platform constructs used for showing UI across processes. (The platform currently uses a SurfaceControlViewhost and generates a SurfacePackage from it.)
  4. Enables ads SDKs in the SDK Runtime to automatically receive notifications when the UI of the ad container changes. If a publisher changes the layout of the ad container, the SDK remains unaware of these changes unless the publisher explicitly calls an API to notify it.
  5. Synchronizes resizes of the ad UI and the ad container without any user-visible jank.
  6. Manages backwards compatibility automatically. SurfacePackage is not available before API level 30. Additionally, on devices where there is no SDK runtime and the SDK is process-local to the publisher, it's wasteful to create a SurfacePackage for an ad when a view can be directly obtained from the SDK. The side channel abstracts this complexity away from the SDK and app developer code.
  7. Enables ad UI to integrate seamlessly with Composables. Jetpack Compose developers who don't work with views can also continue to host UI generated by the SDK developer who still works with views.

UI libraries

The UI libraries abstract away the complexities detailed above and provide the side channel that the publisher and SDK can use to show UI across processes, and keep it updated as the user interacts with it, and with the device.

There are three UI libraries: core, client, and provider. The core library provides the interfaces used by client and provider libraries. The UI provider (typically the SDK) depends on the provider library, and the consumer of the UI (typically the publisher) depends on the client library. Together, the client and provider libraries form the side channel required for creating and maintaining a UI session.

The APIs

The APIs for SDK Runtime UI presentation are as follows:

SandboxedUiAdapter: Created by the SDK, providing a way to obtain content to be displayed in the publisher's UI.

SandboxedSdkView: Created by the publisher, this is a container that holds content obtained through the SandboxedUiAdapter.

Session: Created by the SDK in response to the SandboxedUiAdapter.openSession(). Represents one UI session. call. This forms the SDK end of the communication tunnel between the SDK and the publisher, and receives notifications about changes in SandboxedSdkView, such as window detachments, resizes, or configuration changes.

SessionClient: Created by the client library, this forms the publisher end of the communication tunnel between the SDK and the publisher.

SandboxedSdkUiSessionStateChangedListener: Created by the publisher. A listener for changes to the state of the UI session associated with SandboxedSdkView.

Illustration that shows SDK Runtime UI presentation API relationships.
Relationships between the SDK Runtime UI presentation APIs.

Read the privacysandbox-ui reference documentation for further details on these APIs.

Control flow

The following diagrams show the interaction between the client and provider UI libraries in various scenarios:

The previous diagram shows how the publisher can create a SandboxedSdkView programmatically or through their XML and attach it to a SdkSandboxUiAdapter obtained from the SDK through an SDK-defined API. To observe all UI state changes, the publisher should add a SandboxedSdkUiSessionStateChangedListener to SandboxedSdkView before attaching SdkSandboxUiAdapter.

Illustration showing the open session process.
Obtain UI from the SDK.

This diagram shows how if the publisher's activity handles configuration changes, the client library takes care of forwarding the configuration change to the SDK, so they can update their UI accordingly. For example, this flow can be triggered when the user rotates the device and the publisher declares handling configuration changes in their activity, by setting android:configChanges=["orientation"].

Publisher-initiated UI change.

This diagram illustrates how the SDK can request a change in the ad container using methods on SessionClient. This API is triggered when the SDK wants to resize the ad and needs the publisher to resize the ad container to accommodate the new dimensions. This might happen in response to user interaction, such as mraid.resize().

SDK-initiated UI change.

This diagram shows how the session is closed when the SandboxedSdkView is detached from the window. The session can also be closed at any point (e.g. when the user loses network connectivity) by the SDK by invoking SessionClient.onSessionError().

Closing the UI session.

Z order

The client UI library uses a SurfaceView internally to host the SDK's UI. SurfaceView can use Z order to either show its UI on top of the publisher's window, or below it. This is controlled by the SandboxedSdkView.orderProviderUiAboveClientUi() method, which accepts a boolean setOnTop.

When setOnTop is true, every android.view.MotionEvent on the SandboxedSdkView is sent to the SDK. When false, these are sent to the publisher. By default, motion events are sent to the SDK.

Publishers typically don't need to change the default Z-order of ad views. However, when showing UI that covers an ad, such as a drop-down menu, the Z-order should be temporarily flipped from the default and then restored when the covering UI element is dismissed. We are exploring ways to automate this process in the client UI library.

Scrolling

When the ad UI is ordered Z-above the publisher window, MotionEvents from the ad UI are sent to the SDK. Scroll and fling gestures initiated on the ad UI get special treatment:

  1. Vertical scroll and fling gestures are sent to and handled by the publisher's container. This provides for good UX when the publisher's container that the ad UI is placed in, is vertically scrollable. This doesn't require any extra work on the SDK or publisher's part.
  2. Horizontal scroll and fling gestures are sent to and handled by the SDK. This provides good UX when the ad UI itself is horizontally scrollable (such as an ad carousel).

Implementation guide

The SDK should implement the following:

  • SandboxedUiAdapter: This is returned to the publisher in response to an SDK-defined API, such as loadAd. The openSession() method of this implementation should be used to make an ad request to the SDK's servers and prepare an ad view for that request.
  • Session**: This is returned in response to the SandboxedUiAdapter.openSession call. It provides a way for the client library to obtain the ad UI and notify the SDK about changes to this API. All Session methods should be implemented here.

The publisher should do the following:

  1. Create a SandboxedSdkView, either through XML or programmatically.
  2. Attach a SandboxedSdkUiSessionStateChangedListener to the SandboxedSdkView, to observe changes in UI.
  3. Attach an SDK provided SandboxedUiAdapter to the SandboxedSdkView.
  4. Add SandboxedSdkView to the window as usual, and let the client library take care of creating and maintaining the UI session with the SDK.
  5. At appropriate times, react to changes in the state reported by SandboxedSdkUiSessionChangedListener. For example, if the SDK closes the session unexpectedly, the publisher can replace SandboxedSdkView with a static image, or remove it from their view hierarchy.
  6. When making transitions that might cover the ad UI, such as a drop down menu, temporarily orderProviderUiAboveClientUi to false to position the ad UI below the publisher's window. Once the drop down menu is dismissed, call orderProviderUiAboveClientUi to true.

The future of the platform APIs

Once the UI libraries go into Beta, we plan to deprecate the SDK runtime platform APIs related to UI presentation, namely SdkSandboxManager.requestSurfacePackage() and SandbxedSdkProvider.getView().

Open questions

  1. Are there more common ad UI use cases that the UI libraries should automatically handle?
  2. Which UI frameworks do you use to show ad UI, do you anticipate problems in integrating the UI libraries with these frameworks?
  3. Is scrollable ad UI placed in a scrollable publisher container a common use case for you? What's the directionality of scroll for the ad UI and the container in this case? What behavior do you expect when the user initiates a scroll on the ad UI?