Migrate from Entity Read Files

Entity Read Files (ERFs) are JSON representations of a partner's campaign objects that, on request, are generated daily and made available through Google Cloud Storage.

ERFs were deprecated in June 2021. Starting October 31, 2024, ERFs will officially be sunset and no longer generated. All existing Entity Read File users are encouraged to migrate to the Display & Video 360 API in order to continue retrieving Display & Video 360 resources.

This guide discusses how to migrate from Entity Read Files to the Display & Video 360 API by:

Giving an overview of the differences between the two interfaces
Comparing ERF tables to API services
Providing guidance on entity retrieval through the API
Acknowledging existing data gaps
Presenting a mapping of all ERF fields to comparable API resource fields

Overview

When migrating from ERFs to the Display & Video 360 API, there are a number of key differences to consider, including:

Data freshness. ERFs are generated daily and in bulk while the API retrieves the most up-to-date version of a resource.
Resource structure. The API uses different JSON structures than ERF to represent the same resource types. Some resources, such as public targeting settings, may use a different ID space.
Retrieval method. The Display & Video 360 API only enables the retrieval of resources individually, in paginated lists, or through BigQuery data transfers, in contrast to the raw JSON files provided by ERF.
Scope. As opposed to ERFs, which are scoped by partner ID, most API resources are scoped by advertiser ID. Resources included in responses are limited to resources within that scope.

ERF data representation in the API

Entity Read Files are separated into "Public" and "Private" tables. Public tables provide information that is available and applicable to all users, such as targeting values. Private tables provide data that is specific to a partner, such as creative or line item resources.

The Display & Video 360 API does not use this dichotomy, instead making all of this information retrievable through various services and using different JSON structures. This section compares the information provided through public and private ERF tables to that made available through Display & Video 360 API resources and services.

Public information

ERF public tables provide reference materials for users to use when interpreting the targeting settings of their retrieved private resources and assigning targeting through a subset of Structured Data Files (SDFs) versions uploaded through the UI. These reference materials are the same for all users and consist of a numeric ID, used for mapping, and more descriptive details, such as a display name.

When using the Display & Video 360 API, targeting reference information can be retrieved through the targetingTypes.targetingOptions service. Similar to the public tables, this service provides the IDs and details of targeting options for a specific targeting type. Refer to our existing Set Targeting page for a code example demonstrating targeting option ID retrieval.

Public tables and SDFs

Before SDF v7, Entity Read Files and Structured Data Files use the same ID space for targeting settings. If you are an SDF user using ERF public tables for interpreting or assigning targeting settings using SDF, you can instead download this reference material in CSV format through the Display & Video 360 UI.

Starting with v7, the ID spaces used by a subset of Structured Data Files columns have been updated to decouple SDF from the ERFs and further align with Display & Video 360 API. See the v7 migration guide and reference documentation for more information.

Private resources

ERF private tables provide a daily snapshot of the current settings of private resources owned by a partner. Due to the sheer volume of resources that can be created under a single partner, these files can grow to be very large and difficult to download and process.

In the API, each private table has a corresponding service that provides endpoints for retrieval and management of that resource type. Resources can be retrieved in bulk using each service's respective list method. The JSON structure for each resource is different in the API compared to ERF, utilizing different field names and shared resources.

Certain information available in the ERF representation of a resource, such as a resource's assigned targeting settings or a channel's sites, are represented in the API as children of the original resource and must be retrieved through additional API requests.

Entity retrieval in the API

Display & Video 360 resources can either be retrieved through direct API requests or automatic imports into BigQuery.

Direct API requests

Each resource type is retrievable through a different API service. Resources can be retrieved individually or in bulk using the appropriate service's get or list method, respectively. Important properties of Display & Video 360 API list methods include:

Required scope. Unlike ERFs, which are scoped by partner, most resources in the API are scoped by advertiser. Retrieving all of a resource type, such as line items, under a partner may require an individual list request for each child advertiser of that partner. Exceptions include direct children of a partner, such as advertisers and partner-owned channels.
Pagination. API list methods employ pagination to guarantee that responses are within a reasonable size, limiting most individual request responses, or pages, to 100 resources. If the number of relevant resources is larger than the page size, consecutive list calls are required to retrieve subsequent pages of the full list response. A code example paging a list response is provided in a section of our Targeting guide page regarding retrieving available targeting options .
Additional requests required for targeting retrieval. A resource's targeting settings are not included in its API JSON object, but are instead child resources known as assigned targeting options. These child resources must be retrieved through a separate request. For example, for each line item retrieved through an advertisers.lineItems.list request, a separate advertisers.lineItems.bulkListAssignedTargetingOptions request must be made to retrieve all targeting information.

Optimize resource retrieval

The Display & Video 360 API may require multiple requests to retrieve the same amount of information that is available in a single Entity Read File. Optimizing how you retrieve resources can help retrieve the data you need more efficiently:

Make concurrent requests to the API. The Display & Video 360 API protects infrastructure using requests per advertiser per project rate limits. This quota structure lets you implement a multithreaded solution across multiple advertisers that will reduce the total time it takes to retrieve all necessary resources. Although pagination requires that all resources of a type within a certain scope are retrieved through consecutive calls, retrieval of resources within another scope or of another type can be done concurrently.
Utilize filters and order by parameters in your list calls to retrieve only relevant resources. For example, if you are only interested in line items that have been updated in the last day, you can use the filter parameter of the advertisers.lineItems.list method to only return line items with an updateTime greater than a given timestamp. This can significantly reduce the number of requests that need to be made.
Cache regularly used IDs to avoid unnecessary API requests. Certain reference information, such as targeting options IDs and Google Audience IDs, are relatively stable and can be safely stored to avoid the need for retrieval upon every use. However, cached values should be checked on a weekly basis to account for infrequent changes or deprecations.

See our Quota Optimization guide for more information on how to access the Display & Video 360 API efficiently.

Import into BigQuery

The Display & Video 360 API BigQuery Connector lets you automatically import Display & Video 360 resource configurations directly into BigQuery on a daily basis. The configurations are stored in BigQuery using Display & Video 360 API resource design. A subset of API resources are supported.

See the following cloud documentation for more information about using the Display & Video 360 API BigQuery Connector:

Known API data gaps

There are notable data gaps you might encounter when migrating from ERF to Display & Video 360 API, such as:

Story insertion orders. Story insertion orders are not retrievable through the API and must be retrieved through the Display & Video 360 UI.
A subset of resource fields. A small number of resource fields present in ERF objects are not available in the corresponding resources retrieved through the Display & Video 360 API.

Appendix: Mapping ERF Fields to API

Public table mapping

The tables below map the fields of ERF public tables to existing targeting types and targeting option fields in the Display & Video 360 API. Although the value of one field might map to another, that does not guarantee that they utilize the same data type, enum values, or ID space.

App Collection

Retrievable under targeting type TARGETING_TYPE_APP_CATEGORY.

ERF Field Name	DV360 API Availability
id	`TargetingOption.targetingOptionId` field.
name	`TargetingOption.appCategoryDetails.displayName` field.

Browser

Retrievable under targeting type TARGETING_TYPE_BROWSER.

ERF Field Name	DV360 API Availability
id	`TargetingOption.targetingOptionId` field.
is_mobile	Not available.
name	`TargetingOption.browserDetails.displayName` field.

DataPartner

There is no equivalent resource or fields available in Display & Video 360 API.

DeviceCriteria

Retrievable under targeting types TARGETING_TYPE_OPERATING_SYSTEM, TARGETING_TYPE_DEVICE_MAKE_MODEL, and TARGETING_TYPE_DEVICE_TYPE.

ERF Field Name	DV360 API Availability
id	`TargetingOption.targetingOptionId` field or `DeviceType` enum.
is_mobile	Not available.
name	`TargetingOption.operatingSystemDetails.displayName` field, `TargetingOption.deviceMakeModelDetails.displayName` field, or `DeviceType` enum, depending on targeting type.
criteria_type	`TargetingOption.targetingType` field.
operating_system_id	Not available.
mobile_brand_name	Not available.
mobile_model_name	Not available.
mobile_make_model_id	Not available.
device_type	`DeviceType` enum.

GeoLocation

Retrievable under targeting type TARGETING_TYPE_GEO_REGION.

ERF Field Name	DV360 API Availability
id	`TargetingOption.targetingOptionId` field.
canonical_name	`TargetingOption.geoRegionDetails.displayName` field.
geo_name	Not available.
country_code	Not available.
region_code	Not available.
city_name	Not available.
postal_name	Not available.
dma_code	Not available.

Isp

Retrievable under targeting type TARGETING_TYPE_CARRIER_AND_ISP.

ERF Field Name	DV360 API Availability
id	`TargetingOption.targetingOptionId` field.
is_mobile	Not available.
name	`TargetingOption.carrierAndIspDetails.displayName` field.
secondary_criteria_id	`TargetingOption.targetingOptionId` field.

Language

Retrievable under targeting type TARGETING_TYPE_LANGUAGE.

ERF Field Name	DV360 API Availability
id	`TargetingOption.targetingOptionId` field.
name	Not available. Full display name for a language is available at `TargetingOption.languageDetails.displayName` field.

SiteToPlacementId

There is no equivalent resource or fields available in Display & Video 360 API.

SupportedExchange

Retrievable under targeting type TARGETING_TYPE_EXCHANGE.

ERF Field Name	DV360 API Availability
id	`Exchange` enum.
name	`Exchange` enum.

UniversalSite

There is no equivalent resource or fields available in Display & Video 360 API. Individual sites and apps can be targeted directly under targeting types TARGETING_TYPE_URL and TARGETING_TYPE_APP, respectively. In Display & Video 360, any app or URL can be targeted, but not every app or URL can be reported on. If you would like to remove non-reportable apps and URLs from spending, follow the instructions on the DV360 Help Center.

Private table field mapping

The tables below map the fields of ERF private tables to existing fields or services in the Display & Video 360 API. Although the value of one field may map to another, that does not guarantee that they utilize the same data type, enum values or ID space.

Advertiser

ERF Field Name	DV360 API Availability
common_data.id	`Advertiser.advertiserId` field.
common_data.name	`Advertiser.displayName` field.
common_data.active	`Advertiser.entityStatus` field.
common_data.integration_code	`Advertiser.integrationDetails.integrationCode` field.
partner_id	`Advertiser.partnerId` field.
currency_code	`Advertiser.generalConfig.currencyCode` field.
timezone_code	`Advertiser.generalConfig.timeZone` field.
landing_page_url	`Advertiser.generalConfig.domainUrl` field.
available_channel_ids	Retrievable through `advertisers.channels.list` method.
blacklist_channel_id	Retrievable through `advertisers.targetingTypes.assignedtargetingOptions.list` method under targeting type `TARGETING_TYPE_CHANNEL` . If `AssignedTargetingOption.channelDetails.negative` is true, the channel is negatively-targeted.
dcm_configuration	Not available.
dcm_network_id	`Advertiser.adServerConfig.cmHybridConfig.cmAccountId` field.
dcm_advertiser_id	The `Advertiser.adServerConfig.cmHybridConfig.cmAdvertiserIds` field lists the CM360 Advertiser IDs that share the CM360 Floodlight configuration.
dcm_floodlight_group_id	`Advertiser.adServerConfig.cmHybridConfig.cmFloodlightConfigId` field.
dcm_syncable_site_ids	`Advertiser.adServerConfig.cmHybridConfig.cmSyncableSiteIds` field.
enable_oba_tags	Not available.

Campaign

ERF Field Name	DV360 API Availability
common_data.id	`Campaign.campaignId` field.
common_data.name	`Campaign.displayName` field.
common_data.active	`Campaign.entityStatus` field.
common_data.integration_code	Not available.
advertiser_id	`Campaign.advertiserId` field.
budget	`Campaign.campaignFlight` and `Campaign.campaignBudgets` fields.
frequency_cap	`Campaign.frequencyCap` field.
default_target_list	Retrievable through `advertisers.campaigns.bulkListCampaignAssignedTargetingOptions` method.
uses_video_creatives	Not available.
uses_display_creatives	Not available.
uses_audio_creatives	Not available.
objective	`Campaign.campaignGoal.campaignGoalType` field.
metric	`Campaign.campaignGoal.performanceGoal.performanceGoalType` field.
objective_description	`Campaign.campaignGoal.performanceGoal.performanceGoalString` field.
metric_amount_micros	`Campaign.campaignGoal.performanceGoal.performanceGoalAmountMicros` field.

Creative

ERF Field Name	DV360 API Availability
common_data.id	`Creative.creativeId` field.
common_data.name	`Creative.displayName` field.
common_data.active	`Creative.entityStatus` field.
common_data.integration_code	`Creative.integrationCode` field.
advertiser_id	`Creative.advertiserId` field.
dcm_placement_id	`Creative.cmPlacementId` field.
width_pixels	`Creative.dimensions.widthPixels` field.
height_pixels	`Creative.dimensions.heightPixels` field.
approval_status	`Creative.reviewStatus` field.
expanding_direction	`Creative.expandingDirection` field.
creative_type	`Creative.creativeType` field.

CustomAffinity

ERF Field Name	DV360 API Availability
id	`CustomList.customListId` field.
name	`CustomList.displayName` field.
description	Not available.
advertiser_id	Not available.

FloodlightActivity

ERF Field Name	DV360 API Availability
common_data.id	`FloodlightActivity.floodlightActivityId` field.
common_data.name	`FloodlightActivity.displayName` field.
common_data.active	`FloodlightActivity.servingStatus` field.
common_data.integration_code	Not available.
advertiser_id	The `FloodlightActivity.advertiserIds` field lists all advertisers with access to the floodlight activity under the given partner.
partner_id	Provided by the user when making a request to the `floodlightGroups.floodlightActivities` service.
remarketing_enabled	The `FloodlightActivity.remarketingConfigs` field lists this configuration for each advertiser with access to the floodlight activity under the given partner.
ssl_required	`FloodlightActivity.sslRequired` field.

InsertionOrder

ERF Field Name	DV360 API Availability
common_data.id	`InsertionOrder.insertionOrderId` field.
common_data.name	`InsertionOrder.displayName` field.
common_data.active	`InsertionOrder.entityStatus` field.
common_data.integration_code	`InsertionOrder.integrationDetails.integrationCode` field.
advertiser_id	`InsertionOrder.advertiserId` field.
campaign_id	`InsertionOrder.campaignId` field.
overall_budget	Not available. Can be calculated using contents of the `InsertionOrder.budget.budgetSegments` field.
scheduled_segments	`InsertionOrder.budget.budgetSegments` field.
frequency_cap	`InsertionOrder.frequencyCap` field.
default_partner_costs	`InsertionOrder.partnerCosts` field.
default_target_list	Retrievable through `advertisers.insertionOrders.bulkListInsertionOrderAssignedTargetingOptions` method.

InventorySource

ERF Field Name	DV360 API Availability
id	`InventorySource.inventorySourceId` field.
unclassified	Not available.
inventory_name	`InventorySource.displayName` field.
exchange_id	`InventorySource.exchange` field.
accessing_advertisers	`InventorySource.readWriteAccessors` and `InventorySource.readAdvertiserIds` fields.
external_id	`InventorySource.dealId` field.
min_cpm_micros	`InventorySource.rateDetails.rate.nanos` field, depending on the value of the `InventorySource.rateDetails.inventorySourceRateType` field.
min_cpm_currency_code	`InventorySource.rateDetails.rate.currencyCode` field.

LineItem

ERF Field Name	DV360 API Availability
common_data.id	`LineItem.lineItemId` field.
common_data.name	`LineItem.displayName` field.
common_data.active	`LineItem.entityStatus` field.
common_data.integration_code	`LineItem.integrationDetails.integrationCode` field.
line_item_type	`LineItem.lineItemType` field.
insertion_order_id	`LineItem.insertionOrderId` field.
creative_ids	`LineItem.creativeIds` field.
max_cpm_advertiser_micros	`LineItem.bidStrategy.maximizeSpendAutoBid.maxAverageCpmBidAmountMicros` or `LineItem.bidStrategy.performanceGoalAutoBid.maxAverageCpmBidAmountMicros` fields, depending on the strategy scheme used.
performance_goal	`LineItem.bidStrategy.maximizeSpendAutoBid.performanceGoalType` or `LineItem.bidStrategy.performanceGoalAutoBid.performanceGoalType` fields, depending on the strategy scheme used.
goal_advertiser_micros	`LineItem.bidStrategy.performanceGoalAutoBid.performanceGoalAmountMicros` field.
partner_revenue_model	`LineItem.partnerRevenueModel` field.
cost_tracking_pixels	`LineItem.conversionCounting.floodlightActivityConfigs` field.
budget.start_time_usec	`LineItem.flight.dateRange.startDate` field.
budget.end_time_usec	`LineItem.flight.dateRange.endDate` field.
budget.max_impressions	`LineItem.budget.maxAmount` field if `LineItem.budget.budgetUnit` is `BUDGET_UNIT_IMPRESSIONS` .
budget.max_spend_advertiser_micros	`LineItem.budget.maxAmount` field if `LineItem.budget.budgetUnit` is `BUDGET_UNIT_CURRENCY` .
budget.pacing_type	`LineItem.pacing.pacingPeriod` field.
budget.pacing_max_impressions	`LineItem.pacing.dailyMaxImpressions` field.
budget.pacing_max_spend_advertiser_micros	`LineItem.pacing.dailyMaxMicros` field.
budget.pacing_distribution	`LineItem.pacing.pacingType` field.
frequency_cap	`LineItem.frequencyCap` field.
partner_costs	`LineItem.partnerCosts` field.
target_list	Retrievable through `advertisers.lineItems.bulkListLineItemAssignedTargetingOptions` method.

NegativeKeywordList

ERF Field Name	DV360 API Availability
id	`NegativeKeywordList.negativeKeywordListId` field.
name	`NegativeKeywordList.displayName` field.
advertiser_id	`NegativeKeywordList.advertiserId` field.

Partner

ERF Field Name	DV360 API Availability
common_data.id	`Partner.partnerId` field.
common_data.name	`Partner.displayName` field.
common_data.active	`Partner.entityStatus` field.
common_data.integration_code	Not available.
currency_code	`Partner.generalConfig.currencyCode` field.
exchange_settings	`Partner.exchangeConfig.enabledExchanges` field.
default_partner_costs	Not available.
default_partner_revenue	Not available.
default_target_list	Not available.

Pixel

There is no equivalent resource or fields available in Display & Video 360 API.

UniversalChannel

ERF Field Name	DV360 API Availability
id	`Channel.channelId` field.
name	`Channel.displayName` field.
site_ids	Retrievable through `advertisers.channels.sites.list` and `partners.channels.sites.list` methods, depending on the type of `owner` .
accessing_advertisers	Not available.
is_deleted	Not available.
is_brand_safe_channel	Not available.

UserList

ERF Field Name	DV360 API Availability
id	`FirstAndThirdPartyAudience.firstAndThirdPartyAudienceId` field.
name	`FirstAndThirdPartyAudience.displayName` field.
data_partner_id	Not available.
accessing_advertisers	Not available.
partner_pricing	Not available.
advertiser_pricings	Not available.