Showing posts with label ReportDefinitionService. Show all posts
Showing posts with label ReportDefinitionService. Show all posts

Discover v201109 - AdHoc Reports

Tuesday, November 15, 2011


AdHoc reports is one of the new features we introduced in v201109 of the AdWords API. AdHoc reports are free, synchronous and don’t require a stored report definition. This blog post explains this new feature and covers the best practices.

Downloading the report

AdHoc reports utilizes plain HTTP POST requests to generate reports synchronously from the AdWords API server. A raw HTTP request to download the reports is shown below:

POST /api/adwords/reportdownload/v201109 HTTP/1.1
HOST: adwords.google.com
clientCustomerId: XXX-XXX-XXXX
Content-Type: application/x-www-form-urlencoded
developerToken: XXXX
Authorization: GoogleLogin auth=XXX
returnMoneyInMicros: true
Content-Length: XXXX


__rdxml=URL_ENCODED_DEFINITION_XML

As shown, the report is downloaded by making a POST request to https://adwords.google.com/api/adwords/reportdownload/v201109. The clientCustomerId header specifies the client for which this report is being run. The Authorization header contains the authorization information for downloading the report. If you are using ClientLogin as the authorization method, this header takes the form
Authorization: GoogleLogin auth=AUTH_TOKEN_HERE
If you are using OAuth, the header takes the form
Authorization: OAuth OAUTH_SIGNATURE_HERE
The returnMoneyInMicros header tells the server whether to return money values in micros or not. This is true by default; if set to false, money values are downloaded in actual currency format. The developerToken header contains your AdWords API developer token. The Report Definition is written in XML format and is provided in the __rdxml parameter of the POST body. The request should be encoded in application/x-www-form-urlencoded or multipart/form-data format.

You may also use the GET method instead of POST to download reports by sending your request to https://adwords.google.com/api/adwords/reportdownload/v201109?__rdxml=URL_ENCODED_REPORT_DEFINITION_XML with the same headers as in the POST example. However, we recommend using POST over GET since GET requests are more likely to incur URL length limit errors.

Creating definition xml

A Report Definition corresponds to the serialized form of the ReportDefinition type. For instance, a Report Definition that downloads an AdGroup performance report for the last 7 days in CSV format would look like this:

  <reportDefinition>
    <selector>
      <fields>CampaignId</fields>
      <fields>Id</fields>
      <fields>Impressions</fields>
      <fields>Clicks</fields>
      <fields>Cost</fields>
      <predicates>
        <field>Status</field>
        <operator>IN</operator>
        <values>ENABLED</values>
        <values>PAUSED</values>
      </predicates>
    </selector>
    <reportName>Custom Adgroup Performance Report</reportName>
    <reportType>ADGROUP_PERFORMANCE_REPORT</reportType>
    <dateRangeType>LAST_7_DAYS</dateRangeType>
    <downloadFormat>CSV</downloadFormat>
  </reportDefinition>

Details about the supported report types and their field are available at http://code.google.com/apis/adwords/docs/appendix/reports.html.

Error codes

When downloading AdHoc reports, the AdWords API server responds with HTTP code 200 and report in the response body. In case of an error, one of the two different HTTP response code may be returned:

1. HTTP Status code is 400: This error occurs when the AdWords API server throws an API error. When this happens, the error message will be returned in the server response body. You need to examine your report definition XML and server response body and fix any errors before retrying the call. The response body would look like

!!!2|||-1|||Error message here???

An easy way to parse the error message is to match it against the regex pattern "\!\!\!([^\|]*)\|\|\|(.*)"

2. HTTP Status code is 500: This error implies that the server faced an issue while generating the report. Chances are that this is a temporary error and may go away if you retry the call after waiting for 30 seconds. However, if you get this error even after five retries, you should skip the download and report the issue on the forum.

Using the client libraries

All of the client libraries support AdHoc reports, and takes care of details like making requests with the right HTTP headers and encoding, constructing authorization headers, handling the HTTP response codes, etc. For instance, the DownloadAdHocReports.cs code example shows how to download AdHoc reports using AdWords API .NET library:

Support for cross-client reports

As mentioned in an earlier blog post, AdHoc reports do not support cross-client reports. If you wish to run reports against many clients, you can refer to this blog post for details.

We hope this blog post will help you use this new feature effectively. As usual, If you have any questions about downloading reports, you can ask us on the forum or our upcoming live Hangouts with the Developer Relations team.

  --Anash P. Oommen, AdWords API Team

Use reports to download large data sets

Monday, April 18, 2011


In this post, we will share best practices for downloading large data sets. Specifically, we will outline how to accomplish this using reports and why this is more efficient than paging with selectors.

When you request a large amount of data using generic selectors (for example, keywords), you may find that the responses are slow for very large data sets. First, we advise you not to page past 10,000 records due to the increased amount of time it takes to skip the first 10,000 records. Second, as a rule of thumb, once you exceed 100,000 keywords, it is a good idea to use a report to obtain your data rather than generic selectors.

Reports are designed to handle large quantities of data. In addition, you will also use fewer API units if you request data via reports than if you request the same data via the AdGroupCriterionService. For example, say you would like to obtain all the keywords across all ad groups using paging of 1000 for 100,000 total keywords. This would require 100,000 API units, one per AdGroupCriterion. Defining a report and downloading the data that way will cost significantly less units in the long run. If you define the report with a sliding date range such as TODAY or LAST_WEEK, you can reuse the report each time you need to download your data set. Please see this page for more information about reports.

If you need fields that are not available in reports, you still have some options to increase performance. With generic selectors you can specify only the fields you need; the fewer the number of requested fields, the faster the results will be returned. You can also use predicates to filter and limit the amount of data in each result set. By requesting keywords on a per-campaign or per-ad group basis instead of all at once, you may actually get your results faster.

If you have any questions about using reports with the API, you can ask us on the forum.

- Kevin Winter, AdWords API Team

AdWords API v201101 launch -- providing enhanced reporting, generic selectors, support for filtering and more

Friday, March 04, 2011


We’re pleased to announce the launch of AdWords API v201101, which allow you to more efficiently run reports, as well as implement campaign experiments and other recently released advertising features at scale. Through the AdWords API forum and developer events we’ve heard frequent requests for the newly available services. We’ve highlighted some of the new features below. A complete list of changes is available in the v201101 release notes.

v201101 highlights:
  • Easily run reports across clients: ReportDefinitionService now supports My Client Center (MCC) cross-client reports, which are fetched asynchronously. The ReportDefinition type has two new fields with which you specify retrieval of a cross-client report, and all report types have new fields for use in cross-client reporting. In addition, the following new reports have been added: Geo Performance, Demographic Performance, Ad Extensions Performance,  Destination URL and Creative Conversion. See Report Types for more information.
  • Greater data control and filtering with a single generic selector: The get operations for a number of services now use a single generic selector, rather than service-specific selectors. The generic selector provides more control over the data returned, lets you filter on almost any field, and specify sorting and paging. For information about migrating your code, see the Selector Migration Reference.
  • Try out campaign experiments: A campaign can now try out sets of ads experimentally by using the ExperimentData in the AdGroupAd type. A new set of report fields provide information about how an advertiser experiment performed. The Ad Group Performance, Campaign Performance, Keywords Performance, and Managed Placements Performance reports each have a field that lets you segment on the control arm or experiment arm. In addition, a report field for each statistic indicates the significance in the change of that statistic in the experiment arm. For more information about campaign experiments, see the AdWords Campaign Experiments overview.
  • Differentiate locations for desired targeting: A new setting for the Campaign type --  GeoTargetTypeSetting -- lets you specify how to apply geo targeting. The setting lets you differentiate between the user’s physical location and the location that’s targeted in the search. 
  • Run interest-based advertising at scale (coming soon): Adwords API will fully support interest-based advertising via the ConversionTrackerService, which enables you to create a new conversion event and get the Javascript tag to create new lists. This service will be released in the coming weeks.
Deprecation timeline for previous versions
With the release of equivalent functionality in v201101, the following versions and services will be deprecated:
  • API versions v13 (ReportService and TrafficEstimatorService only), v200909, v201003, v201008
  • BidLandscapeService (being moved to the DataService)
We will be sunsetting these versions and services in August 2011.

As with every new version of the AdWords API, we encourage you to review the resources in the AdWords API client libraries. If you have any questions please post them on the AdWords API forum.

Posted by Katie Wasilenko, AdWords API Team

Harness the power of predicates in your reports

Friday, December 10, 2010


The reporting services of the AdWords API are designed to allow you to download large amounts of data, but it can often be useful to filter out certain rows from the start to cut down on processing and transfer time. DefinedReportJobs created using the v13 ReportService supported a limited set of filtering options, primarily on IDs and statuses. Filtering on a more complex set of criteria would require post processing the report client-side. The ReportDefinitionService introduced in v201003 supports Predicates, a more flexible system of filters that can be used to create very targeted reports.

A predicate is composed of three parts: the field, the operator, and the values. Predicates can be created for almost every field available, indicated by canFilter set to "true" in the results returned from getReportFields().

<rval>
<fieldName>Clicks</fieldName>
<displayFieldName>Clicks</displayFieldName>
<xmlAttributeName>clicks</xmlAttributeName>
<fieldType>Long</fieldType>
<canSelect>true</canSelect>
<canFilter>true</canFilter>
</rval>


There are a fixed set of operators that can be used to compare the value of the field selected. EQUALS can be used to filter for an exact match, while IN can be used to match any of a set of values. GREATER_THAN and LESS_THAN are available for comparing numerical values, and CONTAINS is useful when working with strings. Only one operator can be used per predicate, but multiple predicates can be created for the same field to create more complex logic.

The values used in the predicate depend heavily on the field being operated on. Numerical values should be used for Long, Double, and Integer fields, and arbitrary strings can be used for String fields. The values for Money and Bid fields must always be specified in micros, even if the report is eventually downloaded with whole currency amounts. Although percentage fields like Ctr are returned as whole numbers, the values used in predicates should be the decimal equivalents. Predicates on Enum fields must only use the values of the enum, as outlined in the documentation and in the enumValues property returned by getReportFields().

<rval>
<fieldName>CampaignStatus</fieldName>
<displayFieldName>Campaign state</displayFieldName>
<xmlAttributeName>campaignState</xmlAttributeName>
<fieldType>CampaignStatus</fieldType>
<enumValues>ACTIVE</enumValues>
<enumValues>DELETED</enumValues>
<enumValues>PAUSED</enumValues>
<canSelect>true</canSelect>
<canFilter>true</canFilter>
</rval>


If multiple predicates are used within a Selector they will be combined using AND logic, so that the resulting report will only contain rows that satisfies all of the predicates. It’s not possible to combine predicates using OR logic at this time, but some common use cases can be handled using a single predicate and the IN operator. For example, to match keywords with either the PHRASE or EXACT match type you could use the following predicate:

<predicates>
<field>KeywordMatchType</field>
<operator>IN</operator>
<values>PHRASE</values>
<values>EXACT</values>
</predicates>


When combined together, predicates can be quite powerful. Consider the following example: I own an electronics shop that is starting to sell a wider array of tablet computers, and I want to drive more traffic to that section of my website. I’d like to start by finding all my current keywords that include the word "tablet" and have a high CTR, and then use them to generate new keyword ideas using the TargetingIdeaService. I’m only interested in keywords with greater than a 10% CTR, that have at least 100 impressions, and that are in enabled or paused ad groups.

Using predicates I can create a KEYWORDS_PERFORMANCE_REPORT that only returns me the exact data I am interested in:

<predicates>
<field>CampaignId</field>
<operator>EQUALS</operator>
<values>123456789</values>
</predicates>
<predicates>
<field>AdGroupStatus</field>
<operator>IN</operator>
<values>ENABLED</values>
<values>PAUSED</values>
</predicates>
<predicates>
<field>KeywordText</field>
<operator>CONTAINS</operator>
<values>tablet</values>
</predicates>
<predicates>
<field>Ctr</field>
<operator>GREATER_THAN</operator>
<values>0.10</values>
</predicates>
<predicates>
<field>Impressions</field>
<operator>GREATER_THAN_EQUALS</operator>
<values>100</values>
</predicates>


Predicates are one of many new improvements introduced in the ReportDefinitionService, and if you aren’t using the service yet, now is a great time to start looking into migrating. The service is fully supported in all our client libraries, and we’re available on the forum to answer any questions you may have.

- Eric Koleda, AdWords API Team

A change to currency formatting in report downloads

Tuesday, November 02, 2010


If you're using the new AdWords API ReportDefinitionService, you may have noticed that monetary values in reports are returned as conventional currency instead of micros. At the request of the developer community, we'll be changing the format to micros on February 1, 2011 to make reporting more consistent with other AdWords API services.

A micro value is equal to one million times the conventional currency value, and is a standard we use throughout the AdWords API (including v13 reports) to represent money. For example, values currently returned as "1.50" (which would represent $1.50 for a USD account) will be returned as 1500000.

To help you with this change, we've introduced a new HTTP header flag (available immediately) that allows you to explicitly request the micros format. If you've written code that expects conventional currency values in reports, it's important that you update your code to expect micros and set the HTTP header "returnMoneyInMicros: true" when requesting a report download. Additionally, if you're just beginning to migrate from v13 reports, you should set this header on all your download requests.

To learn more about the new way to run reports via the API, see our blog post from this summer. As always, please post questions to the AdWords API Forum.

- Eric Koleda, AdWords API Team

Discover v201003: Migrating your Account Structure Reports

Tuesday, July 20, 2010


Account structure report is one of the popular report formats in v13 of the AdWords API. This report allows you to download all the account attributes in a single report without performance details. A common use of account structure reports is to sync a local database against the AdWords server without using more expensive get() API calls.

Despite being a useful report format, account structure report is a specialized report format available only through the AdWords API. We designed the new API such that standard report formats can support your needs rather than creating additional specialized report types. Therefore, the new AdWords API doesn’t have a separate account structure report type, but you can use the existing report types supported by ReportDefinitionService to download the same data. This blog post discusses how this can be done, so that you can migrate your existing account structure reports to the v201003 version of the AdWords API. This blog post assumes you are familiar with the usage of ReportDefinitionService, if not, we suggest that you read our introductory blog post first.

Retrieving campaign structure information

You can download all campaigns in an account in the v201003 version of the AdWords API using CAMPAIGN_PERFORMANCE_REPORT. The xml request for adding the report definition is shown below:

<mutate xmlns="https://adwords.google.com/api/adwords/cm/v201003">
  <operations>
    <operator>ADD</operator>
    <operand>
      <selector>
        <fields>Id</fields>
        <fields>Name</fields>
        <fields>Status</fields>
        <fields>Amount</fields>
        </selector>
      <reportName>Campaign performance report #1277301277814.61</reportName>
      <reportType>CAMPAIGN_PERFORMANCE_REPORT</reportType>
      <dateRangeType>TODAY</dateRangeType>
      <downloadFormat>XML</downloadFormat>
    </operand>
  </operations>
</mutate>
Retrieving the ad group structure information

You can download all adgroups in an account in the v201003 version of the AdWords API using ADGROUP_PERFORMANCE_REPORT. The xml request for adding the report definition is shown below:
<mutate xmlns="https://adwords.google.com/api/adwords/cm/v201003">
  <operations>
    <operator>ADD</operator>
    <operand>
      <selector>
        <fields>Id</fields>
        <fields>Name</fields>
        <fields>Status</fields>
        <fields>CampaignId</fields>
      </selector>
      <reportName>AdGroup performance report #1277302184111.48</reportName>
      <reportType>ADGROUP_PERFORMANCE_REPORT</reportType>
      <dateRangeType>TODAY</dateRangeType>
      <downloadFormat>XML</downloadFormat>
    </operand>
  </operations>
</mutate>

If required, you can restrict the report to contain ad groups from only a particular campaign by setting a Predicate. A code example showing the usage of Predicate can be found here.

Retrieving the ad structure information


You can download all ads in an account in the v201003 version of the AdWords API using AD_PERFORMANCE_REPORT. The xml request for adding the report definition is shown below:
<mutate xmlns="https://adwords.google.com/api/adwords/cm/v201003">
  <operations>
    <operator>ADD</operator>
    <operand>
      <selector>
        <fields>Id</fields>
        <fields>Headline</fields>
        <fields>Description1</fields>
        <fields>Description2</fields>
        <fields>DisplayUrl</fields>
        <fields>Url</fields>
        <fields>Status</fields>
        <fields>AdGroupId</fields>
      </selector>
      <reportName>Ad performance report #1277303749892.73</reportName>
      <reportType>AD_PERFORMANCE_REPORT</reportType>
      <dateRangeType>TODAY</dateRangeType>
      <downloadFormat>XML</downloadFormat>
    </operand>
  </operations>
</mutate>
If required, you can restrict the report to contain ads from only a particular ad group by setting a Predicate.

Retrieving keyword and placement structure information


You can download all keywords in an account in the v201003 version of the AdWords API using KEYWORD_PERFORMANCE_REPORT. Similarly, all placements can be downloaded using MANAGED_PLACEMENTS_PERFORMANCE_REPORT. The xml requests for adding the keyword performance report definition are shown below:
<mutate xmlns="https://adwords.google.com/api/adwords/cm/v201003">
  <operations>
    <operator>ADD</operator>
    <operand>
      <selector>
        <fields>AdGroupId</fields>
        <fields>Id</fields>
        <fields>KeywordText</fields>
        <fields>KeywordMatchType</fields>
        <fields>IsNegative</fields>
      </selector>
      <reportName>Keyword performance report #1277353190509.28</reportName>
      <reportType>KEYWORDS_PERFORMANCE_REPORT</reportType>
      <dateRangeType>TODAY</dateRangeType>
      <downloadFormat>XML</downloadFormat>
    </operand>
  </operations>
</mutate>

The xml for placement report is identical to the one shown above, except for the reportType field. Note that have to use the KeywordText field to get the placement url while running MANAGED_PLACEMENTS_PERFORMANCE_REPORT.

Putting it all together

Here’s a code example that puts everything together to download your account structure using reports. The code snippet for parsing xmls and copying streams has been excluded for brevity.

Click here to expand

We've included support for ReportDefinitionService in all of our client libraries to help get you started, so please try it out and share your feedback with us on the forum or the projects' issue trackers.

-- Anash P. Oommen, AdWords API Team

Introduction to ReportDefinitionService

Tuesday, July 13, 2010


Reporting is an integral part of most AdWords API applications. To help you create and manage reports related to your AdWords campaigns, we introduced the ReportDefinitionService in the v201003 version of the AdWords API. In this post, we’ll cover the basics of working with the ReportDefinitionService. If you’ve used the v13 ReportService, you’ll notice that the new ReportDefinitionService differs in many ways from its v13 counterpart.

Retrieving the report fields

To create a report in the v201003 version of AdWords API, you pick a report type of your choice and then retrieve the list of supported fields using getReportFields. The Java code below shows how to retrieve the report fields:

// Get report fields.
ReportDefinitionField[] reportDefinitionFields = 
    reportDefinitionService.getReportFields(
        ReportDefinitionReportType.KEYWORDS_PERFORMANCE_REPORT);
// Display report fields.
System.out.println("Available fields for report:");
for (ReportDefinitionField reportDefinitionField : reportDefinitionFields) {
  System.out.print("\t" + reportDefinitionField.getFieldName() + "("
      + reportDefinitionField.getFieldType() + ")");
}

This feature is quite convenient if you want to display the list of supported fields for a user and allow the users to pick the desired fields. For those who prefer static documentation like in v13, we are working on it, and it will be made available in the near future.

Defining the report

To create a report in the v201003 version of AdWords API, you have to create a report definition first. This is different from the v13 version of API, where you could schedule a report without creating any definitions first. The following Java code snippet creates a report definition:

// Create ad group predicate.
Predicate adGroupPredicate = new Predicate();
adGroupPredicate.setField("AdGroupId");
adGroupPredicate.setOperator(PredicateOperator.EQUALS);
adGroupPredicate.setValues(new String[] {adGroupId});
 
// Create selector.
Selector selector = new Selector();
selector.setFields(new String[] {"AdGroupId", "Id", "KeywordText",
    "KeywordMatchType", "Impressions", "Clicks", "Cost"});
selector.setPredicates(new Predicate[] {adGroupPredicate});
selector.setDateRange(new DateRange(startDate, endDate));
 
// Create report definition.
ReportDefinition reportDefinition = new ReportDefinition();
reportDefinition.setReportName("Keywords performance report”);
reportDefinition.setDateRangeType(ReportDefinitionDateRangeType.CUSTOM_DATE);
reportDefinition.setReportType(ReportDefinitionReportType.KEYWORDS_PERFORMANCE_REPORT);
reportDefinition.setDownloadFormat(DownloadFormat.XML);
reportDefinition.setSelector(selector);
 
// Create operations.
ReportDefinitionOperation operation = new ReportDefinitionOperation();
operation.setOperand(reportDefinition);
operation.setOperator(Operator.ADD);
ReportDefinitionOperation[] operations = new ReportDefinitionOperation[]
    {operation};
 
// Add report definition.
ReportDefinition[] result = reportDefinitionService.mutate(operations);

When successful, the API call creates a report definition under the report section of your account’s Control Panel and Library. You may delete or modify a report definition using the mutate method, with operators as REMOVE and SET respectively. You can also retrieve all report definitions in your account using the get method.

If you’d like to inexpensively validate the report definition before adding it, you can call the mutate method as shown above with the validateOnly header set. This works similar to the validateReportJob method in v13 version of AdWords API. You can refer to our earlier blog post for details on how to use validateOnly headers to validate API calls.

The v201003 version of ReportDefinitionService introduces a new feature called predicates. You can use predicates to filter the results by any field with canFilter=true.. You can also use various operators to define the filtering condition. This is an improvement over v13 where the data could only be filtered using certain predefined fields like campaignId or adGroupId.

The v201003 version of ReportDefinitionService allows you to generate reports for predefined date ranges. This is very useful if you need to download a predefined type of report on a regular basis. For instance, you can create a single report definition with dateRangeType as YESTERDAY, and then use that report definition to download a daily report. This is an improvement over v13 where you needed to schedule a new report every day to accomplish the same task. If you need to download reports for a custom period, you can set dateRangeType as CUSTOM_DATE.

The v201003 ReportDefinitionService currently has some limitations: it lacks support for aggregation types and cross client reports. We are working to add support for these features in a future version of the AdWords API.

Generating and downloading a report

To download a report in the v201003 version of AdWords API, you need to issue a HTTP GET to https://adwords.google.com/api/adwords/reportdownload?__rd=reportDefinitionId. You can get the report definition id from the result returned by the server when you add a report definition as shown in the code sample above. In addition, you need to specify authToken and clientLogin (or clientCustomerId) as http headers to authorize the report download. Note that the clientLogin or clientCustomerId must be the same as the one used while creating the report definition. The sample Java code below downloads a report:

String url = "https://adwords.google.com/api/adwords/reportdownload?__rd="
    + reportDefinitionId;
 HttpURLConnection urlConn = 
    (HttpURLConnection) new URL(url).openConnection();
urlConn.setRequestMethod ("GET");
urlConn.setRequestProperty("Authorization", "GoogleLogin auth="
    + user.getRegisteredAuthToken());
if (user.getClientCustomerId() != null) {
  urlConn.setRequestProperty("clientCustomerId", user.getClientCustomerId());
} else if (user.getClientEmail() != null) {
  urlConn.setRequestProperty("clientEmail", user.getClientEmail());
} else {
  urlConn.setRequestProperty("clientEmail", user.getEmail());
}
urlConn.connect();
copyStream(urlConn.getInputStream(), new FileOutputStream(
    new File(outputFileName)));

The raw HTTP message when executing this code is as shown below:

GET /api/adwords/reportdownload?__rd=XXXXXX HTTP/1.1
Host: adwords.google.com
Accept: */*
Authorization: GoogleLogin auth=XXXXXX
clientEmail: XXXXXX

A major difference between this approach and the v13 approach is that the report generation is inherently synchronous, unlike in v13 where you had to poll regularly using getReportJobStatus to see if a report job completed, and later use getReportDownloadUrl to get the report download url. Also, the report generation in v201003 is much faster than in v13, and most reports complete in a few seconds. The request to download reports will time out after 3 minutes, so we recommend that you use gzipped csv format to minimize transfer size of potentially large reports. GZipped XML format is not yet supported, but we’re working to include this feature in a future version of the AdWords API.

We've included support for ReportDefinitionService in all of our client libraries to help get you started, so please try it out and share your feedback with us on the forum or the projects' issue trackers.

-- Anash P. Oommen, AdWords API Team