Technical information

 

Universes

Data Flow makes several universes (families of analysis) available:

  • Visit
  • Events
  • Publisher (optional)
  • Self-promotion (optional)
  • Rich Media (optional)
  • MV Testing (optional)
  • Orders (optional)
  • Products (optional)
 

Visit

This universe provides all information related to the visit, which never changes during the visit (e.g. traffic source, visit ID, unique visitor ID, geolocation or technical information like the OS or the application version).

In this flow, one row will be considered as one visit. It must contain the Visit metric column in order to work; the output can be 1 or 0 (for a visit with no pages).

 

Sub-universes

Data from Traffic Sources can be very specific if you are using custom variables from Marketing Sources or Custom Organic Sources. While the general Visit flow will give you the source and campaign name, should you need additional information about the sponsored link platform, advertisement creatives, email marketing links clicked etc., you will have to create a sub-universe flow per traffic source.

 

Events

This universe provides information encountered during two types of events: page loads and clicks. One row will represent one of these events. This flow must contain the Clicks and Loads metrics in order to work, even if the result will awlays be 1 or 0 depending on the type of event.

Information such as page labels, chapters, level 2, internal search engine data, clicks and site custom variables are part of the Event flow.

The Visit and Events flows are mandatory in order to extract your raw data. Then, according to the features tagged on your site, you may enrich your data extraction with other flows.

Note: For each universe, you will see just a few metrics available (generally loads and clicks.) The Data Flow API doesn’t provide calculated metrics that rely on an aggregation of several events (e.g. bounce rate, time spent on a page, etc.), nor can you apply Segments or Custom Metrics. However, thanks to the very granular level of detailed information you will get, it is possible to apply custom metric calculations on the client side.

 

Publisher (optional)

This universe provides all information related to the Publisher feature: campaigns, advertisers, creatives, formats, general position, detailed position, variant and URL of the clicked links. It includes the Publisher Clicks and the Publisher Impressions metrics.

 

Self-promotion (optional)

This universe provides all information related to the Self-Promotion feature: campaigns, categories, creatives, formats, products, variants. It includes the Self-Promotion Clicks and the Self-Promotion Impressions metrics.

 

Rich Media (optional)

This universe provides all information related to the Rich Media feature: level 2 sites, media type, themes, content, broadcast.

 

Sub-universes

As the data from Rich Media is divided into 3 main categories, Animation, Audio and Video, you will find a dedicated sub-universe for each of these categories for dedicated dimensions (ad type, post-roll origin, etc.)

 

MV Testing (optional)

This universe provides all information related to the MV Testing feature: tests, waves and creations.

 

Orders (optional)

The SalesTracker option (e-commerce module) is divided into two universes: Orders and Products. The Orders universe provides all information related to Orders analyses: orders, number of orders, sales, new customers, payment methods, promotion codes, shipping methods, status, customer ID.

If you did not subscribe to the SalesTracker module, you can still retrieve basic information about orders : Order id and Sales.

 

Products (optional)

This universe provides all information related to the Products analyses: orders, products, prroduct views, quantity of abandoned products, quantity of purchased products, status, product categories, product reference, promotion codes.

Note: The product views metric will only display data if you activated the storage option on our side. To benefit from this metric, please refer to our product documentation.

 

Page custom variables (optional)

This universe provides all information related to the Page custom variables.

 

Sub-universes

As the data from Page custom variables is specific to a page, you will have to select the page you want to retrieve the variables in the sub-universes.

 

URL structure

The generation of the file will start once the API URL is called on the client side. The extract has the following naming convention:

getdata_< datetime >_< site >_< universe >.csv

To request a full day of data, several API calls will have to be made: 1 per site, 1 per hour, and 1 per universe/sub-universe.

 

Format

Two formats are made available: CSV and JSON. Format can be chosen when configuring the URL or by editing the URL.

Note: we recommend that you use CSV format when possible as this format is lighter than JSON. However, if JSON is more convenient for you, we recommend that you enable compression in your header (for more information, please refer to the Advice and Optimizations) section.

 

Parameters

Here’s the structure of the URL – it is similar to what you may have experienced with Data Query (our Rest API.)

Then, 3 parameters complete this URL:

  • Columns: contains the CSV file columns. The columns must be contained between { } with a comma as a separator.
  • Space: contains the site ID under the s parameter between { }.
  • Period: information on the date and the hour of the request in this format: YYYY-MM-DDTHH. It is also possible to query the Data Flow API over a half hour period with the following period parameter: &period={Mn:{start:’yyyy-MM-ddTHH:mm’,end:’yyyy-MM-ddTHH:mm’}}

Here is an example of a request for our Demo Website; on June 27th 2016 from 4PM to 4.59PM.

Here is an example of a request for our Demo Website; on July 25th 2018 from 9:30AM to 9:59AM.

 

Separators

If no information is mentioned in the URL, the default separator will be a semicolon. If some of your labels already contain a semicolon, you may use another separator. It is possible to add this complementary parameter to the URL: &csvsep=.

Value Separator
csvsep=comma Comma
csvsep=tab Tab
csvsep=scolon Semicolon

Here’s an example:

Please note that URLs need to be saved from the Data Flow interface in order to work (it is not possible to create on-the-fly requests).

 

Authentication

Authentication is required for each of the calls so that data access can be limited to authorized personnel only. In order to access the data, you need to have an AT Internet account with access to the site(s) from which you need to extract data. It is possible to encode the password and login in base64 for security reasons when setting up an automated process.

 

Data availability

Data is retained in Data Flow for the past 48 hours. We recommend that each hour of data be extracted as soon as the hour is complete and the data is available.

In case of emergency, data can be requested from an additional 48 hours back; to only be used if necessary, as the API performances will be reduced compared to API calls processed on the current day.

 

Limits

  • There is no row limitation in this API. A file will contain one hour of data regardless of the number of rows.

  • Only one URL per universe or sub-universe is allowed per site. For example, it is not possible to create an Event flow with internal search engine dimensions and another Event flow with the click dimensions. All the dimensions you need have to be gathered into one flow.

Was this post helpful?
Yes
No
Last update: 31/07/2018