App to EntOS data requirements

Firebolt metrics and manual data sharing
The following page walks through the required data across consumption, quality of service and performance from app partners

Quality of playback experience

Quality of playback experience: Normal Playback Operations

The above is an example of the best-case notifications for a piece of content being loaded and then played
through to the end with no user interaction or network/playback impairment

• mediaPlaying() minus mediaLoadStart() gives TTFV (Time to First Video)

• mediaProgress() is sent every 60 seconds and is used to track resume points for content by the system

• mediaRenditionChange() is sent when there is a change in the selected ABR profile. This can be a change in profile, bitrate, resolution, etc. The new profile parameters are shared as part of the call.

Firebolt Metrics APIs

Quality of playback experience: Buffering during Playback

• In the case where playback is stalled (through buffering, lost connection, etc.) the application must call the mediaWaiting() API to indicate when the starvation begins and then call mediaPlaying() once the content is playing again and visible to the user

• Hence mediaPlaying() minus** mediaWaiting()** indicates the duration of the freeze

Firebolt Metrics APIs

Quality of playback experience: Playback restart

The above is an example of the expected behaviour when content is intentionally paused by the user and then restarted

• mediaPause() is called when media playback is paused due to an intentional pause operation

• mediaPlay() is called when media playback should start again due to unpausing

See requirement TP1/TP2
Firebolt Metrics APIs

Quality of playback experience: Time to Play from trickplay

• Whilst content is playing back the user may choose to use trick modes to fast-forward or rewind.

• In these situations, mediaRateChange() is called to indicate the new playback rate

Note: depending on the change in state, mediaPause() and/or mediaPlay() may be called.
See requirement TP1/TP2
Firebolt Metrics APIs

Quality of playback experience: Time to Play from scrubbing

Whilst content is playing the user may interact with UI elements to move to particular positions in the content

• In this scenario the application should use the** mediaSeeking()** call, followed by mediaSeeked() when the seek is complete

• Based on the user actions, there may be additional** mediaSeeking()** calls without a **mediaSeeked() **call for the prior mediaSeeking()

Note: depending on the change in state, mediaPause() and/or mediaPlay() may be called
See requirement TP1/TP2
Firebolt Metrics APIs

Quality of playback experience: Errors during playback

If an error occurs within your app you can inform the platform of the error which will allow EntOS to triage the issue with other system elements at
the time of the error in addition to tracking the volume of errors occurring

• This is achieved via metrics.error() and allows attributes such as error code, short descriptive string, whether visible to the user and additional elements to be shared for example media asset ID if encountered during playback

See requirement TP1/TP2
Firebolt Metrics APIs

Quality of playback experience

Note1: Data is stored against pseudonymised customer identifiers, device and account. Data as is is not traceable back to an individual customer. With the matching data that EntOS holds separately, if for example a customer is experiencing issues then EntOS would be able to identify the data related to that customer. Manual data access is strictly managed in line with InfoSec policies and procedures. In order to link data back to a subscriber lookups and reverse obfuscation of id required.

In app viewing data

Continue watching

Continue Watching rail (Play Now) provides customers with a way to get back to where they were, as well as signalling new episode availability through automated system logic. This is all driven through viewing data (resume and fully watched).

Firebolt Discovery APIs
Firebolt Metrics APIs

Showpage prioritisation

Viewing data enables prioritisation of the most recent provider when content is
shared across two or more providers, hero’ing the content in Showpages

Firebolt Discovery APIs
Firebolt Metrics APIs

Automated rails

Our recommendations algorithms use viewing from across the base including all partners
where the data is available. This enables features and rails like More like this, Because you
watched, Trending, and sorting order. We continually monitor and evolve our algorithm to ensure that we are recommending the most relevant content for our customers.

Firebolt Discovery APIs
Firebolt Metrics APIs

Viewing data (consumption)

Firebolt Discovery APIs
Firebolt Metrics APIs

Note1: Data is stored against pseudonymised customer identifiers, device and account. Data as is is not traceable back to an individual customer. With the matching data that EntOS holds separately, if for example a customer is experiencing issues then EntOS would be able to identify the data related to that customer. Manual data access is strictly managed in line with InfoSec policies and procedures. In order to link data back to a subscriber lookups and reverse obfuscation of id required.

Performance & Engagement

Performance & engagement: engagement metrics

These metrics enable engagement measures in app, and do not require media asset IDs. This provides an accurate viewing engagement metric versus time spent browsing in app.

Firebolt Metrics APIs

Performance & engagement: errors during browse

If an error occurs within your app you can inform the platform of the error which will allow EntOS to triage the issue with other system elements at the time of the error in addition to tracking the volume of errors occurring

• This is achieved via metrics.error() and allows attributes such as error code, short descriptive string, whether visible to the user and additional elements to be shared for example

• EntOS requires a list of error codes and reasons in advance of testing/trials.

See requirement OR02
Firebolt Metrics APIs

Performance & engagement: quick launch feature

• For a cold or preloaded launch, the application is responsible for calling Lifecycle.ready() once it is ready for the user to interact with it. Using this indicator we can determine TTMU (Time to Meaningful Use)

• Note this applies whenever an application is placed into inactive state.

• When an app is closed (but not necessarily unloaded) the app calls Lifecycle.close() and passes a reason code (CloseReason):
  – userExit, the user explicitly selected an exit control within the app
  – remoteButton, the user pressed the last/back button on their remote from the App’s Home page
  – error, the app encountered an unrecoverable error, and needs to be exited

See requirements QS1 to QS7

Firebolt Lifecycle APIs

Performance & engagement

See developer portal here:
Firebolt Discovery APIs
Firebolt Metrics APIs
Firebolt Lifecycle APIs

Note1: Data is stored against pseudonymised customer identifiers, device and account. Data as is is not traceable back to an individual customer. With the matching data that EntOS holds separately, if for example a customer is experiencing issues then EntOS would be able to identify the data related to that customer. Manual data access is strictly managed in line with InfoSec policies and procedures. In order to link data back to a subscriber lookups and reverse obfuscation of id required.

Manual Metrics

Standard performance metrics

We work with Partners to provide the best possible experience for our customers. As part of this, we meet on a regular basis (determined in engagement) to review metrics, general performance, and anything else that the Partner or EntOS would like to discuss. In order to facilitate this, the Partner should provide metrics monthly or quarterly (with a daily view of data) so that we can discuss where there is opportunity for improvement.

These metrics are outside of automated data shared via Firebolt and intended to support more comparative
insights across peer platforms at intervals determined together.

Standard performance metrics - quality of service

Quality of service metrics - Peer comparisons

Quality of service metrics for EntOS are provided through Firebolt as per the 'Quality of playback experience' and 'In app viewing data' sections. Outside of this, Partners should prepare a comparison of EntOS vs. other platforms using key metrics so we can assess where to focus on improving the customer experience. For example, buffer rates for EntOS versus comparable peer platforms.

Quality of service metrics - Live

For Partners who have live content in their app, the following should be reported monthly with a comparison to other platforms:

• Streaming latency – average time delay in live events

Standard performance metrics - performance and engagment

App performance metrics

All data should be provided for EntOS platforms broken down by territory along with a comparison to comparable peer platforms where applicable to identify and discuss opportunities for improvement.

• Stickiness: Daily Active devices divided by Monthly Active devices; 7 day moving average (with comparison to other platforms)

• Launches per device: # app launches over prior 28 days divided by Monthly Active devices (with comparison to other platforms)

• Conversion: % of app launches that result in viewing event; daily view and 28 day moving average (with comparison to other platforms)

• View Hours: Total # Hours of content viewed per day; daily view, 28 day moving average view, and cumulative sum

• Hours per device: Hours of content streamed over past 28 days divided by Monthly Active devices (with comparison to other platforms)

• Content per session: Average number of pieces of content launched per session in app; daily view and 28 day moving average (with comparison to other platforms)

Content performance metrics

The following data will enable us to ensure we are merchandising Partner content in the best possible way and help identify any opportunities that better cater to our customers’ tastes.

• Top Genres: Most popular genres consumed within the app for EntOS vs. all platforms, broken down by EntOS territories the app is live in.

• Top 10 titles: Top 10 most popular titles overall and for each genre, for EntOS vs. all platforms, broken down by EntOS territories the app is live in.

App marketplace (where applicable)

To help us understand the customer experience of the sign-up journey and where we can make improvements we require the following (if not provided via formal technical integration):

• Average time from purchase to activation; daily view and 28 day moving average

• Average time taken from activation to sign in; daily view and 28 day moving average

• No. of daily and monthly activations. EntOS can provide monthly purchases but is available via AmDocs