Streaming Industry PRISM Study - Methodology

Background

As high-speed internet access spread across US households - and households all over the world - in the 2000’s, a few innovative companies revolutionized the way that consumers paid for access to television shows and movies. Companies like Netflix, Hulu, Amazon Prime Video, HBO Now, and others offered their subscribers access to thousands of hours of popular content available when and where they wanted to watch at a fraction of the cost of traditional media distribution channels like cable and satellite television. The ultra-appealing streaming media platforms drove a massive trend in “cord-cutting” - the term used to describe cable and satellite television customers who cancelled their subscriptions in favor of a high-speed internet connection and a subscription to one (or more) of the rapidly-growing streaming platforms.

The success of platforms like Netflix has attracted the attention of large players in the media industry, many of whom own the underlying rights to the most popular content on third-party streaming services. In recent months, content-owning companies like Disney, NBCUniversal, and Warner Media have announced plans not to renew their agreements with Netflix and other streaming providers who license the rights to include their content in streaming services. Content owners plan to launch their own streaming services, starting with Disney’s “Disney+” in September 2019.

Anticipating the looming shake-up of this massive industry, analysts from Wall Street to Hollywood are predicting a wide range of outcomes for defending market-leaders like Netflix as well as new entrants like Disney. Many analysts’ expect Netflix to suffer tremendously as a result of losing the rights to key assets in its content catalog while major new competitors leverage their deep catalogs and production capabilities to drive subscriptions. Others believe that Netflix has developed the competency, financial resources, and (perhaps most importantly) troves of user data that it can use to produce tomorrow’s blockbuster movies and series.

With all of the speculation about what will happen in the streaming media market, few have provided robust data that is rigorously collected and analyzed to predict customer decisions after the entrance of new providers. The Langston Co applied our flagship analysis approach - called PRISM - to evaluate the streaming market, the pending changes, and what this all means for competitors new and old in the streaming media landscape.

Here are three substantial advantages this new study offers over existing reports on the industry:


Direct responses from a statistically robust sample. The PRISM methodology applied in this study uses data collected directly from more than 1,200 participants across the country. We gathered information about satisfaction and perceptions among this group, sourcing respondents from multiple pools of survey respondents and balancing the sample against distributions of specific variables in the US Census. A detailed description of this study’s sample of respondents for the quantitative analysis is available in the section titled “Sample” farther down this page.

Most publicly available studies of this market have relied on “convenience samples” where respondents are pulled from an available traffic source, such as visitors to a specific website or current subscribers of a single streaming platform. This approach can lead to substantially flawed results because of the biases and pitfalls inherent to convenience sampling.

Usability, features, platform costs, and add-on charges. While many previous studies have gone to great lengths to capture information about the content available on each platform, very few of them capture a full range of information about the additional features, usability, costs, and barriers that customers experience among streaming platforms. Leveraging foundational elements of Customer Value Management - a time-tested market research methodology - the Langston Company collects and considers information about both sides of the decision-making coin - the what I get and what I give. It is the combination of these factors that determine whether someone feels it is worth it to subscribe to a streaming platform.

Relative Importance. Most previous studies have been unable to consider the relative importance of each factor to the others. Indeed, while the importance of content catalog has dominated the discourse around this market, few other purchase drivers have been factored into most arguments. The PRISM methodology allowed the research team to analyze the relative importance of each factor using advanced statistical methods. By comparing the range of satisfaction ratings among each factor from thousands of participants, the Langston Company can identify which factors have a stronger relationship to overall satisfaction than others. This allows us to understand which attributes of a platform’s costs and benefits have the strongest impact on customers’ decisions to subscribe, renew, or cancel each service.

Objectives

The Langston Company conducted this study to capture valuable information on the true drivers of customer decision-making in terms of streaming platform sign-ups and renewal. Specifically, the research team was instructed to pursue the following objectives.

  1. Achieve visibility into how consumer behaviors and expectations will change in light of Disney, NBCUniversal, and Warner Media launching their own platforms.

  2. Dissect drivers of value using data on the current top 3 streaming platforms - Netflix, Hulu, and Amazon Prime Video - to measure (a) what matters to customers in key segments, and (b) how well each company performs along each value driver.

  3. Provide insight into key market segments for successfully attracting and retaining subscribers.

  4. Generate actionable insights for existing players and new entrants as the industry prepares for the launch of additional platforms.

Methodology

The study utilized The Langston Co’s proprietary PRISM methodology. This framework relies on a combination of qualitative and quantitative information about the perceptions of a target group. While the historical application of the Customer Value Management methods have focused on customers of a specific business or industry, partners at The Langston Co have discovered tremendous value in modified applications for other target groups, including entrepreneurs, employees, and migrants.

Qualitative Foundation

In order to understand the nuanced perspectives and opinions of consumers in the streaming media market, The Langston Co’s qualitative research team conducted individual interviews with members of the target sample population. These included a combination of in-person and telephone interviews and the majority of interviews were recorded with permission of the respondent (those where recording permission was refused were not recorded). Interviews were conducted by trained consumer researchers with the goal of uncovering the specific factors and opinions that drive purchase decisions within the streaming media market. Our team also explored the general consumer sentiment toward the market, its current providers, and the anticipated changes to the industry.

Information gathered in the qualitative interviews was used to design the value proposition model (described below) which forms the foundation of the quantitative study. Additionally, results from the interviews were used to understand and interpret findings of the quantitative analysis, although findings were not shared with the analysis team until the conclusion of all data collection and raw analysis of the quantitative sample was complete. Lastly, information from the qualitative interviews was used to understand the general consumer outlook on the streaming media industry in an effort to understand the likely impact of current and potential strategies on customer satisfaction, adoption, and retention.

Value Proposition Model

At the core of the methodology is the concept of value. While the word, value, has been assigned many meanings and uses in our culture, this methodology approaches it in the fundamental sense as describing whether the benefits of something are worth the costs. In the case of this study, The Langston Co. set out to quantify the impacts of various factors on whether customers who consider both the benefits and costs of subscribing to each platform believe that it is a service worth paying for.

The concept of value, in the meaning of “worth,” has very strong connections to the types of behavioral outcomes that people care about. In the case of customer satisfaction, a person’s perception of how much a product or service is worth what they paid is highly predictive of their likelihood to buy that same product again. Further, the perception of value offered by a service - or product, or employer, or any other aspect that people chose - is highly determinant of the way that person talks to others about it. People who believe that the benefits of a streaming platform are very worth the costs of having it will speak about that platform favorably with their friends, family members, and colleagues.

While there are two key components of “worth” - what I put in and what I get out - each of these has several components that factor into them. In the case of what I put in, prospective subscribers of a streaming platform consider things like the monthly cost, any contract commitments, as well as the cost of extra features or add-ons, such as the ability to stream to multiple screens at the same time or view content in High Definition or Ultra-High Definition (4K). In the case of what I get out, prospective subscribers consider both the content itself (including aspects such as variety, overall depth of catalog, and the availability of specific genres) as well as the user experience of the platform (including aspects such as personal recommendations, the ease of finding new shows to watch, and specific features like kids mode and watching shows offline).

The figure below illustrates the main components and sub-components of value in the context of streaming media providers. We call these components “attributes” or “key decision criteria.”

streaming value tree.png

Each of those key decision criteria also breaks down into a number of other influencing factors. As such, the model that describes value for a streaming subscriber can be quite complex and very comprehensive.

Our study further broke down the key decision criteria into several sub-attributes. Because of its branch-like structure, this type of diagram is called a value tree. It forms a portion of the foundation of the PRISM methodology.

Sample

For this survey we used a combination of multiple pools to source participants (sometimes called respondents). The pool included two survey panel providers as well as a pool of respondents independently sourced by The Langston Co researchers using targeted online communities. Balancing sample respondents across multiple sources allows our survey team to mitigate the risk of biases in the data caused by nuances and artifacts in any one provider’s respondent pool.

The sample featured 1,264 responses from individuals aged 25-54 (members of the Millennial and Gen-X generations) because of their high degree of purchasing power, activity in the streaming media market, and presence of children in the household, which has been a major source of speculated success or failure among market players. The survey sample was also balanced against US Census distributions of income, gender, and geographic location to ensure representativeness across these attributes. We employ a series of techniques to identify and remove data that appears suspicious (e.g., straight-lining behavior).

Quantitative Analysis

The qualitative model of value-based consideration allows Langston Co. researchers to conduct several quantitative analyses. To begin the quantitative portion of the study, researchers seek survey responses from a statistically robust sample of the population being studied. In this case, that meant gathering survey feedback from over 1,200 US residents on three streaming platforms - Netflix, Hulu, and Amazon Prime Video - as well as three new platforms expected to launch in the new future - Disney+, an unnamed platform from NBCUniversal, and HBO Max (Warner Media). The streaming platforms selected for this study were based on prominence in the public discussion of the anticipated shake-up of the streaming media market.

During the survey, respondents were asked to identify which of the three existing streaming platforms they are currently familiar with. They were then presented with up to two randomly-selected platforms and asked to provide a satisfaction rating for each branch of the value tree on each of those platforms. Survey respondents also answered questions about their likelihood to exhibit certain behaviors, as well as a number of short open text questions as well as demographic and employment information. Respondents were then given a description of the changing media landscape. They were told that Disney, NBCUniversal, and Warner Media are expected to start their own streaming platforms and would transfer old favorites like The Office, Friends, and Parks & Rec to these new platforms. With this new information, respondents were asked how they anticipated all six platforms (Netflix, Hulu, and Amazon Prime Video as well as Disney+, NBCUniveral’s unnamed platform, and Warner Media’s HBO Max) to fare on dimensions of content quality and overall value, as well as their likelihood to subscribe to each service.

The research team utilized these responses to calculate several useful metrics, starting with an average satisfaction score for each of the key decision criteria and their sub-attributes. These scores were compiled for the sample as a whole, for individual platforms, and for members of different demographic sub-groups, including age and parental status. These scores provide a baseline against which each competitor and group can be compared to understand the relative satisfaction for that group as compared to the overall sample. They also provide a baseline for other media distribution platforms that might be studied in any further expanded sample.

The performance scores provide the opportunity to understand which competitors offer the greatest perceived value - meaning “this platform is worth paying for” - as well as a detailed picture of how different competitors stack up on each of the key purchase criteria and their sub-attributes. This provides the opportunity to build a highly nuanced understanding of the differences between competitors, allowing us to grasp reasons that various streaming platforms attract and retain/lose subscribers of one sort or another.

In addition to calculating relative performance on key decision criteria and overall value, the research team applied specific regression modeling to estimate the relative importance of each sub-attribute to the overall perception of value. By allowing advanced models to derive correlations between sub-attributes and their higher-level key decision criteria, the team is able to grasp an understanding of the actual drivers of perceived value while eliminating social or personal bias of the survey respondents. The relative strength of each variable, as calculated by our research team, is referred to as its impact weight.

With the combination of average scores and impact weights, the research team has built a highly comprehensive understanding of the calculus that drives a sense of value for prospective subscribers to each of the streaming platforms we studied.

For questions about our approach, please contact us.