Attention and saliency on the internet: Evidence from an online recommendation system

Christian Helmers, Pramila Krishnan, Manasa Patnam

25 January 2016

a

A

Is too much choice a bad thing?  There is much evidence from psychology that while more choice is almost always a good thing, too much choice can impede decision making (Schwartz 2004). This ‘paradox of choice’ is arguably best examined in the context of digital markets, which have seen an explosion in choice.

Over the past decade, retail e-commerce has seen a spectacular rise in popularity, with its share of total retail growing from 2.8% to 7.4%. This growth has been accompanied by a dramatic expansion of the number of products made available to consumers shopping online. A consumer wishing to purchase a television on Amazon would be confronted with over 3,000 products, and would need to consider at least 190 options after applying standard search filters.

How does this proliferation in the availability of options affect consumer decision making on the internet? Despite quick and cheap access to information online, consumers are often saddled with the difficult task of choosing among thousands of options made available to them. As a consequence, consumers who are often constrained by limits to their attention may ignore many relevant options.

This casual observation sits awkwardly with traditional choice theory, which relies on the axiom of revealed preference, assuming that an agent considers all feasible options. In new research (Helmers et al. 2015), we show empirically that this assumption no longer holds in the face of very large choice sets, such as those encountered in digital markets. In these settings, consumers may simplify decision-making by only considering a subset of all available products, over which they deliberate their choice (Masatlioglu et al. 2012).

Internet platforms have long recognised this issue and frequently engage in tactics to attract users’ attention towards their products by providing recommendations, thus creating the set for consumers to consider. Popular examples are websites such as Amazon and Netflix that recommend products to consumers based on past purchases or visual similarity of browsed products.

We evaluate the effect of recommendations provided by salient products online and find an increase in the probability that consumers consider recommended products. Conditional on consideration, recommendations have no further effect on the choice of product within the restricted set of options. The consideration effect translates into a sizeable increase in the sales for recommended products of around 6%.

The impact of product recommendations on sales

We use data on purchases from a popular, exclusively online retail store, which offers a list of product recommendations for every product available on the website. Importantly these recommendations are not personalised and instead are based largely on observable attribute similarity.

A casual association of product recommendations and product sales would, however, produce a biased estimate of the effect, because bestselling products could be easily chosen to be recommended by other products. If this is the case we would just pick up the effect of popularity rather than a saliency effect.

To mitigate this issue, we rely on three features of our online marketplace:

  1. The timing of new product arrivals;
  2. The fact that new products are highly salient upon arrival, drawing more attention; and
  3. Regional variation in the composition of recommendation sets.

New products are introduced thrice weekly on the website and are highly viewed upon arrival. Figure 1 plots the novelty effect for new products; it shows that new products are highly popular upon arrival and this effect declines over the week.

Figure 1. Demand for new products

Note: The figure reports coefficient estimates (with 95% confidence intervals) of the effect of entry – being a new product – on the total number of shopping bag additions (per day). The regression specifications controls for product and time fixed effects, as well as for day of the week and weekend.

The figure suggests that new products attract an enormous amount of consumer attention immediately when they are launched on the platform. We analyse what happens to the sales of an existing product after new products recommend it. Since new products are highly salient, the recommended existing products receive a positive ‘saliency shock’ which could affect their likelihood of being considered.

Figure 2 plots the disaggregated saliency effects for each day following the saliency shock along-with their confidence intervals. The figure shows a large 7% increase in total purchases for salient products on the day they receive the saliency shock, with the effect positive but declining over the subsequent few days.

Figure 2. Staggered effects of saliency shock

Note: The figure reports coefficient estimates (with 95% confidence intervals) of the effect of saliency on the total number of shopping bag additions (per day). Saliency is defined as the total number of new products that recommend the target product at any given point of time. The regression specifications controls for product and time fixed effects, as well as for day of the week and weekend.

On average an additional unit of saliency results in a 3-5% increase in total purchases over the three days following the shock.

To confirm these results, we conduct a simple placebo test built around regional differences in recommendation sets for the same new product. We examine whether products that received a saliency shock exclusively in Europe – that is, they were recommended by a new product (which itself was launched globally) only in Europe – increased in any way their US sales. Since consumers in the US are not able to view these products as salient in their region, we should expect no change in the products’ US sales. Our findings indicate that this is indeed the case and that there was no change in US sales of products made salient exclusively in Europe.

Display size effects

In addition to being recommended, product sales will be influenced by the overall size of the recommendation set if consumers have limited attention and can only focus on a restricted number of products at a time. As a result, products that are recommended in smaller sets should receive more attention, increasing their sales, compared to products recommended in larger sets. We find such negative and significant display size effects. An additional product in the recommendation set reduces total purchases by 4%.  Products recommended in sets of 1-3 experience the maximum effect of saliency, seeing an increase of about 22% in their sales, compared to products recommended in sets of 9, which see an increase of only 3.5% in their sales.

Effect of recommendation on the consumer consideration

While our analysis so far has shown that recommendations have a sizeable causal impact on sales, it is unclear how this effect is derived from a consumer’s choice process. It could be conjectured that products that gain saliency are likely to receive more attention by consumers, increasing the probability that they are considered. Even then, it is possible that, independent of increased attention, consumers derive utility from saliency – that is, they have a direct preference for products recommended by new products. To investigate this issue, we estimate a random utility model that recognizes the choice process as sequential and incorporates the formation of a consideration set in the first stage of a consumers’ decision-making process (De los Santos et al. 2012).

Our estimates reveal that saliency has a strong, positive effect on the consideration (of existing products) but no further effect on choice, conditional on consideration. We find that the saliency effect is higher within sub-categories where only a few choices are considered, typically those with a large set of available options.

The model also allows us to estimate counterfactuals that compare how sales shares for products change when consumers have limited versus full information. Figure 3 plots the results from the simulations. Each bar in the figure represents the percentage difference in sales share between when consumers have limited attention and when they have full attention. A negative value indicates that the share under limited information is lower than that under full attention. The x-axis orders a small sample of products by their sales rank (1 being most popular and 12 being least popular).

Figure 3. Limited vs. full attention difference in sales shares

Note: Products 1, 2 and 3 in the figure are categorized as ‘popular products' (observed sales > 10%); products 4 to 12 are categorized as ‘unpopular products.' Dark grey bars show results under the existing recommendation system. Grey bars show the difference between limited and full attention when only popular products are (always) recommended (products 1, 2 and 3). Light grey bars show the difference between limited and full attention when only unpopular products are (always) recommended (products 4 to 12).

Our results show that limited attention disproportionately harms top-selling products simply because they are not considered (dark grey bars). Under full attention however, they are always in a consumer’s consideration set and are frequently chosen based on their superior underlying characteristics. Popular products would, however, stand to gain under limited attention (by up to 4% of sales share difference for product 3), if the website only recommended popular products (medium grey bars), but this increases the concentration of sales towards popular products in the market.

References

De Los Santos, B, A Hortacsu and M R Wildenbeest (2012) “Testing models of consumer search using data on web browsing and purchasing behaviour,” The American Economic Review, 102: 2955–2980.

Helmers, C, P Krishnan and M Patnam (2015), “Attention and Saliency on the Internet: Evidence from an Online Recommendation System”, CEPR DP No. 10939. http://cepr.org/active/publications/discussion_papers/dp.php?dpno=10939

Masatlioglu, Y, D Nakajima and E Y Ozbay (2012) “Revealed attention,” The American Economic Review, 102: 2183–2205.

Schwartz, B (2004) “The paradox of choice: Why more is less”, New York: Ecco.

a

A

Topics:  Frontiers of economic research

Tags:  paradox of choice, online markets, E-commerce, choice, revealed preference, Amazon, Netflix, the Internet, psychology, limited attention, attention, online trade

Christian Helmers

Assistant Professor at the Economics Department, Santa Clara University

Senior Lecturer in the Faculty of Economics, University of Cambridge

Assistant Professor, CREST-ENSAE

Events