The Cold Start Problem: Quantifying the Missed Opportunities in Traditional E-Commerce Recommendations

In most recommender systems, models rely on historical user interaction data to provide personalized recommendations through a process known as collaborative filtering. These models, identify patterns in user behavior and item interactions to suggest products that users are likely to engage with. However, without engagement data, these systems struggle to provide relevant recommendations, leading to a significant missed opportunity in user engagement and sales.

When preparing datasets for collaborative filtering, researchers often apply a process called “thinning,” where users and items with very few interactions are intentionally removed. In research benchmarks, datasets like the MovieLens 100K dataset remove users with fewer than 20 interactions, the Amazon Product Data dataset removes users with fewer than 5 interactions, and the Book-Crossing dataset removes users with fewer than 10 interactions. This thinning process is necessary to ensure stabality in these Collaborative Filtering models, but does beg the question: how many users and items are we excluding from our datasets, and what is the opportunity for retailers in e-commerce in improving the experience of new and anonymous users?

Anonymous Users

In 2022, a study by Braze found that up to 86% of users on retail and e-commerce platforms browse anonymously, supporting similar such studies from Bluecore and PayPal findings on user login preferences. Although these anonymous visitors are often seen as low-value, they actually represent a substantial opportunity for retailers in terms of conversion and new customer acquisition. For retailer, while returning users may have existing credentials, anonymous users are often new visitors who have not yet created an account or logged in and who may be exploring the site for the first time with the eager intention of making a purchase. For digital marketers, if hyper-personalized represents the best experience on your site, and new users are often anonymous, bid prices may be significantly inflated for these users, hurting ad performance and return on ad spend (ROAS).

Click-Through Rates

Even for logged-in users, the need for repeated interactions may present a significant limitation to the effectiveness of traditional recommendation systems. With click-through rates between 2 - 5%, less than half of logged-in users may have enough product click data to included in “thinned” datasets. For online retailers, this means that even if 14% of users are logged in, fewer than 7% of users may have enough data to receive personalized recommendations.

New Products

The cold start problem is not limited to new users; it also affects new products. When a new product is introduced, there is no historical interaction data to inform its relevance to potential customers. This is particularly significant in fast-paced industries like fashion, homeware or technology, where new items are added daily. For example, fast fashion brands like Shein may add up to 10,000 new SKUs daily, to their 600,000 SKU catalog, making it nearly impossible for traditional recommendation systems to keep up with the latest trends and user preferences. In these scenarios, many products may be sold-out before they even have a chance to be recommended to users, leading to lost sales opportunities. The inability to find relevant products can frustrate users, leading to a poor shopping experience and potentially lost sales, and the lack of exposure of new product may significantly impact stock turnover and sales velocity, which are critical metrics for e-commerce success. In these fast fashion scenarios, fewer than 20% of products may survive the “thinning” process, leading to a significant loss of potential sales and user engagement.

UX Matters

Historically, recommender systems have been limited to homepage modules, showing between 10 and 20 products to users. These modules may be precomputed and cached, leading to a lack of real-time personalization. This approach can lead to a poor user experience, as users may not find relevant products or may be shown products that are out of stock, and exhibit changing shopping intent throughout their shopping journey. With around 200 impressions per session, traditional carousels may represent a 5% of users impressions, leading to missed opportunities for engagement and sales.

The Opportunity

The cold start problem presents a significant opportunity for retailers to improve the user experience and increase sales. By leveraging Zero-shot discovery, recommendations can span both new stock and new users, offering real-time hyper-personalized across product listings, search, product page and checkout experiences. This approach can help retailers to better engage with anonymous users, improve click-through rates, and increase the visibility of new products. By providing a more personalized and relevant shopping experience, retailers can increase user engagement, conversion rates, and ultimately, sales. If recommendation only cover 20% of the catalogue, 7% of users and 5-10% of impressions, online retailers may be leaving more than 98% of the user experience on the table, greatly impacting their bottom line and opportunities for growth.