Creating a High-Performance Recommendation System: Unveiling the Construction Process of Etsy’s Multi-Task Canonical Ranker
Introduction:
Introduction:
In the vast marketplace of Etsy, with over 100 million unique listings, it can be overwhelming for buyers to find that perfect item. That’s where our recommendations come in. We provide personalized suggestions tailored to each user’s shopping mission, making it easier to discover unique, trending, and relevant items. These recommendations, or modules, are designed to help users pick up where they left off, explore new interests, and find items from their favorite shops. As an enterprise-scale recommendation system, our modules work in two phases: candidate set selection and candidate set ranking. In this post, we will focus on the ranking phase and discuss how our new canonical rankers are revolutionizing the way we deliver recommendations on Etsy.
Full Article: Creating a High-Performance Recommendation System: Unveiling the Construction Process of Etsy’s Multi-Task Canonical Ranker
Etsy, an online marketplace, is focused on providing personalized recommendations for buyers to help them find unique and special items. With over 100 million listings on the platform, Etsy utilizes recommendation modules to assist users at different stages of their shopping journey.
Two Phases of Recommendation Modules
The recommendation modules on Etsy work in two phases: candidate set selection and candidate set ranking. In the first phase, the objective is to quickly retrieve a relevant set of items from the entire inventory. In the second phase, a more sophisticated machine learning model, known as a ranker, ranks the items in the candidate set based on contextual attributes and item attributes. Contextual attributes include the user’s recent purchases and most clicked categories, while item attributes refer to features such as the item’s title and taxonomy.
Moving Towards Canonical Rankers
Traditionally, Etsy has used a one-to-one approach, where each recommendation module had its own ranker trained exclusively on data collected from that module. However, as the number of modules increased, it became challenging and costly to maintain and iterate on so many rankers. To address this, Etsy introduced canonical rankers.
Canonical rankers are models optimized for specific user engagement metrics, but their objective is to power multiple modules. The goal is to ensure that these rankers perform at least on par with module-specific rankers while being more computationally efficient and cost-effective to train and maintain.
Introducing the Canonical Frequency Ranker
One of Etsy’s first canonical rankers focused on visit frequency. The aim was to identify latent user interests and surface recommendations that would inspire future shopping missions. The ranker was trained to optimize for favorite rate, as it closely related to a user’s intention to return to the site.
To train the ranker, Etsy analyzed various user interactions, including clicks, favorites, collections, and purchases. Favoriting was found to be most closely related to a user’s intention to revisit the site. However, the ranker needed to be cautious about not distracting users from their current shopping session and potentially jeopardizing sales.
Developing the Model Structure
Given the goal to optimize for both favorites and purchases, a multi-task learning framework was adopted. A neural model was chosen due to its ability to handle multi-task architectures effectively. The ranker employed a shared-bottom structure, where common factors between favoriting and purchasing were expressed in shared layers at the bottom of the network, while separate layers were used towards the top.
Improving Model Performance
To further enhance the model’s performance, an expert layer was added following the Multi-gate Mixture Model of Experts (MMOE) framework. This framework allowed favorites and purchases to learn different representations from the embeddings, resulting in more relevant recommendations without significant additional computation cost.
Building a Canonical Ranker
In addition to using data from multiple modules, Etsy considered both training data and model structure when developing the canonical ranker. Offline tests were conducted with a naive multi-task model on eight different modules. The results showed promising performance, validating the effectiveness of the canonical ranker approach.
Conclusion
Etsy’s shift towards canonical rankers aims to improve the efficiency and maintenance of its recommendation modules. By training rankers that can power multiple modules, Etsy can provide personalized recommendations while minimizing engineering costs. The introduction of the canonical frequency ranker exemplifies Etsy’s commitment to optimizing user engagement and providing a go-to destination for shoppers.
Summary: Creating a High-Performance Recommendation System: Unveiling the Construction Process of Etsy’s Multi-Task Canonical Ranker
Etsy, a popular marketplace for unique and handmade items, has implemented a new recommendation system called canonical rankers to improve the efficiency and effectiveness of its modules. These rankers, trained to optimize user engagement metrics such as clickthrough rate and conversion rate, can power multiple modules instead of having separate rankers for each module. The first canonical ranker implemented by Etsy focuses on visit frequency and aims to identify latent user interests and provide recommendations that encourage users to return to the site. The ranker is built using a multi-task learning framework and neural models to predict both favoriting and purchasing probabilities. By utilizing data from multiple modules and considering training data and model structure, Etsy aims to develop more efficient and effective recommendation systems that provide users with personalized and relevant recommendations.
Frequently Asked Questions:
1. Question: What is machine learning and how does it work?
Answer: Machine learning is a subset of artificial intelligence that allows computer systems to learn and improve from data without being explicitly programmed. It involves the development of algorithms and models that can analyze and interpret complex patterns in data, enabling computers to make predictions and decisions. Machine learning algorithms learn from training data to make accurate predictions or take certain actions when faced with new data.
2. Question: What are the different types of machine learning algorithms?
Answer: Machine learning algorithms are broadly categorized into three types: supervised learning, unsupervised learning, and reinforcement learning.
– Supervised learning involves training a model on labeled data, where the input data is paired with corresponding output values. The algorithm learns to map input data to output values by finding patterns and correlations in the training data.
– Unsupervised learning, on the other hand, deals with unlabeled data. The algorithm analyzes the input data to identify patterns or group similar instances together without any predefined labels.
– Reinforcement learning involves training a model through interaction with an environment, where the model learns to take actions and receives feedback or rewards based on its performance.
3. Question: What are the main applications of machine learning?
Answer: Machine learning finds applications across various industries and domains. Some of the main applications include:
– Natural language processing: Machine learning is used in tasks such as text classification, sentiment analysis, chatbots, and language translation.
– Computer vision: It is employed in image recognition, object detection, facial recognition, and autonomous vehicles.
– Fraud detection: Machine learning algorithms are utilized in identifying potential fraudulent activities in financial transactions.
– Recommender systems: These systems make product recommendations based on user preferences and behavior patterns.
– Healthcare: Machine learning plays a crucial role in medical diagnosis, drug discovery, personalized medicine, and patient monitoring.
4. Question: What are the main challenges in machine learning?
Answer: Machine learning faces several challenges, including:
– Data quality and quantity: Obtaining sufficient and high-quality training data is crucial for accurate model training. Insufficient or noisy data can adversely affect the performance of machine learning algorithms.
– Bias and fairness: Bias can be inadvertently introduced into models due to biased training data or algorithmic design. Ensuring fairness and mitigating bias is an ongoing challenge.
– Interpretability: Some complex machine learning models, such as deep neural networks, lack interpretability. Understanding and explaining the decision-making process of these models is a challenge.
– Overfitting and underfitting: Overfitting occurs when a model performs well on training data but fails to generalize to new, unseen data. Underfitting, on the other hand, occurs when a model fails to capture the underlying patterns in the training data.
– Computing power: Some advanced machine learning algorithms require significant computational resources and processing power, making them inaccessible for certain applications or organizations.
5. Question: What are some popular machine learning frameworks and tools?
Answer: There are several popular machine learning frameworks and tools available to assist in developing and deploying machine learning models. Some examples include:
– TensorFlow: Developed by Google, TensorFlow is an open-source deep learning framework widely used for building neural networks and other machine learning models.
– PyTorch: PyTorch is another powerful deep learning framework known for its ease of use and dynamic computational graphs.
– scikit-learn: scikit-learn is a popular Python library that provides a wide range of machine learning algorithms and tools, making it suitable for both beginners and experts.
– Keras: Keras is a user-friendly open-source neural network library written in Python. It offers a high-level API for quick prototyping and experimentation.
– Apache Spark: Spark is an open-source distributed computing system that includes machine learning libraries and APIs, enabling scalable and fast machine learning processing on large datasets.
Remember, machine learning is a vast and evolving field, so it’s essential to stay updated with the latest techniques, algorithms, and tools to make the most out of this exciting technology.
