Engineering Airbnb's Embedding-Based Retrieval System

Introduction

Airbnb's embedding-based retrieval system powers both search and recommendations, helping millions of guests find their perfect accommodation. This case study explores the engineering decisions behind this critical system.

Problem Statement

Airbnb faces unique search challenges:

• Heterogeneous inventory: From shared rooms to luxury villas
• Complex preferences: Location, price, amenities, style
• Two-sided marketplace: Matching guests and hosts

Embedding Architecture

Listing Embeddings

Each listing is represented by embeddings capturing:

• Visual features: From listing photos
• Text features: Description and reviews
• Structured features: Price, location, amenities
• Behavioral features: Booking patterns

User Embeddings

User representations include:

• Search history: Recent and historical searches
• Booking history: Past stays and preferences
• Demographic signals: Where appropriate

Training Approach

# Simplified training objective
def embedding_loss(user_emb, pos_listing_emb, neg_listing_embs):
    pos_score = dot(user_emb, pos_listing_emb)
    neg_scores = dot(user_emb, neg_listing_embs)
    return contrastive_loss(pos_score, neg_scores)

System Architecture

Indexing Pipeline

1. Feature extraction: Process listing content
2. Embedding generation: Neural network inference
3. Index building: HNSW or IVF indices
4. Index deployment: Distribute to serving layer

Serving Pipeline

1. Query encoding: Generate user embedding in real-time
2. ANN search: Find similar listings
3. Re-ranking: Apply business rules and personalization
4. Response: Return ranked results

Challenges and Solutions

Cold Start

• Content-based initialization for new listings
• Location-based fallback for new users
• Exploration mechanisms for discovery

Freshness

• Incremental index updates
• Near real-time embedding refresh
• Availability integration

Impact

• Significant increase in booking conversion
• Improved guest satisfaction scores
• Better host matching for long-term stays

---

Learn more about embedding-based retrieval in our Recommendation Systems at Scale course.