Building AI-Powered Search & Recommendation Engines for SaaS Products

Why Keyword Search Is No Longer Enough

Keyword-based search — the kind powered by Elasticsearch's BM25 or PostgreSQL full-text search — matches documents by the exact terms a user types. This approach has a fundamental limitation: it cannot bridge the vocabulary gap between what users ask and what your content contains.

Consider a B2B SaaS product with a knowledge base. A customer types "cancel my subscription" but the documentation uses "manage billing" and "terminate plan." Keyword search returns nothing useful. The customer opens a support ticket. Multiply this across thousands of users and you have a measurable impact on support costs and satisfaction scores.

Semantic search solves this by converting both queries and content into vector embeddings — dense numerical representations that capture meaning. "Cancel my subscription" and "terminate plan" produce similar vectors because they share semantic intent, even though they share zero keywords. This is the foundational technology behind the AI use cases transforming SaaS products in 2026.

The shift from keyword to semantic search is not binary. The most effective production systems use hybrid search — combining BM25 for exact-match precision with vector similarity for semantic recall, then fusing results with reciprocal rank fusion (RRF). This gives you the best of both approaches: exact matches when users know the right terminology, and semantic matches when they don't.

Semantic Search Architecture

A production semantic search system has four core components: an embedding model, an ingestion pipeline, a vector database, and a retrieval/ranking layer.

Embedding Models

The embedding model converts text into vectors. In 2026, the practical choices for SaaS applications are:

• OpenAI text-embedding-3-large: 3072 dimensions, strong general-purpose performance. Best when you already use the OpenAI API and want simplicity. ~$0.13 per million tokens.
• Cohere embed-v4: Excellent multilingual support and built-in compression. Good for international SaaS products with diverse language requirements.
• Open-source models (e.g., BGE-M3, GTE-large): Self-hosted, zero per-query cost after infrastructure. Best when you need data privacy guarantees or have very high query volumes that make API costs prohibitive.

For most SaaS applications, start with an API-based model. The cost at typical query volumes (under 10M queries/month) is negligible compared to engineering time spent managing self-hosted infrastructure. You can always migrate later — the embedding model is a swappable component if you design the pipeline correctly. Our AI API integration guide covers the pattern for making AI components replaceable.

Ingestion Pipeline

The ingestion pipeline processes your content into searchable embeddings. The key steps are:

1. Chunking: Break documents into passages of 200-500 tokens. Use semantic chunking (splitting at paragraph or section boundaries) rather than fixed-size windows.
2. Enrichment: Add metadata — document title, category, recency, access permissions — that enables filtered search at query time.
3. Embedding generation: Pass chunks through your embedding model. Batch processing keeps API costs low and throughput high.
4. Indexing: Store vectors and metadata in your vector database with the appropriate index type (HNSW for most workloads).

This pipeline runs in batch mode for initial indexing and incrementally as content changes. The data pipeline architecture guide covers ingestion patterns in depth.

Retrieval and Ranking

At query time, the user's search query is embedded with the same model, and the vector database returns the top-K nearest neighbors. But raw vector similarity is only the first stage. A re-ranking layer improves precision by applying a cross-encoder model or business logic (boost recent content, penalize deprecated docs, filter by user permissions) to produce the final ranked results.

"The teams getting the best results from semantic search are not the ones with the fanciest embedding models — they are the ones with the best re-ranking and filtering layers. The embedding gets you recall; the re-ranker gets you precision."

— Will Pienaar, VP of Engineering, Tecton AI, 2025

Recommendation Engine Approaches

While search is user-initiated ("I want X"), recommendations are system-initiated ("you might like X"). There are three primary approaches, each with distinct tradeoffs:

Collaborative Filtering

Collaborative filtering recommends items based on the behavior of similar users. If users A and B both engaged with items 1, 2, and 3, and user A also engaged with item 4, recommend item 4 to user B. This approach requires no understanding of item content — it works purely on interaction patterns.

Strengths: discovers unexpected connections, improves with more users. Weaknesses: suffers from cold-start (new users and new items have no interaction data), requires significant user scale (typically 10K+ active users to be effective).

Content-Based Filtering

Content-based filtering recommends items similar to what a user has already engaged with. If a user read three articles about "API rate limiting," recommend other articles with similar content. This approach uses the same vector embeddings that power semantic search.

Strengths: works with a single user's history (no cold-start for existing items), transparent reasoning ("recommended because you read X"). Weaknesses: creates filter bubbles, cannot discover cross-category interests.

Hybrid Approaches

Production recommendation engines almost always combine both methods. A typical hybrid architecture:

1. Candidate generation: Use collaborative filtering and content-based retrieval to produce 100-500 candidates from a catalog of thousands or millions.
2. Scoring: Apply a learned ranking model that blends collaborative signals, content similarity, recency, popularity, and business rules into a single relevance score.
3. Filtering: Remove items the user has already seen, items they lack permissions for, or items that violate diversity constraints (e.g., no more than 3 items from the same category).
4. Serving: Return the top N items with explanation metadata ("recommended because..." ) for transparency.

Building this as a personalization layer is one of the most impactful AI personalization patterns for SaaS products.

Real-Time vs Batch Recommendation Pipelines

The choice between real-time and batch recommendations depends on how quickly user behavior should influence what they see.

Batch pipelines pre-compute recommendations for all users on a schedule (hourly, daily). They are simpler to build and debug, cost-effective for large catalogs, and appropriate when recommendations do not need to reflect the last few minutes of activity. A nightly job computes "users who viewed X also viewed Y" matrices and stores results in a cache.

Real-time pipelines update recommendations as the user interacts. When a user clicks on an item, the system immediately adjusts subsequent recommendations. This requires event streaming (Kafka or similar), a feature store for low-latency feature lookups, and an inference service that can score candidates within 50-100ms.

The practical pattern for most SaaS products is a hybrid: batch-compute the heavy collaborative filtering models, but blend in real-time signals (current session behavior, time of day, device context) at serving time. This gives the feel of real-time personalization without the infrastructure cost of a fully real-time pipeline.

Comparing Vector Database Options

Your vector database is the storage and retrieval engine for embeddings. Choosing the right one depends on scale, operational complexity tolerance, and existing infrastructure.

Database	Type	Max Vectors	Latency (p99)	Best For	Pricing Model
Pinecone	Managed SaaS	Billions	<50ms	Teams wanting zero ops overhead; fast prototyping to production	Pay per pod/serverless usage
Weaviate	Open-source / Cloud	Billions	<100ms	Multimodal search (text + image); built-in ML module integrations	Free (self-hosted) or managed cloud
Qdrant	Open-source / Cloud	Billions	<50ms	High-performance filtering + vector search; Rust-based efficiency	Free (self-hosted) or managed cloud
pgvector	PostgreSQL extension	~10-50M practical	<200ms	Teams already on PostgreSQL; keeping stack simple; moderate scale	Free (part of PostgreSQL)

Our recommendation: Start with pgvector if you already run PostgreSQL and expect fewer than 10 million vectors. It avoids adding a new database to your stack and the performance is adequate for most early-stage SaaS products. When you outgrow it — either in vector count, query latency requirements, or advanced filtering needs — migrate to Qdrant or Pinecone. The migration is straightforward if your application accesses vectors through an abstraction layer rather than direct database queries.

Personalization Signals That Matter

The quality of recommendations depends on the signals you feed into the system. There are three categories of personalization signals, listed in order of reliability:

Behavioral Signals (Implicit)

These are derived from what users do, not what they say. They are the most reliable because actions are harder to fake than stated preferences:

• Click-through patterns: Which search results and recommendations users click on.
• Dwell time: How long users spend on content after clicking. Long dwell time indicates relevance; quick bounces indicate mismatch.
• Feature usage sequences: The order in which users navigate features reveals workflow patterns and unmet needs.
• Repeat visits: Content or features users return to frequently are strong positive signals.

Contextual Signals

These describe the circumstances of the current session:

• Time of day and day of week: A project manager may want dashboard reports on Monday mornings and detailed analytics on Friday afternoons.
• Device and platform: Mobile users often need different content density than desktop users.
• Account attributes: Company size, industry, plan tier, and feature entitlements shape what is relevant.
• Current task context: What page the user is on, what they were doing before the search.

Explicit Preferences

These are directly stated by users — saved searches, favorited items, notification settings, onboarding questionnaire answers. They are valuable but sparse: most users never configure preferences. Use explicit signals as strong overrides when available, but do not depend on them as your only personalization input.

"Explicit preferences tell you what users think they want. Behavioral signals tell you what they actually want. The best recommendation systems use explicit preferences as initialization and let behavioral signals refine over time."

— Xavier Amatriain, VP of Engineering, LinkedIn, RecSys Conference 2024

Solving the Cold-Start Problem

The cold-start problem occurs when your recommendation engine has insufficient data to make good suggestions. It manifests in three scenarios:

New users (user cold-start): A user signs up and you have zero behavioral data. Solutions:

• Ask 2-3 preference questions during onboarding (but keep it short — every additional question reduces completion rates by 10-15%).
• Use account-level attributes (industry, company size, role) to match against behavioral profiles of similar users.
• Default to popularity-based recommendations — "most used features" and "trending content" — until you accumulate 15-20 behavioral events.

New items (item cold-start): You publish new content or launch a new feature with zero interaction data. Solutions:

• Use content-based similarity to place the new item near related existing items in the embedding space.
• Apply an exploration boost — temporarily increase the new item's ranking score so it gets shown to a sample of users, generating initial interaction data.
• Leverage editorial metadata (tags, categories, difficulty level) to slot new items into existing recommendation clusters.

New system (system cold-start): You are launching the recommendation engine for the first time with no historical interaction data. Solutions:

• Start with content-based recommendations using your existing embeddings. This requires zero interaction data.
• Instrument comprehensive event tracking immediately and let the system accumulate data for 2-4 weeks before enabling collaborative filtering.
• Seed the system with implicit signals from existing analytics (page views, session data) if available.

Measuring Search and Recommendation Quality

You cannot improve what you do not measure. These are the metrics that matter for search and recommendation systems:

Metric	What It Measures	Target Range	When to Use
MRR (Mean Reciprocal Rank)	How high the first relevant result appears in the ranked list	0.3 - 0.6 for search	When users typically need a single best result (e.g., finding a specific document)
NDCG@K (Normalized Discounted Cumulative Gain)	Quality of the top-K results accounting for ranking position	0.4 - 0.7 for top 10	When multiple results are relevant and ranking order matters
CTR (Click-Through Rate)	Percentage of displayed results or recommendations that users click	15-35% for search, 2-8% for recommendations	Online metric for production monitoring; proxy for relevance
Zero-Result Rate	Percentage of searches that return no results	<5%	Identifying coverage gaps in your content or search index

Offline metrics (MRR, NDCG) are measured against labeled evaluation datasets. Build a golden set of 200-500 query-result pairs with human relevance judgments. Online metrics (CTR, zero-result rate) come from production instrumentation. The gap between offline and online metrics reveals how well your evaluation set represents real-world usage.

Run A/B tests when making significant changes to the search or recommendation algorithm. Even seemingly small changes — adjusting the weight between keyword and semantic scores, modifying the re-ranking formula — can produce measurable shifts in user behavior.

Implementation Roadmap: Basic to Advanced

Building search and recommendations is an iterative process. Here is a phased approach that delivers value at each stage:

Phase 1: Semantic Search (Weeks 1-3)

Replace or augment keyword search with hybrid search. Embed your existing content using an API-based model, store vectors in pgvector, and implement reciprocal rank fusion to merge BM25 and vector results. Instrument click-through tracking on all search results. This phase delivers immediate user-visible improvement — fewer zero-result searches and better results for natural-language queries.

Phase 2: Content-Based Recommendations (Weeks 4-6)

Use the embeddings from Phase 1 to power "related items" recommendations. On every content page, show the 5 nearest items in embedding space filtered by category and recency. Add a "recommended for you" section on the dashboard based on the user's recent engagement history. This requires no collaborative data and works from day one.

Phase 3: Behavioral Personalization (Weeks 7-12)

Build the event collection pipeline to capture clicks, dwell time, feature usage, and search queries. Implement user profile vectors — a rolling average of the embeddings of content each user has engaged with. Use these profiles to re-rank search results and recommendations for each user. This is where the system starts feeling personalized rather than generic.

Phase 4: Collaborative Filtering and Real-Time Signals (Weeks 13-20)

Once you have 2-3 months of behavioral data, add collaborative filtering as a candidate source alongside content-based retrieval. Implement a real-time feature store to blend session-level signals (current page, recent clicks) into the scoring model. Build the A/B testing infrastructure to measure the impact of each change. At this stage, your search and recommendation system becomes a competitive differentiator — the kind of capability that increases retention and makes switching costs real.

Each phase builds on the previous one, and each delivers measurable value to users. Resist the temptation to jump to Phase 4 without the instrumentation and data from earlier phases — collaborative filtering without sufficient behavioral data performs worse than simple content-based approaches.