Popular Products Filter

The popular products filter enables users to request prioritized , trending, bestselling, or popular items within any product category. The system supports both Estonian and English queries, as well as mixed-language prompts.

Overview

When users search for "popular books" or "populaarseid mänge", the system:

1. Detects the popularity intent via LLM classification
2. Propagates the isPopularQuery flag through the pipeline
3. Filters products by in_popular_list === true in Convex

Language Support

Estonian Keywords

Keyword	Example Query
populaarseid	"Näita populaarseid raamatuid"
populaarne	"Populaarne kingitus"
populaarsemaid	"Otsin populaarsemaid mänge"
populaarsemad	"Kõige populaarsemad kokaraamatud"
top	"Top raamatud"
bestseller	"Bestseller krimi"
enim ostetud	"Enim ostetud pusled"
parimad müügid	"Parimad müügid mängudest"
kõige ostetumad	"Kõige ostetumad mängud"

English Keywords

Keyword	Example Query
popular	"Show me popular books"
bestseller	"Bestseller crime novels"
trending	"Trending games"
best selling	"Best selling puzzles"
top rated	"Top rated cookbooks"
most purchased	"Most purchased items"

Mixed Language

Mixed language queries work automatically:

Query	Detection
"Popular raamatud"	✅ English keyword detected
"Bestseller mängud"	✅ Both patterns match
"Trending pusled"	✅ English keyword detected

System Flow

1. LLM Detection

The fast classifier (app/api/chat/services/context-understanding/prompts.ts) includes explicit rules:

POPULAARSUSE TUVASTAMINE:
- "populaarseid", "populaarne", "top", "bestseller", "enim ostetud", "parimad müügid" → isPopularQuery: true
- "popular", "bestseller", "trending", "best selling", "top rated" → isPopularQuery: true
- Tavalised päringud ilma populaarsuse viiteta → isPopularQuery: false

2. Context Propagation

The isPopularQuery flag flows through multiple layers:

Key files:

• app/api/chat/services/context-understanding/fast-classifier.ts - LLM response parsing
• app/api/chat/services/context-understanding/classifier-context.ts - Context building
• app/api/chat/services/query-rewriting/index.ts - Query variation metadata

3. Query Rewriting

In query-rewriting/index.ts, the popular flag is propagated to all query variations:

// POPULAR QUERY FLAG PROPAGATION
if (context.isPopularQuery) {
  finalVariations.forEach(v => {
    v.metadata.isPopular = true;
  });
}

4. Product Search Service

The ProductSearchService.searchMulti() passes the flag to Convex:

// product-search.ts line 166
...(variation.metadata.isPopular ? { isPopular: true } : {}),

5. Convex Database Filter

Two locations apply the popular filter:

Text Search Path (convex/actions/searchProducts.ts)

// Lines 2287-2313
if (isPopular && !allowGiftAccessories && pool.length > 0) {
  const beforePopularFilter = pool.length;
  const popularPool = pool.filter(p => p.in_popular_list === true);
  
  // Graceful fallback if no popular products
  if (popularPool.length === 0) {
    console.warn('🌟 No popular products in pool, using all');
  } else {
    pool = popularPool;
    poolSource = 'popular';
  }
}

Category Search Path (convex/queries/getProduct.ts)

// Lines 195-211
if (csv_category) {
  let queryBuilder = ctx.db
    .query('products')
    .withIndex('by_csv_category', (q) => q.eq('csv_category', csv_category));

  if (applyPopularFilter) {
    // Use proper Convex filter builder syntax
    queryBuilder = queryBuilder.filter((q) => q.eq(q.field('in_popular_list'), true));
  }

  const results = await queryBuilder.take(effectiveLimit);
  return results;
}

Database Schema

The products table includes the in_popular_list field:

// convex/schema.ts
products: defineTable({
  // ... other fields
  in_popular_list: v.optional(v.boolean()),
  // ...
})
  .index("by_in_popular_list", ["in_popular_list"])

The index by_in_popular_list enables efficient queries when filtering by popularity without a category constraint.

Search Orchestrator Behavior

Book Category Path Bypass

When isPopularQuery === true, the optimized Book Category Path is bypassed:

// search-orchestrator.ts lines 524-525
const isBookCategoryPath = (isExplicitBookRoute || (hasBookProductType && hasBookCategory)) && 
  giftContext.isPopularQuery !== true;  // Popular queries use standard path

This ensures popular book queries go through the standard text search path where the isPopular flag is properly handled via ProductSearchService.searchMulti().

Category Search Path

For category-specific popular queries (via CategorySearchService):

// search-orchestrator.ts line 571
isPopular: giftContext.isPopularQuery === true,

Multi-Turn Preservation

The isPopularQuery flag is automatically preserved across follow-up turns, including "show more" requests and page reloads.

Preservation Flow

Key Files

File	Purpose
persist-context.ts	Saves isPopularQuery to Convex
fetch-stored-context.ts	Restores isPopularQuery after page reload
apply-preservation.ts	Merges flag from lastSearchParams OR storedGiftContext
setConversationContext.ts	Convex mutation with isPopularQuery field

Example Scenario

Turn 1: "Show me popular crime books"
  → isPopularQuery: true (from LLM)
  → Returns popular crime books
  → Persists isPopularQuery to Convex

Turn 2: "Show more"
  → Restores isPopularQuery: true from lastSearchParams
  → Returns MORE popular crime books (not just any crime books)

[Page Reload]

Turn 3: "Show more"
  → Restores isPopularQuery: true from storedGiftContext
  → Still returns only popular crime books ✅

Graceful Fallback

If no popular products exist for a category, the system falls back to showing all products:

This ensures users always see results, even if no products are marked as popular in that category.

Testing

A test script exists at scripts/test-popular-filter.ts to verify:

1. Database statistics (how many products are marked popular)
2. Filter effectiveness with/without the isPopular flag
3. API integration via quick prompt simulation

Run with:

npx tsx scripts/test-popular-filter.ts

Bug Fix History

December 2024: Convex Filter Syntax Fix

Issue: The popular filter returned 0 results when combined with a category filter.

Root Cause: Incorrect Convex filter syntax in getProduct.ts:

// ❌ WRONG - JavaScript arrow function doesn't work with Convex filter
queryBuilder = queryBuilder.filter((p: any) => p.in_popular_list === true);

Fix: Use proper Convex filter builder syntax:

// ✅ CORRECT - Convex filter builder
queryBuilder = queryBuilder.filter((q) => q.eq(q.field('in_popular_list'), true));

Impact:

• Before: "Popular crime books" returned 0 products (broken)
• After: "Popular crime books" correctly returns only popular crime books

Related Documentation

• Multi-Query Retrieval - How query variations work
• Search Orchestrator Paths - Different search routing
• Query Routing & Detection - Intent classification