Routing Pipeline

The routing pipeline is the intelligent entry point that analyzes every user query and directs it to the most appropriate search strategy. It combines deterministic keyword detection with LLM-based semantic classification to ensure optimal performance and relevance.

Overview

Every query flows through a hybrid routing system that makes split-second decisions about which search path will deliver the best results.

Entry Point & Flow

Request Lifecycle

Key Files:

• Entry: app/api/chat/route.ts
• Orchestration: app/api/chat/orchestrators/parallel-orchestrator.ts
• Context: app/api/chat/orchestrators/context-orchestrator/orchestrate.ts (lines 1-240)
• Routing: app/api/chat/orchestrators/context-orchestrator/llm-query-router.ts (lines 200-360)

Hybrid Routing System

The routing system uses two layers of classification, with deterministic rules taking precedence over LLM decisions.

Layer 1: Deterministic Keyword Detection

Purpose: Guarantee fast paths for explicit product requests without LLM latency.

Implementation: app/api/chat/services/specific-product-detector.ts (lines 1-190)

Detected Categories:

• Gift bags & packaging
• Wrapping paper
• Gift cards
• Puzzles & games
• Greeting cards
• Books by specific criteria
• And 20+ more commodity categories

Benefits:

• Zero LLM latency (~10ms vs ~200ms)
• 100% deterministic (no model variance)
• Always honors user's exact request

Layer 2: LLM Query Router

Purpose: Semantic understanding for nuanced or ambiguous queries.

Implementation: app/api/chat/orchestrators/context-orchestrator/llm-query-router.ts (lines 1-180)

LLM Prompt Structure:

{
  query: "User's message",
  conversationHistory: [...],
  availableRoutes: [
    "SPECIFIC_PRODUCT_FAST_PATH",
    "AUTHOR_SEARCH_PATH", 
    "GIFT_OCCASION_PATH",
    "CLARIFICATION_PATH"
  ],
  instructions: "Classify the query intent..."
}

Response Format:

{
  route: "GIFT_OCCASION_PATH",
  confidence: 0.95,
  reason: "User asking for birthday gift recommendation",
  detectedKeywords: ["birthday", "gift", "mother"],
  authorName?: "J.K. Rowling" // If author route
}

Priority Rules

Both layers enforce the same priority hierarchy:

Rule: Even if LLM says "gift" but keywords match, deterministic detector wins.

Gift Context Guard

Critical Exception

When a gift context is detected alongside hobby/product keywords, the Gift Context Guard prevents the specific product fast path from triggering incorrectly.

Problem Solved:

Without this guard, a query like:

"Kingitus 50-aastasele naisele, kes armastab aiandust"
(Gift for a 50-year-old woman who loves gardening)

Would route to SPECIFIC_PRODUCT_FAST_PATH because "aiandus" (gardening) matches the book category "Aiandusraamatud" (Gardening books). This returns gardening books instead of gardening-related gifts.

Implementation: llm-query-router.ts (lines 232-310)

Gift Context Patterns:

const GIFT_CONTEXT_PATTERNS = [
  /\b(kingitus|kingitust|gift|present)\b/i,
  /\b(sünnipäev|birthday|emadepäev|mother|isadepäev|father)\b/i,
  /\b(jõul|christmas|valentin|valentine|wedding|pulm)\b/i,
  /\b(lapsele|emale|isale|sõbrale|kolleegile|õpetajale|naisele|mehele)\b/i,
  /\d+\s*-?\s*aastase/i  // Age pattern like "50-aastasele"
];

Exceptions (still use fast path):

• Gift bags: kinkekott, kingikott
• Gift cards: kinkekaart, gift card
• Packaging: pakkepaber, pakend

Result:

Query	Without Guard	With Guard
"Kingitus naisele, kes armastab aiandust"	Gardening books	Gardening-related gifts
"Sünnipäevakingitus lapsele, kes armastab puslesid"	Puzzle category	Child-appropriate gifts
"Kinkekott ja pakkepaber"	✅ Gift bags	✅ Gift bags (exception)

Decision Logic Visualization

Real-World Example Flow

Route-Specific Context Construction

Once a route is determined, the Context Orchestrator builds a tailored GiftContext object for that path.

1. Specific Product Fast Path

Goal: Minimal context construction for maximum speed.

Implementation: orchestrate.ts (lines 70-120)

Characteristics:

• No LLM extraction call
• Pre-filled categories from detector
• Intent fixed to product_inquiry
• Latency: ~10ms context construction

2. Clarification Path

Goal: Skip search entirely for invalid queries.

Implementation: orchestrate.ts (lines 120-150)

Triggers:

• Query length < 3 characters
• Keyboard mashing (e.g., "asdfghjkl")
• Only special characters
• Random gibberish

3. Author Search Path

Goal: Comprehensive author catalog without gift filtering.

Implementation: orchestrate.ts (lines 200-220)

Special Handling:

• Categories cleared to avoid filtering (lines 320-340)
• Bypasses multi-query search
• No gift-context extraction needed
• Prioritizes comprehensiveness over relevance

4. Gift/Occasion Path (Default)

Goal: Full semantic understanding for personalized recommendations.

Implementation: orchestrate.ts (lines 150-205)

Deterministic Helpers:

1. Occasion Detection (lines 30-90)

   • Patches missing occasions from text
   • Recognizes: "sünnipäev", "jõulud", "valentine"

2. Keyword Category Extraction (lines 220-250)

   • Maps product keywords to categories
   • Example: "kruus" → "Mugs & Drinkware"

3. Smart Hint Prioritization (lines 250-270)

   • Reorders categories by relevance
   • Birthday + teacher → Educational gifts first

4. Book Category Preservation (book-query-detector.ts lines 1-200)

   • Maintains narrow book categories
   • Only when explicitly requested
   • Prevents gift queries from book-locking

Show More & Context Preservation

The routing system preserves context across follow-up queries.

Implementation:

• Show more detection: orchestrate.ts (lines 340-420)
• Specific context check: query-router.ts (lines 175-205)

Preservation Logic:

if (isShowMoreRequest && hasSpecificProductContext) {
  // Keep deterministic path
  preserveRoute = 'SPECIFIC_PRODUCT_FAST_PATH';
  preserveCategory = previousContext.categoryHints;
} else {
  // Allow context switching for ambiguous follow-ups
  runFullRouting();
}

Search Orchestrator Integration

The meta.route field acts as a contract between routing and search execution.

Implementation: search-orchestrator.ts

Branch Logic:

// Author path
if (meta.route === 'AUTHOR_SEARCH_PATH') {
  return await AuthorSearchService.searchByAuthor(...);
}

// Fast path
if (meta.route === 'SPECIFIC_PRODUCT_FAST_PATH') {
  return await singleVariationSearch(...);
}

// Clarification
if (meta.route === 'CLARIFICATION_PATH') {
  return { products: [], reason: 'clarification_needed' };
}

// Default: Full pipeline
return await multiQuerySearchWithFunnel(...);

Fallback & Recovery

The routing system includes multiple safety nets for edge cases.

Query Validation

Implementation: product-search-handler.ts (lines 202-220)

Fallback Acknowledgement

Implementation: product-search-handler.ts (lines 624-689)

Memory Skip for Fast Paths

Implementation: parallel-orchestrator.ts (lines 486-510)

Reasoning: Memory might contain products from different categories, but deterministic routing must honor exact category requests.

Graceful Degradation

The system handles failures at every layer.

Fallback Chain:

1. Deterministic detection
2. LLM classification
3. Default to GIFT_OCCASION_PATH
4. Parse errors → GIFT_OCCASION_PATH
5. LLM timeout → GIFT_OCCASION_PATH
6. Empty results → Fallback search
7. Still empty → Clarification

Routing Telemetry

Every routing decision is logged for monitoring and debugging.

Metrics Captured:

{
  routingDurationMs: 145,
  routeDecision: "SPECIFIC_PRODUCT_FAST_PATH",
  routeSource: "deterministic_keyword",
  confidence: 1.0,
  
  deterministicMatch: {
    keyword: "kinkekott",
    category: "Kinkekotid ja -pakendid",
    productType: "gift_bags"
  },
  
  llmAttempted: false,
  llmDurationMs: 0,
  
  contextSwitchDetected: false,
  showMoreRequest: false,
  
  warnings: []
}

Implementation: llm-query-router.ts (lines 180-214)

Configuration

Location: app/api/chat/orchestrators/context-orchestrator/llm-query-router.ts

Routing Settings

export const ROUTING_CONFIG = {
  // Enable deterministic keyword detection
  ENABLE_DETERMINISTIC_DETECTION: true,
  
  // Enable LLM router fallback
  ENABLE_LLM_ROUTER: true,
  
  // LLM model for routing
  ROUTING_MODEL: 'llama-3.1-70b-versatile',
  
  // Confidence threshold for LLM routes
  MIN_CONFIDENCE: 0.6,
  
  // Timeout for LLM routing
  ROUTING_TIMEOUT_MS: 3000,
  
  // Default route on failure
  FALLBACK_ROUTE: 'GIFT_OCCASION_PATH',
  
  // Priority order
  ROUTE_PRIORITY: [
    'SPECIFIC_PRODUCT_FAST_PATH',
    'AUTHOR_SEARCH_PATH',
    'GIFT_OCCASION_PATH',
    'CLARIFICATION_PATH'
  ]
};

Key Takeaways

Hybrid Approach

The routing system combines the best of both worlds:

• Deterministic: Fast, reliable, 100% consistent
• LLM-based: Semantic, flexible, handles nuance

Performance Optimization

Different routes have different performance profiles:

• Specific Product: ~150ms total (10ms routing + 140ms search)
• Author: ~200ms total (10ms routing + 190ms search)
• Gift/Occasion: ~500ms total (200ms routing + 300ms search)

Contract-Based Architecture

meta.route serves as a contract:

• Set during routing
• Honored by search orchestrator
• Tracked through fallbacks
• Logged for telemetry

Resilient by Design

Multiple fallback layers ensure graceful degradation:

• Deterministic → LLM → Default route
• Parse errors → Safe defaults
• Empty results → Broader categories
• Still empty → Clarification

Context Preservation

Smart memory management:

• Show more preserves route and category
• Specific products skip memory to honor deterministic categories
• Context switch detection prevents incorrect preservation

Related Documentation

• Search Orchestrator Paths - How routes execute
• Query Routing & Detection - Detailed routing logic
• Context Extraction - Gift context building
• Parallel Orchestrator - Request coordination

---

File References:

• Context Orchestrator: app/api/chat/orchestrators/context-orchestrator/orchestrate.ts
• LLM Router: app/api/chat/orchestrators/context-orchestrator/llm-query-router.ts
• Specific Product Detector: app/api/chat/services/specific-product-detector.ts
• Book Query Detector: app/api/chat/utils/book-query-detector.ts
• Search Orchestrator: app/api/chat/orchestrators/search-orchestrator.ts
• Product Search Handler: app/api/chat/handlers/product-search-handler.ts
• Parallel Orchestrator: app/api/chat/orchestrators/parallel-orchestrator.ts