Context Signals Detection

Overview

The Context Signals Detection System (context-signals.ts) is a critical utility module that determines whether a user query has meaningful, actionable context or just generic fallback defaults. This distinction is essential for deciding whether to execute a product search, ask clarifying questions, or adjust confidence scores.

Location: app/api/chat/utils/context-signals.ts

Purpose & Why We Need It

The Core Problem

When a user sends a vague query like "I need a gift", the LLM might extract:

{
  productType: "Kingitused",  // Generic "Gifts"
  productTypeHints: ["Kodu ja aed", "Ilu ja stiil"],  // Generic hints
  confidence: 0.8  // Appears confident!
}

The system needs to answer: Is this a meaningful signal from the user, or just fallback defaults that the LLM fills in when uncertain?

Why This Matters

Without signal detection:

Vague queries trigger confident searches
System searches for generic "Gifts" instead of asking clarifying questions
Poor results, bad UX

With signal detection:

Recognizes fallback defaults as "no real signal"
Lowers confidence appropriately
Triggers clarifying questions for truly vague queries
Better results, better UX

Core Functions

1. `hasMeaningfulProductSignals(context)`

Purpose: Determines if the context contains real, actionable product information beyond generic fallbacks.

Returns: boolean

true = User has meaningful product intent (search products)
false = Only generic fallbacks (ask clarifying question)

Algorithm:

Code:

export function hasMeaningfulProductSignals(context: GiftContext): boolean {
  const hasExplicitType = typeof context.productType === 'string' 
    && context.productType.trim().length > 0;
  const hasExplicitCategory = typeof context.category === 'string' 
    && context.category.trim().length > 0;
  const hasTypeHints = hasValues(context.productTypeHints);
  const hasCategoryHints = hasValues(context.categoryHints);

  // Check 1: Do we have ANY signals?
  if (!hasExplicitType && !hasExplicitCategory && !hasTypeHints && !hasCategoryHints) {
    return false;  // No signals at all
  }

  // Check 2: Are they only fallback defaults?
  return !hasFallbackGiftSignalsOnly(context);
}

2. `hasFallbackGiftSignalsOnly(context)`

Purpose: Determines if the context contains only generic fallback defaults that the LLM uses when it doesn't have specific information.

Returns: boolean

true = Context only has generic fallbacks
false = Context has custom/specific signals

Fallback Defaults:

Type	Values
Product Type	`"Kingitused"` (Gifts)
Product Type Hints	`["Kingitused", "Kodu ja aed", "Ilu ja stiil", "Joodav ja söödav", "Kontorikaup", "Mängud", "Tehnika"]`
Category Hints	`["Kodukaubad", "Kodu ja aed", "Reisi- ja matkakaubad", "Sport ja harrastused", "Joodav ja söödav", "Ilu ja stiil"]`

Detection Logic:

Code:

export function hasFallbackGiftSignalsOnly(context: GiftContext): boolean {
  const productType = normalize(context.productType);
  
  // Check product type hints for non-fallback values
  const productTypeHints = Array.isArray(context.productTypeHints)
    ? context.productTypeHints.filter(hint => 
        typeof hint === 'string' && hint.trim().length > 0
      )
    : [];
  const hasCustomHint = productTypeHints.some(
    hint => !FALLBACK_HINT_SET.has(hint.trim().toLowerCase())
  );
  
  // Check category hints for non-fallback values
  const categoryHints = Array.isArray(context.categoryHints)
    ? context.categoryHints.filter(hint => 
        typeof hint === 'string' && hint.trim().length > 0
      )
    : [];
  const hasNonFallbackCategoryHint = categoryHints.some(
    hint => !FALLBACK_CATEGORY_HINT_SET.has(hint.trim().toLowerCase())
  );
  
  // Check explicit category
  const hasExplicitCategory = typeof context.category === 'string' 
    && context.category.trim().length > 0;
  const hasNonFallbackExplicitCategory = hasExplicitCategory &&
    !FALLBACK_CATEGORY_HINT_SET.has(context.category!.trim().toLowerCase());

  // If ANY non-fallback signal exists, return false
  if (hasCustomHint || hasNonFallbackCategoryHint || hasNonFallbackExplicitCategory) {
    return false;
  }

  // If no productType, it's a fallback
  if (!productType) {
    return true;
  }

  // Check if productType is the default fallback
  return productType === DEFAULT_PRODUCT_TYPE; // "kingitused"
}

3. `isGiftContextMissing(context)`

Purpose: Checks if gift-specific context (recipient or occasion) is missing.

Returns: boolean

true = No recipient or occasion mentioned
false = Has recipient and/or occasion

Code:

export function isGiftContextMissing(context: GiftContext): boolean {
  return !context.recipient && !context.occasion;
}

Examples:

Query	Recipient	Occasion	Result
"Gift for mother"	`"ema"`	-	`false`
"Valentine day gift"	-	`"valentinipäev"`	`false`
"Gift for mother's birthday"	`"ema"`	`"sünnipäev"`	`false`
"Show me gifts"	-	-	`true`

4. `isBudgetMissing(context)`

Purpose: Checks if budget information is missing.

Returns: boolean

true = No budget info
false = Has min, max, or hint

Code:

export function isBudgetMissing(context: GiftContext): boolean {
  const budget = context.budget;
  if (!budget) return true;

  const hasMin = typeof budget.min === 'number' && Number.isFinite(budget.min);
  const hasMax = typeof budget.max === 'number' && Number.isFinite(budget.max);
  const hasHint = typeof budget.hint === 'string' && budget.hint.trim().length > 0;

  return !hasMin && !hasMax && !hasHint;
}

Examples:

Query	Budget	Result
"Gift under 50 euro"	`{ max: 50 }`	`false`
"Between 20-100 euro"	`{ min: 20, max: 100 }`	`false`
"Affordable gift"	`{ hint: "affordable" }`	`false`
"Show me gifts"	`undefined`	`true`

How It's Used Throughout the System

1. Confidence Scoring (`context-normalizer.ts`)

Purpose: Adjusts confidence based on signal count

Algorithm:

Code Reference: app/api/chat/services/context-understanding/context-normalizer.ts:55-81

const hasProductSignals = hasMeaningfulProductSignals(context);
const giftContextMissing = isGiftContextMissing(context);
const budgetMissing = isBudgetMissing(context);

// Count real signals
const signalCount = [
  hasProductSignals,
  !giftContextMissing,
  !budgetMissing
].filter(Boolean).length;

if (signalCount === 0) {
  // No signals - very low confidence
  resolvedConfidence = Math.min(resolvedConfidence, 0.3);
} else if (signalCount === 1) {
  // Only one signal - still low confidence
  resolvedConfidence = Math.min(resolvedConfidence, 0.55);
}
// 2+ signals: keep original confidence (likely 0.7+)

2. Clarifying Questions (`clarifying-question-handler.ts`)

Purpose: Decides whether to ask clarifying questions instead of searching

Flow:

Code Reference: app/api/chat/handlers/clarifying-question-handler.ts:60-128

export function shouldAskClarifyingQuestion(
  giftContext: GiftContext,
  intentResult: IntentResult,
  debug: boolean
): boolean {
  if (!isClarifyingQuestionsEnabled()) return false;

  const hasLowConfidence = (giftContext.confidence ?? 0) < 0.7;
  const hasProductSignals = hasMeaningfulProductSignals(giftContext);
  const hasOccasionOrRecipient = !isGiftContextMissing(giftContext);

  // CRITICAL: Even fallback types like "Kingitused" count if explicitly present
  const hasExplicitProductType = typeof giftContext.productType === 'string' 
    && giftContext.productType.trim().length > 0;
  const hasExplicitCategory = typeof giftContext.category === 'string' 
    && giftContext.category.trim().length > 0;

  // Always search if user explicitly mentioned a type/category
  if (hasExplicitProductType || hasExplicitCategory) {
    return false;
  }

  // Ask clarifying question if: low confidence + no product signals + no gift context
  if (hasLowConfidence && !hasProductSignals && !hasOccasionOrRecipient) {
    return true;
  }

  return false;
}

3. Parallel Orchestrator (`parallel-orchestrator.ts`)

Purpose: Determines if query is too vague to search

Flow:

Code Reference: app/api/chat/orchestrators/parallel-orchestrator.ts:237-256

const hasMeaningfulSignals = hasMeaningfulProductSignals(contextResult.giftContext);
const giftContextMissing = isGiftContextMissing(contextResult.giftContext);

const hasExplicitProductType = typeof contextResult.giftContext.productType === 'string' 
  && contextResult.giftContext.productType.trim().length > 0;
const hasExplicitCategory = typeof contextResult.giftContext.category === 'string' 
  && contextResult.giftContext.category.trim().length > 0;

let isVagueGiftQuery = isProductSearch && 
                       hasLowConfidence &&
                       !hasMeaningfulSignals &&
                       giftContextMissing;

// If user explicitly requested a product type or category, NEVER treat as vague
if (hasExplicitProductType || hasExplicitCategory) {
  isVagueGiftQuery = false;
}

Complete Example Walkthroughs

Example 1: Vague Query → Clarifying Question

User: "I need a gift"

Context Extracted:

{
  intent: "product_search",
  productType: "Kingitused",  // Fallback default
  productTypeHints: ["Kodu ja aed", "Ilu ja stiil"],  // Fallback hints
  recipient: undefined,
  occasion: undefined,
  budget: undefined,
  confidence: 0.8  // LLM appears confident
}

Signal Analysis:

Result: Clarifying question asked (correct behavior)

Example 2: Specific Query → Direct Search

User: "Show me gift cards"

Context Extracted:

{
  intent: "product_search",
  productType: "Kinkekaart",  // Specific type
  productTypeHints: [],
  category: undefined,
  recipient: undefined,
  occasion: undefined,
  budget: undefined,
  confidence: 0.85
}

Signal Analysis:

Result: Direct search executed (correct behavior)

Example 3: Mixed Query → Search with Context

User: "Valentine day gifts under 100 euro"

Context Extracted:

{
  intent: "valentines_day_gift",
  occasion: "valentinipäev",  // Explicit occasion
  productType: "Kingitused",  // Fallback
  productTypeHints: ["Raamat", "Kodu ja aed", "Ilu ja stiil"],  // Mixed
  budget: { max: 100 },
  recipient: undefined,
  confidence: 0.7
}

Signal Analysis:

Result: Multi-category search executed (correct behavior)

⚠️ Critical Design Considerations

1. The Fallback Paradox

Problem: What if a user explicitly requests a fallback type?

Example: "Show me gifts" (user wants generic gifts category)

Context:

{
  productType: "Kingitused",  // Is this fallback or explicit request?
}

Solution: The system uses additional explicit checks:

// CRITICAL FIX in clarifying-question-handler.ts and parallel-orchestrator.ts
const hasExplicitProductType = typeof giftContext.productType === 'string' 
  && giftContext.productType.trim().length > 0;

// If productType exists (even if it's "Kingitused"), user requested it explicitly
if (hasExplicitProductType) {
  // ALWAYS search, never ask clarifying question
  return false;
}

This distinguishes:

Empty/undefined productType → LLM couldn't extract anything → vague
"Kingitused" present → User explicitly wants gifts → search

2. Signal Hierarchy

Not all signals are equal:

Priority	Signal Type	Weight	Example
🥇 High	Specific product type	Critical	`"Kinkekaart"`, `"Raamat"`
🥇 High	Non-fallback category	Critical	`"Krimi ja põnevus"`
🥈 Medium	Gift context	Important	`recipient: "ema"`
🥈 Medium	Budget	Important	`{ max: 50 }`
🥉 Low	Fallback hints	Minor	`["Kingitused", "Kodu ja aed"]`

Design: System prioritizes specific product signals over gift context when deciding to search.

3. Three-Stage Detection

Stage 1: hasMeaningfulProductSignals - Any data at all?
Stage 2: hasFallbackGiftSignalsOnly - Real or just defaults?
Stage 3: Count signal types (product + gift + budget) - How confident?

🔧 Key Implementation Files

1. Signal Detection Core

File: app/api/chat/utils/context-signals.ts
Lines: 1-86
Functions:

hasFallbackGiftSignalsOnly(context) - Detects fallback-only contexts
hasMeaningfulProductSignals(context) - Detects meaningful signals
isGiftContextMissing(context) - Checks gift context
isBudgetMissing(context) - Checks budget info

2. Fallback Defaults Configuration

File: app/api/chat/orchestrators/context-orchestrator/gift-defaults.ts
Lines: 1-29
Constants:

DEFAULT_GIFT_PRODUCT_TYPE = "Kingitused"
DEFAULT_GIFT_HINTS = 7 generic product types
FALLBACK_GIFT_CATEGORY_HINTS = 6 generic categories

3. Confidence Normalization

File: app/api/chat/services/context-understanding/context-normalizer.ts
Lines: 48-95
Function: normalizeContext(context, userMessage)
Uses: All 4 signal detection functions to adjust confidence

4. Clarifying Question Logic

File: app/api/chat/handlers/clarifying-question-handler.ts
Lines: 60-128
Function: shouldAskClarifyingQuestion(giftContext, intentResult, debug)
Uses: hasMeaningfulProductSignals, isGiftContextMissing

5. Orchestration Logic

File: app/api/chat/orchestrators/parallel-orchestrator.ts
Lines: 237-256
Uses: All signal functions to determine vague query handling

Signal Detection Truth Table

Product Signals	Gift Context	Budget	Signal Count	Confidence	Action
Fallback only	Missing	Missing	0	0.3	Ask Question
Fallback only	Present	Missing	1	0.55	Maybe Search
Fallback only	Present	Present	2	0.7+	Search
Specific type	Missing	Missing	1	0.55	Search
Specific type	Present	Missing	2	0.7+	Search
Specific type	Present	Present	3	0.7+	Search

Testing Signal Detection

Test Case 1: Completely Vague

Input: "test"

Expected Context:

{
  intent: "unknown",
  productType: undefined,
  productTypeHints: undefined,
  recipient: undefined,
  occasion: undefined,
  budget: undefined
}

Expected Results:

hasMeaningfulProductSignals() → false
isGiftContextMissing() → true
isBudgetMissing() → true
Signal count: 0
Confidence: 0.3
Action: Ask clarifying question

Test Case 2: Fallback Only

Input: "I need a gift"

Expected Context:

{
  intent: "product_search",
  productType: "Kingitused",  // Fallback
  productTypeHints: ["Kodu ja aed", "Ilu ja stiil"],  // Fallbacks
  recipient: undefined,
  occasion: undefined,
  budget: undefined
}

Expected Results:

hasFallbackGiftSignalsOnly() → true
hasMeaningfulProductSignals() → false
isGiftContextMissing() → true
isBudgetMissing() → true
Signal count: 0
Confidence: 0.3
Action: Ask clarifying question

Test Case 3: Specific Product

Input: "Show me gift cards"

Expected Context:

{
  intent: "product_search",
  productType: "Kinkekaart",  // Specific, not fallback
  productTypeHints: [],
  recipient: undefined,
  occasion: undefined,
  budget: undefined
}

Expected Results:

hasFallbackGiftSignalsOnly() → false
hasMeaningfulProductSignals() → true
isGiftContextMissing() → true
isBudgetMissing() → true
Signal count: 1
Confidence: 0.55
Action: Execute search

Test Case 4: Full Context

Input: "Gift for mother's birthday under 50 euro"

Expected Context:

{
  intent: "birthday_gift",
  productType: "Kingitused",  // Fallback, but...
  productTypeHints: ["Raamat", "Kodu ja aed"],  // Has non-fallback "Raamat"
  recipient: "ema",
  occasion: "sünnipäev",
  budget: { max: 50 }
}

Expected Results:

hasFallbackGiftSignalsOnly() → false ("Raamat" is not fallback)
hasMeaningfulProductSignals() → true
isGiftContextMissing() → false
isBudgetMissing() → false
Signal count: 3
Confidence: 0.7+
Action: Execute multi-category search

Configuration

Fallback Defaults

Location: app/api/chat/orchestrators/context-orchestrator/gift-defaults.ts

// Default product type when LLM is uncertain
export const DEFAULT_GIFT_PRODUCT_TYPE = 'Kingitused';

// Generic product type hints (fallbacks)
export const DEFAULT_GIFT_HINTS: readonly string[] = [
  'Kingitused',      // Generic gifts
  'Kodu ja aed',     // Home & garden
  'Ilu ja stiil',    // Beauty & style
  'Joodav ja söödav',// Food & drink
  'Kontorikaup',     // Office supplies
  'Mängud',          // Games
  'Tehnika'          // Tech
];

// Generic category hints (fallbacks)
export const FALLBACK_GIFT_CATEGORY_HINTS: readonly string[] = [
  'Kodukaubad',
  'Kodu ja aed',
  'Reisi- ja matkakaubad',
  'Sport ja harrastused',
  'Joodav ja söödav',
  'Ilu ja stiil'
];

Best Practices

When Adding New Product Types

Decide if it's a fallback or specific type:
- Fallback: Generic, used when uncertain (add to DEFAULT_GIFT_HINTS)
- Specific: User-requested category (don't add to fallbacks)

Example - Adding "Jewelry":

// Specific type - do NOT add to fallbacks
// Let signal detection treat it as meaningful

Example - Adding "Decorations":

// Generic fallback - add to DEFAULT_GIFT_HINTS
export const DEFAULT_GIFT_HINTS = [
  ...existing,
  'Dekoratsioonid'  // Decorations
];

When Tuning Signal Detection

Make detection more strict (more clarifying questions):

// Increase signal threshold
const signalCount = [
  hasProductSignals,
  !giftContextMissing,
  !budgetMissing
].filter(Boolean).length;

if (signalCount < 2) {  // Changed from < 1
  resolvedConfidence = 0.3;
}

Make detection more lenient (fewer clarifying questions):

// Accept fallbacks as meaningful
export function hasMeaningfulProductSignals(context: GiftContext): boolean {
  const hasExplicitType = /* ... */;
  
  // Accept any type, even fallbacks
  return hasExplicitType || hasExplicitCategory || hasTypeHints || hasCategoryHints;
}

🐛 Debugging

Enable Debug Logging

export CHAT_DEBUG_LOGS=true

Output:

 SIGNAL ANALYSIS: {
  hasMeaningfulProductSignals: false,
  hasFallbackGiftSignalsOnly: true,
  isGiftContextMissing: true,
  isBudgetMissing: true,
  signalCount: 0,
  adjustedConfidence: 0.3,
  action: "ask_clarifying_question"
}

Common Issues

Issue 1: "Specific query gets clarifying question"

Symptom: User says "Show me books" but system asks "What type of gift?"

Diagnosis:

Check if "Raamat" is in DEFAULT_GIFT_HINTS (it shouldn't be)
Verify hasFallbackGiftSignalsOnly() returns false for "Raamat"

Issue 2: "Vague query triggers search"

Symptom: User says "gift" and system searches instead of asking

Diagnosis:

Check if explicit type/category override is triggering
Verify confidence adjustment is working (signalCount === 0 → confidence = 0.3)

Query Specificity Detection - Query type classification
Context Extraction - How context is extracted from queries
Intent Classification - Intent taxonomy and routing
Fast Classifier - Pre-classification before extraction

Overview​

Purpose & Why We Need It​

The Core Problem​

Why This Matters​

Core Functions​

1. hasMeaningfulProductSignals(context)​

2. hasFallbackGiftSignalsOnly(context)​

3. isGiftContextMissing(context)​

4. isBudgetMissing(context)​

How It's Used Throughout the System​

1. Confidence Scoring (context-normalizer.ts)​

2. Clarifying Questions (clarifying-question-handler.ts)​

3. Parallel Orchestrator (parallel-orchestrator.ts)​

Complete Example Walkthroughs​

Example 1: Vague Query → Clarifying Question​

Example 2: Specific Query → Direct Search​

Example 3: Mixed Query → Search with Context​

⚠️ Critical Design Considerations​

1. The Fallback Paradox​

2. Signal Hierarchy​

3. Three-Stage Detection​

🔧 Key Implementation Files​

1. Signal Detection Core​

2. Fallback Defaults Configuration​

3. Confidence Normalization​

4. Clarifying Question Logic​

5. Orchestration Logic​

Signal Detection Truth Table​

Testing Signal Detection​

Test Case 1: Completely Vague​

Test Case 2: Fallback Only​

Test Case 3: Specific Product​

Test Case 4: Full Context​

Configuration​

Fallback Defaults​

Best Practices​

When Adding New Product Types​

When Tuning Signal Detection​

🐛 Debugging​

Enable Debug Logging​

Common Issues​

Issue 1: "Specific query gets clarifying question"​

Issue 2: "Vague query triggers search"​

Related Documentation​

Overview

Purpose & Why We Need It

The Core Problem

Why This Matters

Core Functions

1. `hasMeaningfulProductSignals(context)`

2. `hasFallbackGiftSignalsOnly(context)`

3. `isGiftContextMissing(context)`

4. `isBudgetMissing(context)`

How It's Used Throughout the System

1. Confidence Scoring (`context-normalizer.ts`)

2. Clarifying Questions (`clarifying-question-handler.ts`)

3. Parallel Orchestrator (`parallel-orchestrator.ts`)

Complete Example Walkthroughs

Example 1: Vague Query → Clarifying Question

Example 2: Specific Query → Direct Search

Example 3: Mixed Query → Search with Context

⚠️ Critical Design Considerations

1. The Fallback Paradox

2. Signal Hierarchy

3. Three-Stage Detection

🔧 Key Implementation Files

1. Signal Detection Core

2. Fallback Defaults Configuration

3. Confidence Normalization

4. Clarifying Question Logic

5. Orchestration Logic

Signal Detection Truth Table

Testing Signal Detection

Test Case 1: Completely Vague

Test Case 2: Fallback Only

Test Case 3: Specific Product

Test Case 4: Full Context

Configuration

Fallback Defaults

Best Practices

When Adding New Product Types

When Tuning Signal Detection

🐛 Debugging

Enable Debug Logging

Common Issues

Issue 1: "Specific query gets clarifying question"

Issue 2: "Vague query triggers search"

Related Documentation