Skip to main content

Pipeline Lifecycle & Flow

This document describes the complete request-response cycle of the message streaming pipeline, from user input to final message persistence.

Complete Lifecycle Stages​

1. Prepare Request State​

Location: pipeline.ts:33-119

Actions:

  • Reset all metric pills to initial state
  • Start performance timers
  • Set loading flags and disable input
  • Clear product cards and thinking indicators
  • Detect "show more" intent from user message
  • Build optimistic user message with timestamp
  • Create placeholder assistant message

Why: Ensures clean state for each request and provides immediate user feedback.

2. Build Payload​

Location: pipeline.ts:137-215

Actions:

  • Hydrate past conversation messages
  • Attach exclude IDs (previously shown products)
  • Include last search params for "show more"
  • Load optional precomputedContext from sessionStorage
  • Cap agent iterations to maxSteps: 5
  • Add conversation metadata

Why: Provides backend with full context for intelligent response generation.

3. Persist User Message​

Location: pipeline.ts:226-266

Actions:

  • Fire-and-forget persistence via saveMessage
  • Log failures without blocking UI
  • Store message with timestamp and role

Why: Ensures conversation history is saved even if streaming fails.

4. Stream Response​

Location: pipeline.ts:274-900

Actions:

  • POST to /api/chat with SSE
  • Enforce request timeout (30s) and stream timeout (45s)
  • Parse incoming events: text, product-metadata, smart-suggestions, tool-output, perf, done
  • Update UI/product state in real-time
  • Capture timings:
    • TTFC (Time to First Chunk)
    • Search duration
    • Pool metrics
    • Context extraction time
    • Rerank and diversity timings

Why: Provides real-time streaming experience with comprehensive metrics.

5. Finalize Assistant Message​

Location: pipeline.ts:902-1018

Actions:

  • Compute prompt metrics (tokens, model info)
  • Preserve context data and smart suggestions
  • Attach products and coupon information
  • Mark message as complete (streamingComplete: true)
  • Silent persistence without overwriting live message

Why: Ensures complete message with all metadata is saved for future reference.

Visual Flow Diagram​

Timing Breakdown​

Typical request timing profile:

User Input (t=0ms)
├─ Prepare State: 5-10ms
├─ Build Payload: 10-20ms
├─ Send Request: t=30ms
│
Backend Processing (30ms - 1200ms)
├─ Context Extraction: fast
├─ Search Pipeline: ~200-300ms
├─ LLM Generation starts: very fast
│
First Chunk Arrives (TTFC): t=800ms ✓
├─ Text streaming begins
├─ Product metadata parsed
├─ Context pills updated
│
Products Arrive: t=1400ms
├─ Tool output events
├─ Product cards displayed
├─ Skeletons filled
│
Stream Complete: t=2500ms
├─ Done event received
├─ Finalize message
├─ Silent persistence
└─ Cleanup: t=2600ms

State Transitions​

Key Variables Tracked​

Throughout the lifecycle, these variables maintain state:

// Timing
- firstEventTime: number | null
- searchDurationMs: number | null
- ttfcMs: number | null

// Content
- accumulatedText: string
- productCards: ProductCard[]
- contextData: ContextData | null

// Metrics
- contextExtractionTimeMs: number | null
- rerankTimeMs: number | null
- diversityTimeMs: number | null
- poolMetrics: PoolMetrics | null

// Flags
- streamingComplete: boolean
- toolsExpected: boolean
- isError: boolean

Error Recovery Points​

At each stage, the pipeline can gracefully handle failures:

  1. Prepare State: Invalid input → show validation error
  2. Build Payload: Missing context → use defaults
  3. Persist User: Save fails → log but continue
  4. Stream Response: Timeout → show error message and retry option
  5. Finalize: Persistence fails → keep UI state, retry silently