Message Streaming Pipeline Overview

The useMessageStreaming pipeline is the thin, composable layer that turns a user's input into a streamed assistant response with products, metrics, and persisted context. It lives under app/chat/hooks/useMessageStreaming/pipeline.ts and is orchestrated by useMessageStreaming in index.ts.

Purpose

The pipeline serves as the bridge between the user interface and the backend orchestration system, handling:

• Request preparation and payload construction
• Server-Sent Events (SSE) streaming
• Real-time UI state updates
• Product and metric management
• Context preservation across conversations
• Error handling and resilience

Architecture Position

Core Responsibilities

1. Request Management

• Build and validate payloads
• Attach conversation context
• Manage exclude lists for "show more" functionality
• Include last search parameters

2. Streaming Control

• Establish SSE connections
• Parse incoming events
• Handle timeouts and errors
• Manage request lifecycle

3. State Synchronization

• Update UI in real-time
• Manage product cards
• Display metrics and context pills
• Control skeleton loading states

4. Persistence

• Save user messages (fire-and-forget)
• Persist assistant responses with metadata
• Store context data for future requests

Key Files

app/chat/hooks/useMessageStreaming/
├── index.ts                    # Main hook orchestrator
├── pipeline.ts                 # Core pipeline logic
├── parser.ts                   # SSE event parsing
└── config.ts                   # Configuration constants

Integration Points

• Context System: Receives and preserves contextData for pills
• Product Management: Handles product cards and skeletons
• Metrics System: Captures and displays performance metrics
• Smart Suggestions: Processes and displays followup actions
• Error Handling: Manages timeouts and fallback states

Performance Targets

Metric	Target	Purpose
TTFC (Time to First Chunk)	<800ms	User sees immediate response
Request Timeout	30s	Catch hung requests
Inactivity Timeout	45s	Handle stalled streams
Skeleton Delay	160 chars or 800ms	Show product placeholders

Related Documentation

• Lifecycle & Flow - Detailed request/response cycle
• Models & AI - LLM configuration by phase
• Event Handling - SSE event types and processing
• Resilience - Error handling and timeouts
• Extension Guide - How to add new features