yao/agent/output/message/STREAMING.md

# Message Streaming Architecture

This document explains the hierarchical streaming architecture for Agent/LLM/MCP message delivery.

## Overview

The streaming system uses a hierarchical structure to handle complex scenarios including:

- Single LLM calls with multiple message types (thinking, tool calls, text)
- Agent logic with multiple sequential operations (LLM → MCP → LLM)
- Concurrent/parallel calls to multiple LLMs or MCPs
- Real-time delta updates for streaming responses

## Hierarchical Structure

```
Agent Stream (entire conversation)
  └─ ThreadID (concurrent stream, optional: T1, T2, T3...)
      └─ BlockID (output block/section: B1, B2, B3...)
          └─ MessageID (logical message: M1, M2, M3...)
              └─ ChunkID (stream fragment: C1, C2, C3...)
```

## Field Definitions

### Message Struct Fields

```go
type Message struct {
    // Core fields
    Type  string                 `json:"type"`
    Props map[string]interface{} `json:"props,omitempty"`

    // Streaming control
    ChunkID   string `json:"chunk_id,omitempty"`
    MessageID string `json:"message_id,omitempty"`
    BlockID   string `json:"block_id,omitempty"`
    ThreadID  string `json:"thread_id,omitempty"`

    // Delta control
    Delta       bool   `json:"delta,omitempty"`
    DeltaPath   string `json:"delta_path,omitempty"`
    DeltaAction string `json:"delta_action,omitempty"`

    // ...
}
```

### Field Responsibilities

| Field       | Generated By         | Purpose                            | Example Values                       | Required                            |
| ----------- | -------------------- | ---------------------------------- | ------------------------------------ | ----------------------------------- |
| `ChunkID`   | System (auto)        | Deduplication, ordering, debugging | `C1`, `C2`, `C3`                     | Always                              |
| `MessageID` | LLM Provider/Handler | Delta merge target                 | `M1`, `M2`, `M3` or `thinking_msg_1` | Required for delta scenarios        |
| `BlockID`   | Agent Logic          | UI block/section rendering         | `B1`, `B2`, `B3` or `llm_response_1` | Required when Agent controls blocks |
| `ThreadID`  | Agent Logic          | Concurrent stream distinction      | `T1`, `T2`, `T3` or `thread_llm1`    | Optional (concurrent only)          |

### Detailed Field Explanation

#### ChunkID (Stream Fragment Identifier)

- **Purpose**: Uniquely identifies each chunk in the stream
- **Generated**: Automatically by the system (sequential: M1, M2, M3...)
- **Used For**:
  - Deduplication (prevent duplicate chunks)
  - Ordering (maintain correct sequence)
  - Debugging (trace message flow)
- **Scope**: Unique within entire Agent stream
- **Always Present**: Yes

**Example:**

```json
{"chunk_id": "C1", "type": "text", "props": {"content": "Hello"}}
{"chunk_id": "C2", "type": "text", "props": {"content": " World"}}
{"chunk_id": "C3", "type": "thinking", "props": {"content": "..."}}
```

#### MessageID (Logical Message Identifier)

- **Purpose**: Groups multiple chunks into one logical message via delta merging
- **Generated**: By LLM Provider or Stream Handler
- **Used For**:
  - Delta merge target (frontend merges all chunks with same MessageID)
  - Distinguishing different messages within a group
- **Scope**: Unique within a Group
- **Present When**: Delta streaming is used

**Example:**

```json
// Multiple chunks combine into one "thinking" message
{"chunk_id": "C1", "message_id": "M1", "type": "thinking", "props": {"content": "Let me"}, "delta": true}
{"chunk_id": "C2", "message_id": "M1", "type": "thinking", "props": {"content": " think"}, "delta": true}
{"chunk_id": "C3", "message_id": "M1", "type": "thinking", "props": {"content": "..."}, "delta": true}

// Another independent message
{"chunk_id": "C4", "message_id": "M2", "type": "text", "props": {"content": "Hello"}, "delta": true}
```

#### BlockID (Output Block Identifier)

- **Purpose**: Represents one output block/section (e.g., one LLM call, one MCP call)
- **Generated**: By Agent logic
- **Used For**:
  - Frontend UI block/section rendering (visual blocks)
  - Distinguishing different operations (LLM vs MCP vs custom logic)
  - Organizing related messages together
- **Scope**: Unique within entire Agent stream
- **Present When**: Agent explicitly controls output blocks

**Key Concept**: Block represents a semantic unit of work from Agent's perspective, NOT from LLM's perspective. Each block is rendered as a distinct UI section in the frontend.

**Example:**

```json
// BLOCK 1: LLM Response (contains thinking + tool_call + text)
{"chunk_id": "C1", "block_id": "B1", "message_id": "M1", "type": "thinking", ...}
{"chunk_id": "C2", "block_id": "B1", "message_id": "M2", "type": "tool_call", ...}
{"chunk_id": "C3", "block_id": "B1", "message_id": "M3", "type": "text", ...}

// BLOCK 2: MCP Call
{"chunk_id": "C4", "block_id": "B2", "message_id": "M4", "type": "loading", ...}
{"chunk_id": "C5", "block_id": "B2", "message_id": "M5", "type": "text", ...}

// BLOCK 3: Another LLM Response
{"chunk_id": "C6", "block_id": "B3", "message_id": "M6", "type": "text", ...}
```

#### ThreadID (Concurrent Stream Identifier)

- **Purpose**: Distinguishes concurrent/parallel output streams
- **Generated**: By Agent logic when spawning concurrent operations
- **Used For**:
  - Separating outputs from parallel LLM/MCP calls
  - Maintaining independent streaming contexts
- **Scope**: Unique within entire Agent stream
- **Present When**: Agent makes concurrent calls (optional)

**Example:**

```json
// Main thread
{"chunk_id": "C1", "thread_id": "T1", "block_id": "B1", "message_id": "M1", "type": "text", ...}

// Parallel MCP calls
{"chunk_id": "C2", "thread_id": "T2", "block_id": "B2", "message_id": "M2", "type": "text", ...}
{"chunk_id": "C3", "thread_id": "T3", "block_id": "B3", "message_id": "M3", "type": "text", ...}
```

## Usage Scenarios

### Scenario 1: Simple Text Message

**No streaming, no grouping**

```json
{
  "chunk_id": "C1",
  "type": "text",
  "props": { "content": "Hello World" }
}
```

**Fields Used:**

- `chunk_id`: C1 (auto-generated)
- No `message_id`, `block_id`, or `thread_id` needed

---

### Scenario 2: LLM Streaming Response (Single Message)

**LLM streams one text message**

```json
{"chunk_id": "C1", "message_id": "M1", "type": "text", "props": {"content": "Hello"}, "delta": true}
{"chunk_id": "C2", "message_id": "M1", "type": "text", "props": {"content": " World"}, "delta": true}
{"chunk_id": "C3", "message_id": "M1", "type": "text", "props": {"content": "!"}, "delta": true}
```

**Fields Used:**

- `chunk_id`: C1, C2, C3 (unique per chunk)
- `message_id`: M1 (same for all, merge target)
- `delta`: true

**Frontend Behavior:**

- Merge all chunks with `message_id: "M1"` into one message
- Display: "Hello World!"

---

### Scenario 3: Agent-Controlled LLM Call (One Block)

**Agent wraps LLM response in an output block**

```typescript
// Agent code starts a block for the LLM response
// System generates block_id: "B1"
// LLM returns thinking + tool_call + text
// Agent ends the block
```

```json
// LLM chunks within block B1
{"chunk_id": "C1", "message_id": "M1", "block_id": "B1", "type": "thinking", "props": {...}, "delta": true}
{"chunk_id": "C2", "message_id": "M1", "block_id": "B1", "type": "thinking", "props": {...}, "delta": true}

{"chunk_id": "C3", "message_id": "M2", "block_id": "B1", "type": "tool_call", "props": {...}}

{"chunk_id": "C4", "message_id": "M3", "block_id": "B1", "type": "text", "props": {...}, "delta": true}
{"chunk_id": "C5", "message_id": "M3", "block_id": "B1", "type": "text", "props": {...}, "delta": true}
```

**Fields Used:**

- `chunk_id`: C1~C5 (unique per chunk)
- `message_id`: M1, M2, M3 (per logical message)
- `block_id`: B1 (all belong to same LLM call)
- `delta`: true (for streaming messages)

**Frontend Behavior:**

- Render one block/section for `block_id: "B1"`
- Within this block, show 3 messages:
  - Thinking message (chunks C1+C2 merged into M1)
  - Tool call message (chunk C3 = M2)
  - Text message (chunks C4+C5 merged into M3)

---

### Scenario 4: Agent Sequential Operations (Multiple Blocks)

**Agent orchestrates: LLM → MCP → LLM**

```typescript
// Agent code orchestrates three sequential operations:
// 1. Block B1: First LLM call
// 2. Block B2: MCP call
// 3. Block B3: Second LLM call
```

```json
// BLOCK 1: First LLM call
{"chunk_id": "C1", "message_id": "M1", "block_id": "B1", "type": "text", ...}
{"chunk_id": "C2", "message_id": "M1", "block_id": "B1", "type": "text", ...}

// BLOCK 2: MCP call
{"chunk_id": "C3", "message_id": "M2", "block_id": "B2", "type": "loading", ...}
{"chunk_id": "C4", "message_id": "M3", "block_id": "B2", "type": "text", ...}

// BLOCK 3: Second LLM call
{"chunk_id": "C5", "message_id": "M4", "block_id": "B3", "type": "text", ...}
{"chunk_id": "C6", "message_id": "M4", "block_id": "B3", "type": "text", ...}
```

**Frontend Behavior:**

- Render 3 distinct blocks/sections:
  1. Block 1 (B1): LLM response with text
  2. Block 2 (B2): MCP call with loading + result
  3. Block 3 (B3): LLM response with text

---

### Scenario 5: Concurrent Operations (Blocks + Threads)

**Agent uses concurrent handler to make parallel calls**

```typescript
// Agent orchestrates parallel operations within one block (B1)
// The concurrent handler automatically assigns thread_id to each operation:
// - MCP call for weather (thread_id: "T1")
// - MCP call for news (thread_id: "T2")
// - LLM call for summary (thread_id: "T3")
//
// Messages from different threads may arrive in any order
```

```json
// Same block, different threads (may arrive in any order)
{"chunk_id": "C1", "message_id": "M1", "block_id": "B1", "thread_id": "T1", "type": "text", "props": {"content": "Weather: Sunny"}}
{"chunk_id": "C2", "message_id": "M2", "block_id": "B1", "thread_id": "T2", "type": "text", "props": {"content": "News: ..."}}
{"chunk_id": "C3", "message_id": "M1", "block_id": "B1", "thread_id": "T1", "type": "text", "props": {"content": ", 25°C"}}
{"chunk_id": "C4", "message_id": "M3", "block_id": "B1", "thread_id": "T3", "type": "text", "props": {"content": "Summary..."}}
```

**Fields Used:**

- `chunk_id`: C1, C2, C3, C4 (unique per chunk, chronological order)
- `message_id`: M1, M2, M3 (per operation/message)
- `block_id`: B1 (all belong to same parallel operation block)
- `thread_id`: T1, T2, T3 (distinguish concurrent operations)

**Frontend Behavior:**

- Render one block for `block_id: "B1"`
- Within this block, separate messages by `thread_id`:
  - Thread T1 (Weather): M1 (chunks C1+C3 merged) → "Weather: Sunny, 25°C"
  - Thread T2 (News): M2 (chunk C2)
  - Thread T3 (Summary): M3 (chunk C4)
- Or interleave by `chunk_id` order (C1, C2, C3, C4) to show real-time arrival

---

## Summary

| Field       | Level       | Purpose            | Example    |
| ----------- | ----------- | ------------------ | ---------- |
| `ChunkID`   | System      | Transport/debug    | C1, C2, C3 |
| `MessageID` | LLM/Handler | Delta merging      | M1, M2, M3 |
| `BlockID`   | Agent       | UI blocks/sections | B1, B2, B3 |
| `ThreadID`  | Agent       | Concurrency        | T1, T2, T3 |

**Key Insight**: Each field serves a distinct purpose at a specific layer of the architecture. This hierarchical design supports simple single-message scenarios while enabling complex Agent orchestration with concurrent operations. Blocks provide natural UI boundaries for organizing related messages.