yao/agent/output

Output Module

The output module provides a unified API for sending messages to different client types (CUI, OpenAI-compatible, etc.) with support for streaming and rich media content.

Architecture

agent/output/
├── message/              # Core types and interfaces (no dependencies)
│   ├── types.go         # Message, Group, Props structures
│   └── interfaces.go    # Writer, Adapter, Factory interfaces
├── adapters/            # Client-specific adapters
│   ├── cui/             # CUI adapter (native DSL)
│   │   ├── adapter.go
│   │   └── writer.go
│   └── openai/          # OpenAI adapter (converts to OpenAI format)
│       ├── adapter.go
│       ├── converter.go
│       ├── writer.go
│       ├── types.go
│       └── factory.go
├── output.go            # Main API (Send, GetWriter, etc.)
├── builtin.go           # Helper functions for built-in types
└── BUILTIN_TYPES.md     # Documentation for built-in types

DSL Structure

Message Structure

The universal message DSL is a JSON structure that supports streaming, rich media, and incremental updates:

type Message struct {
    // Core fields
    Type  string                 `json:"type"`            // Message type (e.g., "text", "image", "action")
    Props map[string]interface{} `json:"props,omitempty"` // Type-specific properties

    // Streaming control
    ID    string `json:"id,omitempty"`    // Unique message ID (for merging in streaming)
    Delta bool   `json:"delta,omitempty"` // Whether this is an incremental update
    Done  bool   `json:"done,omitempty"`  // Whether the message is complete

    // Delta update control (for incremental props updates)
    DeltaPath   string `json:"delta_path,omitempty"`   // Which field to update (e.g., "content", "items.0.name")
    DeltaAction string `json:"delta_action,omitempty"` // How to update ("append", "replace", "merge", "set")

    // Type correction (for streaming type inference)
    TypeChange bool `json:"type_change,omitempty"` // Marks this as a type correction message

    // Message grouping (for semantically related messages)
    GroupID    string `json:"group_id,omitempty"`    // Parent message group ID
    GroupStart bool   `json:"group_start,omitempty"` // Marks the start of a group
    GroupEnd   bool   `json:"group_end,omitempty"`   // Marks the end of a group

    // Metadata
    Metadata *Metadata `json:"metadata,omitempty"` // Timestamp, sequence, trace ID
}

Field Descriptions

Core Fields

• Type (required): Determines how the message should be rendered

  • Built-in types: text, thinking, loading, tool_call, error, image, audio, video, action, event
  • Custom types: Any string (frontend must have corresponding component)

• Props (optional): Type-specific properties passed to the rendering component

  • For text: {"content": "Hello"}
  • For image: {"url": "...", "alt": "..."}
  • For custom types: Any JSON-serializable data

Streaming Control

• ID (optional): Unique identifier for message tracking

  • Used to merge multiple delta updates into a single message
  • Auto-generated if not provided
  • Example: "msg_1234567890_9876543210"

• Delta (optional): Marks this as an incremental update

  • true: Append/update to existing message with same ID
  • false: Complete message (default)
  • Used for streaming LLM responses

• Done (optional): Marks message as complete

  • true: No more updates will come for this message ID
  • false: More updates may follow
  • Typically sent as final message in a delta sequence

Delta Update Control

For complex, structured messages that need field-level updates:

• DeltaPath (optional): JSON path to the field being updated

  • Simple: "content" (updates props.content)
  • Nested: "user.name" (updates props.user.name)
  • Array: "items.0.title" (updates props.items[0].title)

• DeltaAction (optional): How to apply the delta update

  • "append": Concatenate to existing string/array
  • "replace": Replace entire value
  • "merge": Merge objects (shallow merge)
  • "set": Set new field (if doesn't exist)

Type Correction

• TypeChange (optional): Indicates message type was corrected
  • Used when initial type inference was wrong
  • Frontend should re-render with new type
  • Example: Initially sent as text, corrected to thinking

Message Grouping

For grouping semantically related messages (e.g., image + caption):

• GroupID (optional): Identifier for the message group
• GroupStart (optional): Marks the beginning of a group
• GroupEnd (optional): Marks the end of a group

Metadata

• Metadata (optional): Additional message metadata

  type Metadata struct {
      Timestamp int64  // Unix nanoseconds
      Sequence  int    // Message sequence number
      TraceID   string // For debugging/logging
  }
  

Message Examples

Simple Text Message

{
  "type": "text",
  "props": {
    "content": "Hello, world!"
  }
}

Streaming Text (Delta Updates)

// First chunk
{
  "id": "msg_123",
  "type": "text",
  "delta": true,
  "props": {
    "content": "Hello"
  }
}

// Second chunk (appends)
{
  "id": "msg_123",
  "type": "text",
  "delta": true,
  "props": {
    "content": ", world"
  }
}

// Final chunk (marks done)
{
  "id": "msg_123",
  "type": "text",
  "delta": true,
  "done": true,
  "props": {
    "content": "!"
  }
}

Complex Type with Nested Updates

// Initial message
{
  "id": "msg_456",
  "type": "table",
  "props": {
    "columns": ["Name", "Age"],
    "rows": []
  }
}

// Add first row
{
  "id": "msg_456",
  "type": "table",
  "delta": true,
  "delta_path": "rows",
  "delta_action": "append",
  "props": {
    "rows": [{"name": "Alice", "age": 30}]
  }
}

// Add second row
{
  "id": "msg_456",
  "type": "table",
  "delta": true,
  "delta_path": "rows",
  "delta_action": "append",
  "props": {
    "rows": [{"name": "Bob", "age": 25}]
  }
}

Type Correction

// Initial guess (text)
{
  "id": "msg_789",
  "type": "text",
  "delta": true,
  "props": {
    "content": "Let me think..."
  }
}

// Correction (actually thinking)
{
  "id": "msg_789",
  "type": "thinking",
  "type_change": true,
  "props": {
    "content": "Let me think..."
  }
}

Message Group

// Group start
{
  "group_id": "grp_001",
  "group_start": true
}

// Image in group
{
  "type": "image",
  "group_id": "grp_001",
  "props": {
    "url": "https://example.com/photo.jpg",
    "alt": "Beautiful sunset"
  }
}

// Caption in group
{
  "type": "text",
  "group_id": "grp_001",
  "props": {
    "content": "Captured at Golden Gate Bridge"
  }
}

// Group end
{
  "group_id": "grp_001",
  "group_end": true
}

Key Design Decisions

1. Separate message Package

To avoid circular dependencies, all core types and interfaces are defined in the message sub-package:

• message.Message - Universal message DSL
• message.Writer - Interface for writing messages
• message.Adapter - Interface for format conversion

This allows:

• handlers → output → message ✅
• output/adapters → message ✅
• No circular dependencies!

2. Adapter Pattern

Different clients require different formats:

CUI Clients:

{
  "type": "text",
  "props": { "content": "Hello" }
}

OpenAI Clients:

{
  "choices": [
    {
      "delta": { "content": "Hello" }
    }
  ]
}

Adapters handle the transformation automatically based on ctx.Accept.

3. Built-in Types

10 standardized message types with defined Props structures:

Type	Purpose	CUI	OpenAI
text	Text content	Direct	delta.content
thinking	LLM reasoning	Direct	delta.reasoning_content
loading	Progress indicator	Direct	delta.reasoning_content
tool_call	Function calls	Direct	delta.tool_calls
error	Error messages	Direct	error
image	Images	Render	 markdown
audio	Audio	Player	Link
video	Video	Player	Link
action	System commands	Execute	Silent
event	Lifecycle events	Track	Conditional (stream_start converted to link)

Usage

Basic Usage

import (
    "github.com/yaoapp/yao/agent/output"
    "github.com/yaoapp/yao/agent/output/message"
)

// Send a text message
msg := output.NewTextMessage("Hello world")
output.Send(ctx, msg)

// Send a loading indicator
loading := output.NewLoadingMessage("Searching knowledge base...")
output.Send(ctx, loading)

// Send an image
img := output.NewImageMessage("https://example.com/image.jpg", "Description")
output.Send(ctx, img)

// Send an error
err := output.NewErrorMessage("Connection failed", "TIMEOUT")
output.Send(ctx, err)

Streaming Messages

// Send delta (incremental) updates
msg := &message.Message{
    ID:    "msg_123",
    Type:  message.TypeText,
    Delta: true,  // Incremental update
    Props: map[string]interface{}{
        "content": "Hello",
    },
}
output.Send(ctx, msg)

// Mark as complete
msg.Delta = false
msg.Done = true
msg.Props["content"] = "Hello world!"  // Full content
output.Send(ctx, msg)

Custom Writers

// Register a custom writer factory
factory := &MyCustomFactory{}
output.SetWriterFactory(factory)

// Now all calls to output.Send will use your custom writer

Integration with Handlers

The handlers package uses the output module for streaming:

func DefaultStreamHandler(ctx *context.Context) context.StreamFunc {
    return func(chunkType context.StreamChunkType, data []byte) int {
        switch chunkType {
        case context.ChunkText:
            msg := output.NewTextMessage(string(data))
            output.Send(ctx, msg)
        case context.ChunkThinking:
            msg := output.NewThinkingMessage(string(data))
            output.Send(ctx, msg)
        // ... handle other types
        }
        return 0 // Continue
    }
}

Context-based Routing

The output module automatically selects the right writer based on ctx.Accept:

ctx.Accept	Writer	Format
standard	OpenAI	OpenAI-compatible SSE
cui-web	CUI	Universal DSL JSON
cui-native	CUI	Universal DSL JSON
cui-desktop	CUI	Universal DSL JSON

Writer Caching

Writers are cached per context to avoid recreating them:

// Get or create writer (cached)
writer := output.GetWriter(ctx)

// Clear cache when done
output.Close(ctx)  // Also closes the writer

See Also

• BUILTIN_TYPES.md - Complete documentation of built-in message types
• adapters/openai/README.md - OpenAI adapter documentation