- Removed the job system integration from the robot execution flow, transitioning to a dedicated ExecutionStore for managing execution records. - Updated the design documentation to reflect the new architecture, emphasizing the relationship between robots and concurrent executions. - Revised the API to return execution IDs instead of job IDs, ensuring clarity in execution tracking. - Enhanced logging mechanisms to utilize the kun/log package for better traceability of execution phases. - Updated tests and documentation to align with the removal of job-related structures and the introduction of execution management.
57 KiB
Robot Agent - Implementation TODO
Based on DESIGN.md and TECHNICAL.md Test environment:
source yao/env.local.shTest assistants:yao-dev-app/assistants/robot/
Workflow: Human-AI Collaboration
Important: Follow this workflow strictly for each sub-task.
┌─────────────────────────────────────────────────────────────────┐
│ Implementation Workflow │
├─────────────────────────────────────────────────────────────────┤
│ 1. AI: Implement code for current sub-task │
│ 2. AI: Present code for review (DO NOT write tests yet) │
│ 3. Human: Review code, provide feedback │
│ 4. AI: Iterate based on feedback │
│ 5. Human: Confirm "LGTM" or "Approved" │
│ 6. AI: Write tests for the approved code │
│ 7. Human: Review tests │
│ 8. AI: Run tests, fix if needed │
│ 9. Human: Confirm sub-task complete, move to next │
└─────────────────────────────────────────────────────────────────┘
Rules:
| Rule | Description |
|---|---|
| One sub-task at a time | Focus only on current sub-task |
| No tests before approval | Wait for human "LGTM" before writing tests |
| No jumping ahead | Do not implement future phases |
| Ask if unclear | When in doubt, ask before proceeding |
Core Principle
- Phase 1-2: Types + Skeleton (code compiles)
- Phase 3: Complete scheduling system (Cache + Pool + Trigger + Dedup + Job), executor is stub
- Phase 4-9: Implement executor phases one by one (P0 → P5)
- Phase 10: API completion, end-to-end tests
- Monitoring: Provided by Job system, no separate implementation
Phase 1: Types & Interfaces ✅
Goal: Define all types, enums, interfaces. No logic, no external deps.
Status: Complete - 88.4% test coverage, all tests passing
1.1 Enums (types/enums.go)
Phase- execution phases (inspiration, goals, tasks, run, delivery, learning)ClockMode- clock trigger modes (times, interval, daemon)TriggerType- trigger sources (clock, human, event)ExecStatus- execution status (pending, running, completed, failed, cancelled)RobotStatus- robot status (idle, working, paused, error, maintenance)InterventionAction- human actions (task.add, goal.adjust, etc.)Priority- priority levels (high, normal, low)DeliveryType- delivery types (email, webhook, process, notify)DedupResult- dedup results (skip, merge, proceed)EventSource- event sources (webhook, database)LearningType- learning types (execution, feedback, insight)TaskSource- task sources (auto, human, event)ExecutorType- executor types (assistant, mcp, process)TaskStatus- task status (pending, running, completed, failed, skipped, cancelled)InsertPosition- insert positions (first, last, next, at)
1.2 Context (types/context.go)
Contextstruct - robot execution contextNewContext()- constructorUserID(),TeamID()- helper methods
1.3 Config Types (types/config.go)
Config- main config structTriggers,TriggerSwitch- trigger enable/disableClock- clock config with validationIdentity- role, duties, rulesQuota- concurrency limits with defaultsKB,DB- knowledge base and database configLearn- learning configResources,MCPConfig- available agents and toolsDelivery- output delivery configEvent- event trigger config
1.4 Core Types (types/robot.go)
Robotstruct - runtime robot representationRobotmethods -CanRun(),RunningCount(),AddExecution(),RemoveExecution(),GetExecution(),GetExecutions()Executionstruct - single execution instanceTriggerInput- stored trigger inputCurrentState- current executing stateGoals- P1 output (markdown)Task- planned task (structured)TaskResult- task execution resultDeliveryResult- delivery outputLearningEntry- knowledge to save
1.5 Clock Context (types/clock.go)
ClockContextstruct - time context for P0NewClockContext()- constructor
1.6 Inspiration (types/inspiration.go)
InspirationReportstruct - P0 output
1.7 Request/Response (types/request.go)
InterveneRequest- human intervention requestEventRequest- event trigger requestExecutionResult- trigger resultRobotState- robot status query result
1.8 Interfaces (types/interfaces.go)
ManagerinterfaceExecutorinterfacePoolinterfaceCacheinterfaceDedupinterfaceStoreinterface
1.9 Errors (types/errors.go)
- Config errors
- Runtime errors
- Phase errors
1.10 Tests
types/enums_test.go- enum validationtypes/config_test.go- config validationtypes/clock_test.go- clock context creationtypes/robot_test.go- robot methods
Phase 2: Skeleton Implementation ✅
Goal: Create all packages with empty/stub implementations. Code compiles.
Status: Complete - All packages compile successfully, no circular dependencies
2.1 Utils (utils/) ✅
utils/convert.go- JSON, map, struct conversions (implement)utils/time.go- time parsing, formatting, timezone (implement)utils/id.go- ID generation (nanoid) (implement)utils/validate.go- validation helpers (implement)- Test:
utils/utils_test.go
2.2 Package Skeletons ✅ (stubs only, implemented in Phase 3)
Create empty structs and stub methods that return nil/empty/success:
cache/cache.go- Cache struct, stub methodsdedup/dedup.go- Dedup struct, stub methodsstore/store.go- Store struct, stub methodspool/pool.go- Pool struct, stub methodsplan/plan.go- Plan struct, stub methodstrigger/trigger.go- trigger dispatcher stubexecutor/executor.go- Executor struct, stubExecute()manager/manager.go- Manager struct, stub methods
2.3 API Skeletons ✅
api/api.go- Go API facade (all function signatures, return errors)api/process.go- Yao Process registration (all processes, return errors)api/jsapi.go- JSAPI registration (all methods, return errors)
2.4 Root ✅
robot.go- package entryInit()- placeholderShutdown()- placeholder
2.5 Compile Test ✅
- All packages compile without errors
- All imports resolve correctly
- No circular dependencies
Phase 3: Complete Scheduling System ✅
Goal: Implement complete scheduling system. Executor is stub (simulates success).
Status: Complete - All 7 sub-tasks done, 80+ integration tests passing
This phase delivers a fully working scheduling pipeline:
Trigger → Manager → Cache → Dedup → Pool → Worker → Executor(stub) → Job
✅ 3.1 Cache Implementation (COMPLETE)
cache/cache.go- Cache struct with thread-safe mapcache/load.go- load robots from__yao.memberwheremember_type='robot'andautonomous_mode=true- Implemented pagination (100 robots per page)
- Configurable model name via
SetMemberModel()
cache/refresh.go- refresh single robot, periodic full refresh (every hour)- Test: load/refresh with real DB
- Created comprehensive integration tests with real database
- Tests cover Load, LoadByID, Refresh, ListByTeam, GetByStatus
- All tests passing with proper cleanup
✅ 3.2 Pool Implementation (COMPLETE)
pool/pool.go- worker pool with configurable size (global limit)- Default config: 10 workers, 100 queue size
- Configurable via
pool.NewWithConfig()
pool/queue.go- priority queue (sorted by: robot priority, trigger type, wait time)- Two-level limit: global queue + per-robot queue
- Priority: Robot Priority × 1000 + Trigger Priority × 100
pool/worker.go- worker goroutines, dispatch to executor- Non-blocking quota check with re-enqueue
- Graceful shutdown support
- Test: submit jobs, verify execution order, verify concurrency limits
- 15 test cases covering all edge cases
- All tests passing
✅ 3.3 Manager Implementation (COMPLETE)
Note: Manager is the scheduling core, depends on completed Cache and Pool.
manager/manager.go- Manager structStart()- load cache, start pool, start ticker goroutineStop()- graceful shutdown (wait for running, drain queue)Tick()- main loop:- Get all cached robots
- For each robot with clock trigger enabled
- Check if should execute (times/interval/daemon modes)
- Submit to pool
TriggerManual()- manual trigger for testing/API- Clock modes: times, interval, daemon
- Day matching for times mode
- Timezone handling
- Skip paused/error/maintenance robots
- Test: manager start/stop, tick cycle, manual trigger, clock modes, goroutine leak
✅ 3.4 Trigger Implementation (COMPLETE)
trigger/trigger.go- validation and helper functionsValidateIntervention()- validate human intervention requestsValidateEvent()- validate event trigger requestsBuildEventInput()- build TriggerInput from event requestGetActionCategory()/GetActionDescription()- action helpers
trigger/clock.go- ClockMatcher for clock trigger matchingtimesmode: match specific times (09:00, 14:00)intervalmode: run every X duration (30m, 1h)daemonmode: restart immediately after completion- Timezone handling
- Day-of-week filtering
trigger/control.go- ExecutionController for pause/resume/stop- Track/Untrack executions
- Pause/Resume execution
- Stop execution (cancel context)
- WaitIfPaused() for executor integration
manager/manager.go- integrated trigger handlingIntervene()- human intervention handlerHandleEvent()- event trigger handlerPauseExecution()/ResumeExecution()/StopExecution()ListExecutions()/ListExecutionsByMember()
- Tests:
trigger/trigger_test.go,trigger/clock_test.go,trigger/control_test.go- Validation tests for intervention and event requests
- Clock matching tests for all modes
- ExecutionController lifecycle tests
- Manager integration tests for Intervene/HandleEvent
✅ 3.5 Execution Storage (COMPLETE)
- ExecutionStore - execution record persistence
- Execution data stored in
__yao.agent_executiontable - All phase outputs (Inspiration, Goals, Tasks, Results, Delivery, Learning)
- Status and phase tracking
- Logging via
kun/logpackage - Localization support (en-US, zh-CN)
- Execution data stored in
- Test: execution storage, status tracking
store/execution_test.go- execution store tests- All tests passing with real database
✅ 3.6 Executor Architecture (COMPLETE)
Pluggable executor architecture with multiple execution modes:
executor/
├── types/
│ ├── types.go # Executor interface, Config types
│ └── helpers.go # Shared helper functions
├── standard/
│ ├── executor.go # Real Agent execution (production)
│ ├── agent.go # AgentCaller for LLM calls
│ ├── input.go # InputFormatter for prompts
│ ├── inspiration.go # P0: Inspiration phase
│ ├── goals.go # P1: Goals phase
│ ├── tasks.go # P2: Tasks phase
│ ├── run.go # P3: Run phase
│ ├── delivery.go # P4: Delivery phase
│ └── learning.go # P5: Learning phase
├── dryrun/
│ └── executor.go # Simulated execution (testing/demo)
├── sandbox/
│ └── executor.go # Container-isolated (NOT IMPLEMENTED)
└── executor.go # Factory functions
Execution Modes:
| Mode | Use Case | Status |
|---|---|---|
| Standard | Production with real Agent calls | ✅ Implemented |
| DryRun | Tests, demos, scheduling tests | ✅ Implemented |
| Sandbox | Container-isolated execution | ⬜ Not Implemented |
⚠️ Sandbox Mode: Requires container-level isolation (Docker/gVisor/Firecracker) for true security. Current placeholder behaves like DryRun. Future feature.
executor/types/types.go-Executorinterface,PhaseExecutorinterfaceexecutor/types/helpers.go-BuildTriggerInput()shared helperexecutor/executor.go- Factory functions (New,NewDryRun,NewWithMode)executor/standard/executor.go- Real execution with Job integrationexecutor/standard/phases.go- Phase implementations (P0-P5)executor/dryrun/executor.go- Simulated execution with callbacksexecutor/sandbox/executor.go- Placeholder (NOT IMPLEMENTED)- Manager integration - accepts
Executorinterface via config - Tests use DryRun mode for scheduling/concurrency tests
3.7 Integration Test (End-to-End Scheduling) ✅
- Create test robot in
__yao.memberwith clock config - Start manager
- Wait for clock trigger
- Verify:
- Robot loaded to cache
- Clock trigger matched
- Job submitted to pool
- Worker picked up job
- Executor stub called
- Job execution recorded
- Logs written
- Test human intervention trigger
- Test event trigger
- Test concurrent executions (multiple robots)
- Test quota enforcement (per-robot limit)
- Test pause/resume/stop
Test Files Created:
manager/integration_test.go- Core scheduling flow (Cache→Pool→Executor)manager/integration_clock_test.go- Clock trigger modes (times/interval/daemon)manager/integration_human_test.go- Human intervention trigger testsmanager/integration_event_test.go- Event trigger testsmanager/integration_concurrent_test.go- Concurrent execution & quota testsmanager/integration_control_test.go- Pause/Resume/Stop tests
Test Coverage:
- 27 top-level test functions
- 80+ sub-tests covering all verification points
- 3x run stability verified
Phase 4: Agent Call Infrastructure ✅
Goal: Implement unified Agent/Assistant calling mechanism. This is the foundation for all phase implementations (P0-P5).
Architecture Note:
- Prompt construction is handled by Assistant layer (
prompts.ymlin each assistant) - Executor only prepares input data (ClockContext, InspirationReport, etc.) and calls Assistant
- Assistant framework handles prompt rendering, LLM API calls, streaming
Implemented:
- A unified way to call assistants with streaming support
- Input data formatting for each phase
- Response parsing (markdown and structured data via
gou/text) - Multi-turn conversation support
4.1 Agent Caller Implementation ✅
executor/agent.go-AgentCallerstruct withSkipOutput,SkipHistory,SkipSearch,ChatIDexecutor/agent.go-Call(ctx, assistantID, messages)- basic call with full responseexecutor/agent.go-CallWithMessages(ctx, assistantID, userContent)- convenience methodexecutor/agent.go-CallWithSystemAndUser(ctx, assistantID, systemContent, userContent)executor/agent.go- handle assistant not found errorexecutor/agent.go- handle LLM API errors gracefullyexecutor/agent.go-CallResult.GetJSON()/GetJSONArray()- parse JSON response usinggou/textexecutor/agent.go-Conversationstruct for multi-turn dialoguesexecutor/agent.go-Conversation.Turn(),RunUntil(),Reset(),WithSystemPrompt()executor/agent.go- Useagentcontext.Noop()logger to suppress debug output
4.2 Input Formatters ✅
executor/input.go-FormatClockContext(clockCtx, robot)- format clock context as message contentexecutor/input.go-FormatInspirationReport(report)- format P0 output for P1 inputexecutor/input.go-FormatTriggerInput(input)- format Human/Event trigger for P1 inputexecutor/input.go-FormatGoals(goals, robot)- format P1 output for P2 inputexecutor/input.go-FormatTasks(tasks)- format P2 output for P3 inputexecutor/input.go-FormatTaskResults(results)- format P3 output for P4/P5 inputexecutor/input.go-FormatExecutionSummary(exec)- format full execution for P5 inputexecutor/input.go-BuildMessages(),BuildMessagesWithSystem()- helper methods
4.3 Test Assistants ✅
yao-dev-app/assistants/tests/robot-single/- Single-turn test assistantyao-dev-app/assistants/tests/robot-conversation/- Multi-turn conversation test assistant
4.4 Tests ✅
executor/agent_test.go- 22 test cases for AgentCaller and Conversationexecutor/input_test.go- 20 test cases for InputFormatter- Verify: assistant can be called and returns response
- Verify: multi-turn conversation maintains state
- Verify: input data is well-formatted for assistant prompts
- Verify: JSON/YAML extraction from LLM output works correctly
Phase 5: Test Scenario & Assistants Setup ✅
Goal: Create realistic test scenarios with all required assistants.
Architecture:
┌─────────────────────────────────────────────────────────────────────────────┐
│ 6 Generic Phase Agents (P0-P5) │
├─────────────────────────────────────────────────────────────────────────────┤
│ inspiration │ goals │ tasks │ validation │ delivery │ learning │
│ (P0) │ (P1) │ (P2) │ (P3) │ (P4) │ (P5) │
└───────────────┴─────────┴─────────┴──────────────┴────────────┴─────────────┘
↓ P2 assigns tasks to
┌─────────────────────────────────────────────────────────────────────────────┐
│ Expert Agents (Task Executors) │
├─────────────────────────────────────────────────────────────────────────────┤
│ text-writer │ web-reader │ data-analyst │ summarizer │ ... │
│ (Generate) │ (Fetch URL) │ (Analyze) │ (Summarize) │ │
└───────────────┴──────────────┴────────────────┴──────────────┴─────────────┘
Test Strategy:
- Phase Agents (P0-P5) are generic and reusable across all robot types
- Expert Agents are specialized for specific tasks (text, web, data, etc.)
- Each P0-P5 test uses different expert combinations to cover real scenarios
- Tests use
interval: 1sorTriggerManual()for easy triggering (no time dependency)
5.1 Directory Structure
yao-dev-app/assistants/
├── robot/ # Generic Phase Agents
│ ├── inspiration/ # P0: Analyze clock context, generate insights
│ │ ├── package.yao
│ │ └── prompts.yml
│ ├── goals/ # P1: Generate prioritized goals
│ │ ├── package.yao
│ │ └── prompts.yml
│ ├── tasks/ # P2: Split goals into executable tasks
│ │ ├── package.yao
│ │ └── prompts.yml
│ ├── validation/ # P3: Validate task results
│ │ ├── package.yao
│ │ └── prompts.yml
│ ├── delivery/ # P4: Format and deliver results
│ │ ├── package.yao
│ │ └── prompts.yml
│ └── learning/ # P5: Summarize execution, extract insights
│ ├── package.yao
│ └── prompts.yml
│
└── experts/ # Expert Agents (Task Executors)
├── text-writer/ # Generate text content (reports, emails, summaries)
│ ├── package.yao
│ └── prompts.yml
├── web-reader/ # Fetch and parse web page content
│ ├── package.yao
│ └── prompts.yml
├── data-analyst/ # Analyze data, generate insights
│ ├── package.yao
│ └── prompts.yml
└── summarizer/ # Summarize long text into key points
├── package.yao
└── prompts.yml
5.2 Generic Phase Agents
5.2.1 Inspiration Agent (P0)
robot/inspiration/package.yao- config with model, temperaturerobot/inspiration/prompts.yml- system prompt:- Input: Clock context (time, day, markers), robot identity
- Output: Markdown report with Summary, Highlights, Opportunities, Risks
- Style: Analytical, context-aware
5.2.2 Goals Agent (P1)
robot/goals/package.yao- configrobot/goals/prompts.yml- system prompt:- Input: Inspiration report OR trigger input (human/event)
- Output: Prioritized goals in markdown (High/Normal/Low)
- Style: Strategic, actionable
5.2.3 Tasks Agent (P2)
robot/tasks/package.yao- configrobot/tasks/prompts.yml- system prompt:- Input: Goals, available expert agents list
- Output: Structured task list (JSON) with executor assignments
- Style: Detailed, executable
5.2.4 Validation Agent (P3)
robot/validation/package.yao- configrobot/validation/prompts.yml- system prompt:- Input: Task result, expected outcome
- Output: Validation result (pass/fail, issues, suggestions)
- Style: Critical, thorough
5.2.5 Delivery Agent (P4)
robot/delivery/package.yao- configrobot/delivery/prompts.yml- system prompt:- Input: Full execution context (P0-P3 results)
- Output: Formatted delivery content
- Style: Clear, professional
5.2.6 Learning Agent (P5)
robot/learning/package.yao- configrobot/learning/prompts.yml- system prompt:- Input: Full execution summary
- Output: Insights, patterns, improvement suggestions
- Style: Reflective, insightful
5.3 Expert Agents (Task Executors)
5.3.1 Text Writer
experts/text-writer/package.yao- configexperts/text-writer/prompts.yml- system prompt:- Input: Topic, key points, style (formal/casual), length
- Output: Generated text content
- Use cases: Weekly reports, email drafts, summaries
5.3.2 Web Reader
experts/web-reader/package.yao- config with hooksexperts/web-reader/prompts.yml- system prompt:- Input: URL or topic to search
- Output: Extracted content, key information
- Use cases: News fetching, competitor monitoring, research
experts/web-reader/src/fetch.ts- HTTP fetching utilitiesexperts/web-reader/src/fetch_test.ts- 19 test cases (100% pass)experts/web-reader/src/index.ts- Create/Next hooks
5.3.3 Data Analyst
experts/data-analyst/package.yao- configexperts/data-analyst/prompts.yml- system prompt:- Input: Data description, analysis goal
- Output: Analysis report, trends, insights
- Use cases: Sales analysis, performance review
5.3.4 Summarizer
experts/summarizer/package.yao- configexperts/summarizer/prompts.yml- system prompt:- Input: Long text content
- Output: Concise summary with key points
- Use cases: Document summarization, meeting notes
5.4 Test Scenarios
Each phase test uses different expert combinations:
| Test | Phase | Trigger | Expert Agents Used | Verification |
|---|---|---|---|---|
| T1 | P0 | Clock (interval) | - | Clock → Inspiration report |
| T2 | P1 | Clock | - | Inspiration → Goals |
| T3 | P1 | Human | - | User input → Goals |
| T4 | P2 | Clock | text-writer, web-reader | Goals → Tasks with executors |
| T5 | P3 | Clock | text-writer | Task exec → Result validation |
| T6 | P3 | Human | summarizer | Task exec → Result validation |
| T7 | P4 | Clock | - | Results → Delivery format |
| T8 | P5 | Clock | - | Full execution → Insights |
| T9 | E2E | Clock | text-writer, summarizer | Full P0→P5 flow |
| T10 | E2E | Human | web-reader, data-analyst | Full P1→P5 flow |
5.5 Verification
- All 6 Phase Agents load correctly (
robot.inspiration,robot.goals, etc.) - All 4 Expert Agents load correctly (
experts.text-writer,experts.web-reader, etc.) - Web Reader
fetch.tsutilities tested (19 tests, 100% pass)
Phase 6: P0 Inspiration Implementation ✅
Goal: Implement P0 (Inspiration Agent). Clock trigger → P0 → stub P1-P5.
Depends on: Phase 4 (Agent Call Infrastructure), Phase 5 (Assistants Setup)
Status: COMPLETED
6.1 P0 Implementation
executor/inspiration.go-RunInspiration(ctx, exec, data)- real implementationexecutor/inspiration.go- build prompt usingInputFormatter.FormatClockContext()executor/inspiration.go- call Inspiration Agent usingAgentCallerexecutor/inspiration.go- parse response toInspirationReport(markdown content)types/robot.go- addedGetRobot()/SetRobot()methods for Executionexecutor/executor.go- set robot reference on execution creation
6.2 Tests
executor/inspiration_test.go- P0 with real LLM call (8 test cases)- Test: clock context correctly formatted in prompt
- Test: robot identity included in prompt
- Test: markdown report generated with expected sections
- Test: handles LLM errors gracefully (robot nil, agent not found)
- Test: uses clock from trigger input or creates new one
InputFormatter.FormatClockContext()unit tests (4 test cases)
6.3 Notes
executor_test.gotemporarily moved to.bak- will restore when all phases implemented- P0 uses
robot.inspirationtest agent fromyao-dev-app/assistants/robot/inspiration/
Phase 7: P1 Goals Implementation ✅
Goal: Implement P1 (Goal Generation Agent). P0 → P1 → stub P2-P5.
Depends on: Phase 6 (P0 Inspiration)
Status: COMPLETED
7.1 P1 Implementation
executor/goals.go-RunGoals(ctx, exec, data)- real implementationexecutor/goals.go- build prompt with inspiration report (Clock trigger)executor/goals.go- build prompt with trigger input (Human/Event trigger)executor/goals.go- call Goals Agent usingAgentCallerexecutor/goals.go- parse response toGoalsstruct (JSON with content + delivery)executor/goals.go- handle Human/Event trigger (skip P0, use input directly)executor/goals.go- include robot identity in promptexecutor/goals.go- include available resources in promptexecutor/goals.go-ParseDelivery()- parse delivery target from JSONexecutor/goals.go-IsValidDeliveryType()- validate delivery types
7.2 Tests
executor/goals_test.go- P1 with real LLM call (14 test cases)- Test: inspiration report in prompt (Clock trigger)
- Test: user input in prompt (Human trigger)
- Test: event data in prompt (Event trigger)
- Test: goals markdown generated with priorities
- Test: delivery parsing from agent response
- Test: error handling (robot nil, agent not found, empty input)
- Test: fallback behavior (no inspiration → clock context)
ParseDelivery()unit tests (8 test cases covering edge cases)IsValidDeliveryType()unit tests
7.3 Notes
- P1 uses
robot.goalstest agent fromyao-dev-app/assistants/robot/goals/ - Goals Agent returns JSON:
{ "content": "...", "delivery": {...} } - Delivery is optional; if not present or invalid,
Goals.Deliveryis nil - Available resources (agents, MCP, KB, DB) are passed to agent for achievable goal generation
Phase 8: P2 Tasks Implementation ✅
Goal: Implement P2 (Task Planning Agent). P1 → P2 → stub P3-P5.
Depends on: Phase 7 (P1 Goals)
Status: COMPLETED
8.1 Validation Agent Setup (Prerequisite for P3) ✅
Note: Validation Agent was already set up in Phase 5.
robot/validation/package.yao- Validation Agent config (DeepSeek V3, temperature 0.2)robot/validation/prompts.yml- validation prompts- Input: Task result, expected outcome, validation rules
- Output: Validation result (pass/fail, score, issues, suggestions)
8.2 P2 Implementation ✅
executor/tasks.go-RunTasks(ctx, exec, data)- real implementationexecutor/tasks.go- build prompt with goals (usingFormatGoals)executor/tasks.go- include available tools/agents in promptexecutor/tasks.go- include delivery target in prompt (for task output format)executor/tasks.go- call Tasks Agent usingAgentCallerexecutor/tasks.go- parse response to[]Task(structured JSON)executor/tasks.go- validate task structure (executor type, ID, messages)executor/tasks.go-ParseTasks(),ParseTask(),ParseMessages()helpersexecutor/tasks.go-SortTasksByOrder()- ensure correct execution sequenceexecutor/tasks.go-ValidateExecutorExists()- optional executor existence checkexecutor/tasks.go-ValidateTasksWithResources()- validation with warningsexecutor/input.go-FormatGoals()updated to include Delivery Target
8.3 Tests ✅
executor/tasks_test.go- P2 with real LLM call (7 integration tests)- Test: goals included in prompt
- Test: available tools listed in prompt
- Test: delivery target included in prompt
- Test: structured tasks generated
- Test: each task has valid executor type and ID
- Test: each task has expected output and validation rules
ParseTasksunit tests (5 tests)ValidateTasksunit tests (5 tests)SortTasksByOrderunit tests (4 tests)ValidateExecutorExistsunit tests (7 tests)ValidateTasksWithResourcesunit tests (3 tests)ParseExecutorTypeunit tests (5 tests)IsValidExecutorTypeunit tests (2 tests)FormatGoalswith delivery target tests (4 tests)
8.4 Notes
- Tasks Agent returns JSON:
{ "tasks": [...] } - Each task includes: id, executor_type, executor_id, messages, expected_output, validation_rules, order
- Tasks are sorted by
orderfield after parsing - Executor existence is optionally validated (warnings only, doesn't block)
- Delivery target from P1 is passed to P2 so tasks can produce appropriate output format
Phase 9: P3 Run Implementation ✅
Goal: Implement P3 (Task Execution + Validation). P2 → P3 → stub P4-P5.
Depends on: Phase 8 (P2 Tasks + Validation Agent)
Status: Complete
9.1 Implementation ✅
executor/run.go-RunExecution(ctx, exec, data)- real implementationRunConfig- configuration (ContinueOnFailure, ValidationThreshold, MaxTurnsPerTask)- Sequential task execution with progress tracking
- Task status updates (Running → Completed/Failed/Skipped)
ContinueOnFailureoption for graceful failure handling- Previous task results passed as context to subsequent tasks
executor/runner.go-Runnerstruct for task executionExecuteWithRetry()- multi-turn conversation flow for assistant tasksexecuteNonAssistantTask()- single-call execution for MCP/ProcessexecuteAssistantWithMultiTurn()- AI assistant with conversation supportExecuteMCPTask()- MCP tool execution (format:clientID.toolName)ExecuteProcessTask()- Yao process executionBuildTaskContext()- context with previous resultsBuildAssistantMessages()- build messages for assistantFormatPreviousResultsAsContext()- format previous results as contextextractOutput()- extract output from CallResultgenerateDefaultReply()- fallback reply generation
executor/validator.go- Two-layer validation system- Layer 1: Rule-based validation using
yao/assert - Layer 2: Semantic validation using Validation Agent
ValidateWithContext()- validation with multi-turn supportisComplete()- determine if expected result is obtainedcheckNeedReply()- determine if conversation should continuegenerateFeedbackReply()- generate validation feedback for next turndetectNeedMoreInfo()- detect if assistant needs clarificationconvertStringRule()- natural language rules to assertionsparseRules()- JSON and string rule parsingmergeResults()- combine rule and semantic results
- Layer 1: Rule-based validation using
9.2 Assert Package ✅
Created new yao/assert package for universal assertion/validation:
assert/types.go-Assertion,Result,AssertionOptionstypesassert/asserter.go-Asserterwith 8 assertion types:equals- exact matchcontains- substring checknot_contains- negative substring checkjson_path- JSON path extraction and comparisonregex- regex pattern matchingtype- type checking (with optional path)script- custom script validationagent- AI agent validation
assert/helpers.go-ValidateOutput(),ExtractPath(),ToString(),GetType()assert/asserter_test.go- 98.7% test coverage
9.3 Tests
Completed:
assert/asserter_test.go- 40+ test cases (98.7% coverage)types/robot_test.go- Task structure tests with validation rulestasks_test.go- ParseTasks with validation rules format- Validation rules format aligned with
prompts.ymlguidelines
Completed Tests:
executor/standard/run_test.go- P3 RunExecution tests ✅- Test: tasks executed in order (
TestRunExecutionBasic) - Test: task status updates (
TestRunExecutionTaskStatus) - Test: remaining tasks marked as skipped on failure
- Test: error handling (robot nil, no tasks, non-existent assistant)
- Test: rule-based and semantic validation (
TestRunExecutionValidation) - Test: previous results passed as context to subsequent tasks
- Test: tasks executed in order (
executor/standard/runner_test.go- Runner tests ✅- Test: ExecuteWithRetry with multi-turn conversation flow
- Test: max turns limit enforcement
- Test: BuildTaskContext with previous results
- Test: FormatPreviousResultsAsContext formatting
- Test: BuildAssistantMessages with task content
- Test: FormatMessagesAsText (string, multipart, map)
- Test: MCP and Process tasks (skipped - requires runtime)
executor/standard/validator_test.go- Validator tests ✅- Test: ValidateWithContext with multi-turn state
- Test: isComplete determination logic
- Test: checkNeedReply scenarios
- Test: convertStringRule for natural language rules
- Test: parseRules for JSON assertions (equals, regex, json_path, type)
- Test: validateSemantic with Validation Agent
- Test: mergeResults logic (rule + semantic)
Completed:
- Test: ContinueOnFailure option (run_test.go) ✅
stops_on_first_failure_when_ContinueOnFailure_is_falsecontinues_execution_when_ContinueOnFailure_is_truemultiple_failures_with_ContinueOnFailure
Phase 10: P4 Delivery Implementation
Goal: Implement P4 (Delivery). P3 → P4 → stub P5.
Depends on: Phase 9 (P3 Run)
10.1 Execution Persistence (Prerequisite) ✅
Background: Each Robot execution (P0-P5) needs persistent storage for UI history queries.
yao/models/agent/execution.mod.yao- Execution record model (agent_executiontable)- id, execution_id (unique)
- member_id (globally unique), team_id
- trigger_type (enum: clock, human, event)
- Status tracking (synced with runtime Execution):
- status (enum: pending, running, completed, failed, cancelled)
- phase (enum: inspiration, goals, tasks, run, delivery, learning)
- current (JSON) - current executing state (task_index, progress)
- error - error message if failed
- input (JSON) - trigger input
- Phase outputs (P0-P5):
- inspiration (JSON) - P0 output
- goals (JSON) - P1 output
- tasks (JSON) - P2 output
- results (JSON) - P3 output
- delivery (JSON) - P4 output
- learning (JSON) - P5 output
- Timestamps: start_time, end_time, created_at, updated_at
- Relations: member (hasOne __yao.member)
agent/robot/store/execution.go- Execution record storageSave(ctx, record)- create or update execution recordGet(ctx, execID)- get execution by IDList(ctx, opts)- query execution history with filtersUpdatePhase(ctx, execID, phase, data)- update current phase and dataUpdateStatus(ctx, execID, status, error)- update execution statusUpdateCurrent(ctx, execID, current)- update current executing stateDelete(ctx, execID)- delete execution recordFromExecution(exec, robotID)- convert runtime Execution to recordToExecution()- convert record to runtime Execution
- Tests:
agent/robot/store/execution_test.go(9 test groups, all passing) - Integrate into Executor - call
UpdatePhase()after each phase completes- Added
SkipPersistenceconfig option toexecutor/types/Config - Added
ExecutionStoretoexecutor/standard/Executor - Save execution record at start of
Execute() - Call
UpdatePhase()after each phase completes inrunPhase() - Call
UpdateStatus()on status changes (running, completed, failed)
- Added
10.2 Messenger Attachment Support ✅
Conclusion: All email providers now support attachments.
Implementation Status:
| Provider | Attachment Support | Implementation |
|---|---|---|
| Twilio/SendGrid | ✅ Supported | buildAttachments() - base64 encoded |
| Mailgun | ✅ Supported | sendEmailWithAttachments() - multipart/form-data |
| SMTP (mailer) | ✅ Supported | buildMessageWithAttachments() - MIME multipart/mixed |
Features Supported:
- Regular attachments (Content-Disposition: attachment)
- Inline attachments (Content-Disposition: inline) with Content-ID for HTML embedding
- Multiple attachments per email
- Automatic content type detection
- Base64 encoding for SMTP (RFC 2045 compliant, 76-char line wrapping)
Tests Added:
messenger/providers/mailgun/mailgun_test.go:TestSend_EmailWithAttachments_MockServerTestSend_EmailWithInlineAttachment_MockServerTestSend_EmailWithAttachments_RealAPI
messenger/providers/mailer/mailer_test.go:TestBuildMessage_WithAttachments(single, multiple, inline, no attachments)TestSend_EmailWithAttachments_RealAPI
// messenger/types/types.go
type Attachment struct {
Filename string `json:"filename"`
ContentType string `json:"content_type"`
Content []byte `json:"content"`
Inline bool `json:"inline,omitempty"`
CID string `json:"cid,omitempty"`
}
Supported channels:
- Email - Full attachment support
- SMS - No attachment (text only)
- WhatsApp - TBD
10.3 Type Updates (Prerequisite) ✅
- Update
types/enums.go- UpdateDeliveryTypeenum- Remove
DeliveryFile - Add
DeliveryProcess
- Remove
- Update
types/robot.go- Delivery types for new architectureDeliveryResult- update to new structure (RequestID, Content, Results[])- Add
DeliveryContentstruct - Add
DeliveryAttachmentstruct - Add
DeliveryRequeststruct - Add
DeliveryContextstruct - Add
DeliveryPreferencesstruct (with Email, Webhook, Process) - Add
EmailPreference,EmailTargetstructs - Add
WebhookPreference,WebhookTargetstructs - Add
ProcessPreference,ProcessTargetstructs - Add
ChannelResultstruct (with Target field)
- Update
types/enums_test.go- Update DeliveryType tests - Update
types/robot_test.go- Update delivery result tests
10.4 Delivery Agent Setup
robot/delivery/package.yao- Delivery Agent configrobot/delivery/prompts.yml- delivery prompts- Input: Full execution context (P0-P3 results)
- Output: DeliveryContent (Summary, Body, Attachments) - only content, no channels
- Agent focuses on content generation, NOT channel selection
10.5 Delivery Content Structure
// DeliveryRequest - pushed to Delivery Center
// No Channels - Delivery Center decides based on preferences
type DeliveryRequest struct {
Content *DeliveryContent `json:"content"` // Agent-generated content
Context *DeliveryContext `json:"context"` // Tracking info
}
// DeliveryContent - Content generated by Delivery Agent (only content)
type DeliveryContent struct {
Summary string `json:"summary"` // Brief 1-2 sentence summary
Body string `json:"body"` // Full markdown report
Attachments []DeliveryAttachment `json:"attachments,omitempty"` // Output artifacts from P3
}
// DeliveryAttachment - Task output attachment with metadata
type DeliveryAttachment struct {
Title string `json:"title"` // Human-readable title
Description string `json:"description,omitempty"` // What this artifact is
TaskID string `json:"task_id,omitempty"` // Which task produced this
File string `json:"file"` // Wrapper: __<uploader>://<fileID>
}
// DeliveryContext - tracking info
type DeliveryContext struct {
MemberID string `json:"member_id"` // Robot member ID (globally unique)
ExecutionID string `json:"execution_id"`
TriggerType TriggerType `json:"trigger_type"` // clock | human | event
TeamID string `json:"team_id"`
}
Key Design:
- Agent only generates content (Summary, Body, Attachments)
- Delivery Center decides channels based on Robot/User preferences
- If webhook configured, every execution pushes automatically
File Wrapper:
- Format:
__<uploader>://<fileID> - Parse:
attachment.Parse(value)→(uploader, fileID, isWrapper) - Read:
attachment.Base64(ctx, value)→ base64 content
Delivery Channels (each supports multiple targets):
| Channel | Description | Multiple Targets |
|---|---|---|
email |
Send via yao/messenger | ✅ Multiple recipients |
webhook |
POST to external URL | ✅ Multiple URLs |
process |
Yao Process call | ✅ Multiple processes |
notify |
In-app notification | Future (auto by subscriptions) |
10.6 Implementation
P4 Entry (executor/delivery.go):
RunDelivery(ctx, exec, data)- P4 entry point- Call Delivery Agent to generate content (only content, no channels)
- Build DeliveryRequest (Content + Context)
- Push to Delivery Center
- Store DeliveryResult in exec.Delivery
Delivery Center (executor/delivery_center.go):
DeliveryCenter.Deliver(ctx, request)- main entry- Read Robot/User delivery preferences
- Iterate through all enabled targets for each channel
- Aggregate ChannelResults into DeliveryResult
Channel Handlers (each supports multiple targets):
sendEmail()- uses yao/messenger- Convert DeliveryAttachment to messenger.Attachment
- Support multiple EmailTarget
- Support custom subject_template per target
- Use
Robot.RobotEmailas From address (if configured) - Use global
DefaultEmailChannel()for messenger channel selection
postWebhook()- POST JSON- POST DeliveryContent as JSON payload
- Support multiple WebhookTarget
- Support custom headers per target
callProcess()- Yao Process call- DeliveryContent as first arg
- Support multiple ProcessTarget
- Support additional args per target
10.7 Tests
executor/delivery_test.go- P4 delivery- Test: Delivery Agent generates content (only content)
- Test: DeliveryCenter reads preferences
- Test: Multiple email targets (TestDeliveryCenterEmail)
- Test: Multiple webhook targets
- Test: Multiple process targets (TestDeliveryCenterProcess)
- Test: Mixed channels (email + webhook + process) (TestDeliveryCenterAllChannels)
- Test: sendEmail with attachments (TestDeliveryCenterEmail)
- Test: postWebhook with custom headers
- Test: callProcess with args (TestDeliveryCenterProcess)
- Test: Partial success (some targets fail)
- Test: DeliveryResult aggregation
Phase 11: API & Integration (MVP)
Goal: Complete Go API and end-to-end tests. Main flow: P0 → P1 → P2 → P3 → P4.
Depends on: Phase 10 (P4 Delivery)
Note: Process handlers and JSAPI are optional wrappers, moved to Phase 12. Go API is sufficient for MVP integration.
11.1 Go API Implementation
api/types.go- API request/response typesapi/lifecycle.go- manager lifecycleStart()/Stop()- manager lifecycleStartWithConfig(config)- start with custom configIsRunning()- check if system is running
api/robot.go- robot query functionsGetRobot(memberID)- get robot by member IDListRobots(query)- list robots with filteringGetRobotStatus(memberID)- get robot runtime status
api/trigger.go- trigger functionsTrigger(memberID, request)- main trigger entry pointTriggerManual(memberID, triggerType, data)- manual trigger for testingIntervene(memberID, request)- human interventionHandleEvent(memberID, request)- event trigger
api/execution.go- execution query and controlGetExecution(execID)- get execution by IDListExecutions(memberID, query)- list executionsGetExecutionStatus(execID)- get execution with runtime statusPauseExecution(execID)/ResumeExecution(execID)/StopExecution(execID)
api/api.go- package documentation- Tests:
api/*_test.go(black-box tests, 16 test cases)
11.2 End-to-End Tests
- Full clock trigger flow (P0 → P1 → P2 → P3 → P4) -
e2e_clock_test.go - Human intervention flow (P1 → P2 → P3 → P4) -
e2e_human_test.go - Event trigger flow (P1 → P2 → P3 → P4) -
e2e_event_test.go - Concurrent execution test -
e2e_concurrent_test.go - Pause/Resume/Stop test -
e2e_control_test.go
Phase 12: OpenAPI Integration
Goal: HTTP endpoints for Robot Agent management and triggers.
Depends on: Phase 11 (API & Integration), Frontend UI Design
Note: This phase will be planned in detail after frontend UI design is complete. The API endpoints will be designed based on actual UI requirements.
12.1 Planned Features
- HTTP endpoints for robot management (CRUD)
- HTTP endpoints for human intervention triggers
- Webhook endpoints for external events
- WebSocket for real-time execution status updates
- Authentication and authorization integration
12.2 Design Dependencies
- Frontend dashboard design (robot list, status, controls)
- Execution history UI design
- Human intervention UI design
- Real-time notification requirements
Phase 13: Advanced Features
Goal: P5 Learning, Process/JSAPI wrappers, dedup, plan queue.
Note: These are optional features. Main flow works without them.
13.1 Process & JSAPI Wrappers
Note: These are convenience wrappers around Go API for Yao ecosystem integration.
api/process.go- implement Process handlersrobot.Start/robot.Stoprobot.Trigger/robot.Intervene/robot.HandleEventrobot.Pause/robot.Resume/robot.Stoprobot.Get/robot.Listrobot.GetExecution/robot.ListExecutions
api/jsapi.go- implement JSAPI for JavaScript runtime- Tests for Process and JSAPI
13.2 P5 Learning Implementation
Background: P5 Learning is async, runs after P4 Delivery completes. User doesn't wait for it. Results stored in private KB for future reference.
13.2.1 Learning Agent Setup
robot/learning/package.yao- Learning Agent configrobot/learning/prompts.yml- learning prompts
13.2.2 Store Implementation
store/store.go- Store interface and structstore/kb.go- KB operations (create, save, search)store/learning.go- save learning entries to private KB
13.2.3 Implementation
executor/learning.go-RunLearning(ctx, exec, data)- real implementationexecutor/learning.go- extract learnings from executionexecutor/learning.go- call Learning Agentexecutor/learning.go- save to private KB
13.2.4 Tests
executor/learning_test.go- P5 learning- Test: learnings extracted from execution
- Test: learnings saved to KB
- Test: KB can be queried for past learnings
13.3 Fast Dedup (Time-Window)
Note: Manager has
// TODO: dedup checkcomment placeholder. Integrate after implementation.
dedup/dedup.go- Dedup structdedup/fast.go- fast in-memory time-window dedup- Key:
memberID:triggerType:window - Check before submit
- Mark after submit
- Key:
- Integrate into Manager.Tick()
- Test: dedup check/mark, window expiry
13.4 Semantic Dedup
dedup/semantic.go- call Dedup Agent for goal/task level dedup- Dedup Agent setup (
assistants/robot/dedup/) - Test: semantic dedup with real LLM
13.5 Plan Queue
plan/plan.go- plan queue implementation- Store planned tasks/goals
- Execute at next cycle or specified time
plan/schedule.go- schedule for later- Test: plan queue operations
Note: Monitoring is provided by Job system (Activity Monitor UI). No separate implementation needed.
Test Assistants Structure
yao-dev-app/assistants/robot/
├── inspiration/ # P0: Inspiration Agent
│ ├── package.yao
│ └── prompts.yml
├── goals/ # P1: Goal Generation Agent
│ ├── package.yao
│ └── prompts.yml
├── tasks/ # P2: Task Planning Agent
│ ├── package.yao
│ └── prompts.yml
├── validation/ # P3: Validation Agent
│ ├── package.yao
│ └── prompts.yml
├── delivery/ # P4: Delivery Agent
│ ├── package.yao
│ └── prompts.yml
├── learning/ # P5: Learning Agent
│ ├── package.yao
│ └── prompts.yml
└── dedup/ # Deduplication Agent
├── package.yao
└── prompts.yml
Notes
Test Environment Setup
- Environment Variables: Run
source yao/env.local.shbefore tests - Test Preparation: Use
testutils.Prepare(t)to load config, KB, and agents
package robot_test
import (
"testing"
"github.com/yaoapp/yao/agent/testutils"
)
func TestExample(t *testing.T) {
// Load environment config (from YAO_TEST_APPLICATION)
// This loads: config, connectors, KB, agents, models, etc.
testutils.Prepare(t)
defer testutils.Clean(t)
// Your test code here
}
Test Conventions
- Black-box Tests: All tests in
*_testpackage (external package) - Real LLM Calls: Use
gpt-4oordeepseekconnectors for agent tests - Incremental: Each phase builds on previous, all tests must pass before next phase
- No Skip: Do NOT use
t.Skip()except fortesting.Short()(CI mode) - Must Assert: Every test MUST have result validation assertions
func TestWithLLM(t *testing.T) {
// Only allowed Skip: testing.Short() for CI
if testing.Short() {
t.Skip("Skipping integration test")
}
testutils.Prepare(t)
defer testutils.Clean(t)
// Your test code...
result, err := SomeFunction()
// MUST have assertions - no empty tests!
assert.NoError(t, err)
assert.NotNil(t, result)
assert.Equal(t, expected, result.Field)
}
Test Rules
| Rule | Description |
|---|---|
| No arbitrary Skip | Only testing.Short() skip allowed |
| Must assert | Every test must validate results |
| No empty tests | Tests without assertions will fail review |
| Real calls | LLM tests use real API calls, not mocks |
Key Environment Variables
| Variable | Description |
|---|---|
YAO_TEST_APPLICATION |
Test app path (yao-dev-app) |
OPENAI_TEST_KEY |
OpenAI API key |
DEEPSEEK_API_KEY |
DeepSeek API key |
YAO_DB_DRIVER |
Database driver (mysql/sqlite3) |
YAO_DB_PRIMARY |
Database connection string |
Progress Tracking
| Phase | Status | Description |
|---|---|---|
| 1. Types & Interfaces | ✅ | All types, enums, interfaces |
| 2. Skeleton | ✅ | Empty stubs, code compiles |
| 3. Scheduling System | ✅ | Cache + Pool + Trigger + Job + Executor architecture |
| 4. Agent Infra | ✅ | AgentCaller, InputFormatter, test assistants |
| 5. Test Scenarios | ✅ | Phase agents (P0-P5), expert agents |
| 6. P0 Inspiration | ✅ | Inspiration Agent integration |
| 7. P1 Goals | ✅ | Goal Generation Agent integration |
| 8. P2 Tasks | ✅ | Task Planning Agent integration |
| 9. P3 Run | ✅ | Task execution + validation + yao/assert + multi-turn conversation |
| 10. P4 Delivery | ✅ | Output delivery (email/webhook/process, notify future) |
| 11. API & Integration | ✅ | Go API, end-to-end tests (main flow: P0→P1→P2→P3→P4) |
| 12. OpenAPI | ⬜ | HTTP endpoints (depends on frontend UI design) |
| 13. Advanced | ⬜ | Process/JSAPI, P5 Learning, dedup, plan queue, Sandbox |
Legend: ⬜ Not started | 🟡 In progress | ✅ Complete
Main Flow (MVP): P0 Inspiration → P1 Goals → P2 Tasks → P3 Run → P4 Delivery ✅ OpenAPI (Phase 12): HTTP endpoints - planned after frontend UI design Advanced (Phase 13): P5 Learning (async), Process/JSAPI, Dedup, Plan Queue, Sandbox
Quick Commands
# Setup environment
source yao/env.local.sh
# Run all robot tests
go test -v ./agent/robot/...
# Run specific phase tests
go test -v ./agent/robot/types/...
go test -v ./agent/robot/cache/...
go test -v ./agent/robot/pool/...
go test -v ./agent/robot/executor/...
# Run with coverage
go test -cover ./agent/robot/...