yao/agent/robot/TODO.md

Robot Agent - Implementation TODO

Based on DESIGN.md and TECHNICAL.md Test environment: source yao/env.local.sh Test assistants: yao-dev-app/assistants/robot/

---

Workflow: Human-AI Collaboration

Important: Follow this workflow strictly for each sub-task.

┌─────────────────────────────────────────────────────────────────┐
│                     Implementation Workflow                      │
├─────────────────────────────────────────────────────────────────┤
│  1. AI: Implement code for current sub-task                     │
│  2. AI: Present code for review (DO NOT write tests yet)        │
│  3. Human: Review code, provide feedback                        │
│  4. AI: Iterate based on feedback                               │
│  5. Human: Confirm "LGTM" or "Approved"                         │
│  6. AI: Write tests for the approved code                       │
│  7. Human: Review tests                                         │
│  8. AI: Run tests, fix if needed                                │
│  9. Human: Confirm sub-task complete, move to next              │
└─────────────────────────────────────────────────────────────────┘

Rules:

Rule	Description
One sub-task at a time	Focus only on current sub-task
No tests before approval	Wait for human "LGTM" before writing tests
No jumping ahead	Do not implement future phases
Ask if unclear	When in doubt, ask before proceeding

---

Core Principle

• Phase 1-2: Types + Skeleton (code compiles)
• Phase 3: Complete scheduling system (Cache + Pool + Trigger + Dedup + Job), executor is stub
• Phase 4-9: Implement executor phases one by one (P0 → P5)
• Phase 10: API completion, end-to-end tests
• Monitoring: Provided by Job system, no separate implementation

---

Phase 1: Types & Interfaces ✅

Goal: Define all types, enums, interfaces. No logic, no external deps.

Status: Complete - 88.4% test coverage, all tests passing

1.1 Enums (types/enums.go)

• Phase - execution phases (inspiration, goals, tasks, run, delivery, learning)
• ClockMode - clock trigger modes (times, interval, daemon)
• TriggerType - trigger sources (clock, human, event)
• ExecStatus - execution status (pending, running, completed, failed, cancelled)
• RobotStatus - robot status (idle, working, paused, error, maintenance)
• InterventionAction - human actions (task.add, goal.adjust, etc.)
• Priority - priority levels (high, normal, low)
• DeliveryType - delivery types (email, file, webhook, notify)
• DedupResult - dedup results (skip, merge, proceed)
• EventSource - event sources (webhook, database)
• LearningType - learning types (execution, feedback, insight)
• TaskSource - task sources (auto, human, event)
• ExecutorType - executor types (assistant, mcp, process)
• TaskStatus - task status (pending, running, completed, failed, skipped, cancelled)
• InsertPosition - insert positions (first, last, next, at)

1.2 Context (types/context.go)

• Context struct - robot execution context
• NewContext() - constructor
• UserID(), TeamID() - helper methods

1.3 Config Types (types/config.go)

• Config - main config struct
• Triggers, TriggerSwitch - trigger enable/disable
• Clock - clock config with validation
• Identity - role, duties, rules
• Quota - concurrency limits with defaults
• KB, DB - knowledge base and database config
• Learn - learning config
• Resources, MCPConfig - available agents and tools
• Delivery - output delivery config
• Event - event trigger config

1.4 Core Types (types/robot.go)

• Robot struct - runtime robot representation
• Robot methods - CanRun(), RunningCount(), AddExecution(), RemoveExecution(), GetExecution(), GetExecutions()
• Execution struct - single execution instance
• TriggerInput - stored trigger input
• CurrentState - current executing state
• Goals - P1 output (markdown)
• Task - planned task (structured)
• TaskResult - task execution result
• DeliveryResult - delivery output
• LearningEntry - knowledge to save

1.5 Clock Context (types/clock.go)

• ClockContext struct - time context for P0
• NewClockContext() - constructor

1.6 Inspiration (types/inspiration.go)

• InspirationReport struct - P0 output

1.7 Request/Response (types/request.go)

• InterveneRequest - human intervention request
• EventRequest - event trigger request
• ExecutionResult - trigger result
• RobotState - robot status query result

1.8 Interfaces (types/interfaces.go)

• Manager interface
• Executor interface
• Pool interface
• Cache interface
• Dedup interface
• Store interface

1.9 Errors (types/errors.go)

• Config errors
• Runtime errors
• Phase errors

1.10 Tests

• types/enums_test.go - enum validation
• types/config_test.go - config validation
• types/clock_test.go - clock context creation
• types/robot_test.go - robot methods

---

Phase 2: Skeleton Implementation ✅

Goal: Create all packages with empty/stub implementations. Code compiles.

Status: Complete - All packages compile successfully, no circular dependencies

2.1 Utils (utils/) ✅

• utils/convert.go - JSON, map, struct conversions (implement)
• utils/time.go - time parsing, formatting, timezone (implement)
• utils/id.go - ID generation (nanoid) (implement)
• utils/validate.go - validation helpers (implement)
• Test: utils/utils_test.go

2.2 Package Skeletons ✅ (stubs only, implemented in Phase 3)

Create empty structs and stub methods that return nil/empty/success:

• cache/cache.go - Cache struct, stub methods
• dedup/dedup.go - Dedup struct, stub methods
• store/store.go - Store struct, stub methods
• pool/pool.go - Pool struct, stub methods
• job/job.go - job helper stubs
• plan/plan.go - Plan struct, stub methods
• trigger/trigger.go - trigger dispatcher stub
• executor/executor.go - Executor struct, stub Execute()
• manager/manager.go - Manager struct, stub methods

2.3 API Skeletons ✅

• api/api.go - Go API facade (all function signatures, return errors)
• api/process.go - Yao Process registration (all processes, return errors)
• api/jsapi.go - JSAPI registration (all methods, return errors)

2.4 Root ✅

• robot.go - package entry
  • Init() - placeholder
  • Shutdown() - placeholder

2.5 Compile Test ✅

• All packages compile without errors
• All imports resolve correctly
• No circular dependencies

---

Phase 3: Complete Scheduling System

Goal: Implement complete scheduling system. Executor is stub (simulates success).

This phase delivers a fully working scheduling pipeline:

Trigger → Manager → Cache → Dedup → Pool → Worker → Executor(stub) → Job

✅ 3.1 Cache Implementation (COMPLETE)

• cache/cache.go - Cache struct with thread-safe map
• cache/load.go - load robots from __yao.member where member_type='robot' and autonomous_mode=true
  • Implemented pagination (100 robots per page)
  • Configurable model name via SetMemberModel()
• cache/refresh.go - refresh single robot, periodic full refresh (every hour)
• Test: load/refresh with real DB
  • Created comprehensive integration tests with real database
  • Tests cover Load, LoadByID, Refresh, ListByTeam, GetByStatus
  • All tests passing with proper cleanup

✅ 3.2 Pool Implementation (COMPLETE)

• pool/pool.go - worker pool with configurable size (global limit)
  • Default config: 10 workers, 100 queue size
  • Configurable via pool.NewWithConfig()
• pool/queue.go - priority queue (sorted by: robot priority, trigger type, wait time)
  • Two-level limit: global queue + per-robot queue
  • Priority: Robot Priority × 1000 + Trigger Priority × 100
• pool/worker.go - worker goroutines, dispatch to executor
  • Non-blocking quota check with re-enqueue
  • Graceful shutdown support
• Test: submit jobs, verify execution order, verify concurrency limits
  • 15 test cases covering all edge cases
  • All tests passing

✅ 3.3 Manager Implementation (COMPLETE)

Note: Manager is the scheduling core, depends on completed Cache and Pool.

• manager/manager.go - Manager struct
  • Start() - load cache, start pool, start ticker goroutine
  • Stop() - graceful shutdown (wait for running, drain queue)
  • Tick() - main loop:
    1. Get all cached robots
    2. For each robot with clock trigger enabled
    3. Check if should execute (times/interval/daemon modes)
    4. Submit to pool
  • TriggerManual() - manual trigger for testing/API
  • Clock modes: times, interval, daemon
  • Day matching for times mode
  • Timezone handling
  • Skip paused/error/maintenance robots
• Test: manager start/stop, tick cycle, manual trigger, clock modes, goroutine leak

3.4 Trigger Implementation

• trigger/trigger.go - trigger dispatcher (routes to clock/intervene/event)
• trigger/clock.go - clock trigger
  • times mode: match specific times (09:00, 14:00)
  • interval mode: run every X duration (30m, 1h)
  • daemon mode: restart immediately after completion
  • Timezone handling
• trigger/intervene.go - human intervention
  • Parse action (task.add, goal.adjust, etc.)
  • Build TriggerInput with Messages
• trigger/event.go - event handling
  • Webhook event dispatch
  • Database change event dispatch
• trigger/control.go - execution control
  • Pause execution
  • Resume execution
  • Cancel/Stop execution
• Test: clock matching (all modes), intervention handling, event dispatch

3.5 Job Integration

• job/job.go - create job
  • job_id: robot_exec_{execID}
  • category_id: autonomous_robot
  • Metadata: member_id, team_id, trigger_type, exec_id
• job/execution.go - execution lifecycle
  • Create execution on trigger
  • Update status on phase change
  • Complete/fail on finish
• job/log.go - write phase logs
  • Log phase start/end
  • Log errors
• Test: job creation, execution tracking, log writing

3.6 Executor Stub Enhancement

• executor/executor.go - enhance stub implementation
  • Execute() - simulate full execution with Job integration
    1. Create Execution record + Job
    2. Update phase: P0 → P1 → P2 → P3 → P4 → P5
    3. Log phase transitions
    4. Return success with mock data
• Test: verify stub called, verify phase progression, verify job logs

3.7 Integration Test (End-to-End Scheduling)

• Create test robot in __yao.member with clock config
• Start manager
• Wait for clock trigger
• Verify:
  • Robot loaded to cache
  • Clock trigger matched
  • Job submitted to pool
  • Worker picked up job
  • Executor stub called
  • Job execution recorded
  • Logs written
• Test human intervention trigger
• Test event trigger
• Test concurrent executions (multiple robots)
• Test quota enforcement (per-robot limit)
• Test pause/resume/stop

---

Phase 4: Executor - P0 Inspiration

Goal: Implement P0 (Inspiration Agent). Clock trigger → P0 → stub P1-P5.

4.1 Test Assistant Setup

Create yao-dev-app/assistants/robot/ directory:

• inspiration/package.yao - Inspiration Agent config
• inspiration/prompts.yml - P0 prompts
• inspiration/src/index.ts - hooks if needed

4.2 P0 Implementation

• executor/inspiration.go - build prompt with ClockContext
• executor/inspiration.go - call Inspiration Agent
• executor/inspiration.go - parse response to InspirationReport
• executor/prompt.go - BuildInspirationPrompt()

4.3 Tests

• executor/inspiration_test.go - P0 with real LLM call
• Verify: clock context in prompt
• Verify: markdown report generated

---

Phase 5: Executor - P1 Goals

Goal: Implement P1 (Goal Generation Agent). P0 → P1 → stub P2-P5.

5.1 Test Assistant Setup

• goals/package.yao - Goal Generation Agent config
• goals/prompts.yml - P1 prompts

5.2 P1 Implementation

• executor/goals.go - build prompt with inspiration report
• executor/goals.go - call Goal Agent
• executor/goals.go - parse response to Goals (markdown)
• executor/prompt.go - BuildGoalsPrompt()

5.3 Tests

• executor/goals_test.go - P1 with real LLM call
• Verify: inspiration report in prompt
• Verify: goals markdown generated

---

Phase 6: Executor - P2 Tasks

Goal: Implement P2 (Task Planning Agent). P1 → P2 → stub P3-P5.

6.1 Test Assistant Setup

• tasks/package.yao - Task Planning Agent config
• tasks/prompts.yml - P2 prompts

6.2 P2 Implementation

• executor/tasks.go - build prompt with goals
• executor/tasks.go - call Task Agent
• executor/tasks.go - parse response to []Task (structured)
• executor/prompt.go - BuildTasksPrompt()

6.3 Tests

• executor/tasks_test.go - P2 with real LLM call
• Verify: goals in prompt
• Verify: structured tasks generated

---

Phase 7: Executor - P3 Run

Goal: Implement P3 (Task Execution). P2 → P3 → stub P4-P5.

7.1 Implementation

• executor/run.go - iterate tasks
• executor/run.go - call executor (assistant/mcp/process)
• executor/run.go - collect results
• executor/agent.go - unified agent call method

7.2 Validation Agent Setup

• validation/package.yao - Validation Agent config
• validation/prompts.yml - validation prompts

7.3 Tests

• executor/run_test.go - P3 with real agent calls
• Verify: tasks executed in order
• Verify: results collected

---

Phase 8: Executor - P4 Delivery

Goal: Implement P4 (Delivery). P3 → P4 → stub P5.

8.1 Test Assistant Setup

• delivery/package.yao - Delivery Agent config
• delivery/prompts.yml - delivery prompts

8.2 Implementation

• executor/delivery.go - build delivery content
• executor/delivery.go - send via configured channel (email/file/webhook/notify)

8.3 Tests

• executor/delivery_test.go - P4 delivery
• Verify: delivery sent (mock or real)

---

Phase 9: Executor - P5 Learning

Goal: Implement P5 (Learning). Full execution flow complete.

9.1 Test Assistant Setup

• learning/package.yao - Learning Agent config
• learning/prompts.yml - learning prompts

9.2 Store Implementation

• store/kb.go - KB operations (create, save, search)
• store/learning.go - save learning entries to private KB

9.3 Implementation

• executor/learning.go - extract learnings from execution
• executor/learning.go - call Learning Agent
• executor/learning.go - save to private KB

9.4 Tests

• executor/learning_test.go - P5 learning
• Verify: learnings saved to KB

---

Phase 10: API & Integration

Goal: Complete API implementation, end-to-end tests.

10.1 API Implementation

• api/api.go - implement all Go API functions
• api/process.go - implement all Process handlers
• api/jsapi.go - implement JSAPI

10.2 End-to-End Tests

• Full clock trigger flow (P0 → P5)
• Human intervention flow (P1 → P5)
• Event trigger flow (P1 → P5)
• Concurrent execution test
• Pause/Resume/Stop test

10.3 Integration with OpenAPI

• HTTP endpoints for human intervention
• Webhook endpoints for events

---

Phase 11: Advanced Features

Goal: Implement dedup, semantic dedup, plan queue.

11.1 Fast Dedup (Time-Window)

Note: Manager has // TODO: dedup check comment placeholder. Integrate after implementation.

• dedup/dedup.go - Dedup struct
• dedup/fast.go - fast in-memory time-window dedup
  • Key: memberID:triggerType:window
  • Check before submit
  • Mark after submit
• Integrate into Manager.Tick()
• Test: dedup check/mark, window expiry

11.2 Semantic Dedup

• dedup/semantic.go - call Dedup Agent for goal/task level dedup
• Dedup Agent setup (assistants/robot/dedup/)
• Test: semantic dedup with real LLM

11.3 Plan Queue

• plan/plan.go - plan queue implementation
  • Store planned tasks/goals
  • Execute at next cycle or specified time
• plan/schedule.go - schedule for later
• Test: plan queue operations

Note: Monitoring is provided by Job system (Activity Monitor UI). No separate implementation needed.

---

Test Assistants Structure

yao-dev-app/assistants/robot/
├── inspiration/           # P0: Inspiration Agent
│   ├── package.yao
│   └── prompts.yml
├── goals/                 # P1: Goal Generation Agent
│   ├── package.yao
│   └── prompts.yml
├── tasks/                 # P2: Task Planning Agent
│   ├── package.yao
│   └── prompts.yml
├── validation/            # P3: Validation Agent
│   ├── package.yao
│   └── prompts.yml
├── delivery/              # P4: Delivery Agent
│   ├── package.yao
│   └── prompts.yml
├── learning/              # P5: Learning Agent
│   ├── package.yao
│   └── prompts.yml
└── dedup/                 # Deduplication Agent
    ├── package.yao
    └── prompts.yml

---

Notes

Test Environment Setup

1. Environment Variables: Run source yao/env.local.sh before tests
2. Test Preparation: Use testutils.Prepare(t) to load config, KB, and agents

package robot_test

import (
    "testing"
    "github.com/yaoapp/yao/agent/testutils"
)

func TestExample(t *testing.T) {
    // Load environment config (from YAO_TEST_APPLICATION)
    // This loads: config, connectors, KB, agents, models, etc.
    testutils.Prepare(t)
    defer testutils.Clean(t)

    // Your test code here
}

Test Conventions

1. Black-box Tests: All tests in *_test package (external package)
2. Real LLM Calls: Use gpt-4o or deepseek connectors for agent tests
3. Incremental: Each phase builds on previous, all tests must pass before next phase
4. No Skip: Do NOT use t.Skip() except for testing.Short() (CI mode)
5. Must Assert: Every test MUST have result validation assertions

func TestWithLLM(t *testing.T) {
    // Only allowed Skip: testing.Short() for CI
    if testing.Short() {
        t.Skip("Skipping integration test")
    }

    testutils.Prepare(t)
    defer testutils.Clean(t)

    // Your test code...
    result, err := SomeFunction()

    // MUST have assertions - no empty tests!
    assert.NoError(t, err)
    assert.NotNil(t, result)
    assert.Equal(t, expected, result.Field)
}

Test Rules

Rule	Description
No arbitrary Skip	Only testing.Short() skip allowed
Must assert	Every test must validate results
No empty tests	Tests without assertions will fail review
Real calls	LLM tests use real API calls, not mocks

Key Environment Variables

Variable	Description
YAO_TEST_APPLICATION	Test app path (yao-dev-app)
OPENAI_TEST_KEY	OpenAI API key
DEEPSEEK_API_KEY	DeepSeek API key
YAO_DB_DRIVER	Database driver (mysql/sqlite3)
YAO_DB_PRIMARY	Database connection string

---

Progress Tracking

Phase	Status	Description
1. Types & Interfaces	✅	All types, enums, interfaces
2. Skeleton	✅	Empty stubs, code compiles
3. Scheduling System	⬜	Cache + Pool + Trigger + Dedup + Job (executor stub)
4. P0 Inspiration	⬜	Inspiration Agent integration
5. P1 Goals	⬜	Goal Generation Agent integration
6. P2 Tasks	⬜	Task Planning Agent integration
7. P3 Run	⬜	Task execution (assistant/mcp/process)
8. P4 Delivery	⬜	Output delivery (email/file/webhook/notify)
9. P5 Learning	⬜	Learning Agent + KB save
10. API & Integration	⬜	Complete API, end-to-end tests
11. Advanced	⬜	Semantic dedup, plan queue

Legend: ⬜ Not started | 🟡 In progress | ✅ Complete

---

Quick Commands

# Setup environment
source yao/env.local.sh

# Run all robot tests
go test -v ./agent/robot/...

# Run specific phase tests
go test -v ./agent/robot/types/...
go test -v ./agent/robot/cache/...
go test -v ./agent/robot/pool/...
go test -v ./agent/robot/executor/...

# Run with coverage
go test -cover ./agent/robot/...