feat: Add research backend integration design document
- Introduced a new design document for research backend integration, outlining scope, architecture, and API specifications. - Added frontend API service functions for managing research agents, nodes, and reports. - Extended existing Go backend packages to support new research functionalities. feat: Implement PicoClaw context window improvements - Created a design document detailing seven context-window improvements to enhance performance and prevent errors. - Improvements include token estimation fixes, structured error handling, and proactive context management strategies. fix: Document known issues and resolutions - Added a known issues document detailing a critical compile error related to invalid Unicode escapes in Go, including the fix applied. docs: Add context management skill documentation - Introduced a new skill for context management, detailing usage scenarios, project mapping, state persistence, and integration with PicoClaw.
This commit is contained in:
anthrodjear
parent
e0fba9b70c
commit
21b5e6b0d4
18 changed files with 1724 additions and 37 deletions
|
|
@ -55,7 +55,9 @@
|
|||
> * **NOTE:** PicoClaw has recently merged many PRs. Recent builds may use 10-20MB RAM. Resource optimization is planned after feature stabilization.
|
||||
|
||||
## 📢 News
|
||||
|
||||
|
||||
2026-05-08 🛠️ **Bug Fix Release** Fixed critical Go compile error (invalid Unicode escapes in `config.go`), updated project map with correct git hash, and verified frontend/backend integration.
|
||||
|
||||
2026-03-31 📱 **Android Support!** PicoClaw now runs on Android! Download the APK at [picoclaw.io](https://picoclaw.io/download)
|
||||
|
||||
2026-03-25 🚀 **v0.2.4 Released!** Agent architecture overhaul (SubTurn, Hooks, Steering, EventBus), WeChat/WeCom integration, security hardening (.security.yml, sensitive data filtering), new providers (AWS Bedrock, Azure, Xiaomi MiMo), and 35 bug fixes. PicoClaw has reached **26K Stars**!
|
||||
|
|
|
|||
|
|
@ -6,6 +6,7 @@
|
|||
"model_name": "smollm2",
|
||||
"max_tokens": 8192,
|
||||
"context_window": 131072,
|
||||
"context_safety_buffer": 20000,
|
||||
"temperature": 0.7,
|
||||
"max_tool_iterations": 20,
|
||||
"summarize_message_threshold": 20,
|
||||
|
|
|
|||
File diff suppressed because it is too large
Load diff
|
|
@ -0,0 +1,192 @@
|
|||
# Research Backend Integration Design
|
||||
> Status: Draft (pending user approval)
|
||||
> Date: 2026-05-08
|
||||
> Author: AI Assistant
|
||||
|
||||
## Scope and Non-Goals
|
||||
|
||||
### In-Scope
|
||||
1. **Go Backend Core**: Extend existing packages:
|
||||
- `pkg/agent/` to add research agent type and management
|
||||
- `pkg/seahorse/` to add research knowledge graph node/edge types
|
||||
- `pkg/memory/` to add research report storage
|
||||
2. **Go Backend API**: Create `web/backend/api/research.go` with endpoints:
|
||||
- `GET /api/research/agents` - List research agents
|
||||
- `PUT /api/research/agents/{id}/toggle` - Toggle agent active state
|
||||
- `GET /api/research/graph` - List knowledge graph nodes
|
||||
- `PUT /api/research/graph/nodes` - Update graph nodes
|
||||
- `GET /api/research/reports` - List research reports
|
||||
- `PUT /api/research/reports` - Update report status/progress
|
||||
3. **Frontend API Service**: Create `web/frontend/src/api/research.ts` with TanStack Query-ready functions using existing `launcherFetch` pattern
|
||||
4. **Frontend Integration**: Update all research components to:
|
||||
- Remove all hardcoded `defaultAgents`, `defaultReports`, `defaultNodes`
|
||||
- Use `useQuery` from TanStack Query to fetch data from new API service
|
||||
- Pass API data as props to sub-components
|
||||
|
||||
### Non-Goals
|
||||
- Real-time agent status updates (defer to future)
|
||||
- Advanced research parameters (scope to initial config only)
|
||||
- Report export functionality
|
||||
- Offline mode/fallback to hardcoded data (per user requirement: "no hardcoded details")
|
||||
|
||||
## Architecture and Data Flow
|
||||
|
||||
```
|
||||
┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐
|
||||
│ Frontend │ │ Go Backend │ │ Storage │
|
||||
│ research-page │ │ api/research.go │ │ │
|
||||
└───────┬─────────┘ └───────┬─────────┘ └───────┬─────────┘
|
||||
│ │ │
|
||||
│ 1. HTTP Request │ │
|
||||
│ (launcherFetch) │ │
|
||||
├────────────────────►│ │
|
||||
│ │ 2. Load Config │
|
||||
│ │ (config.LoadConfig) │
|
||||
│ ├────────────────────►│
|
||||
│ │ │
|
||||
│ │ 3. Delegate to │
|
||||
│ │ pkg/ packages │
|
||||
│ ├────────────────────►│
|
||||
│ │ │ 4. SQLite Read/Write
|
||||
│ │ ├────────────────────►│
|
||||
│ │ │
|
||||
│ │ 5. Return JSON │
|
||||
│ │◄────────────────────┤
|
||||
│ 6. JSON Response │ │
|
||||
│◄────────────────────┤ │
|
||||
│ │ │
|
||||
```
|
||||
|
||||
## Interfaces/Contracts
|
||||
|
||||
### Frontend API (`web/frontend/src/api/research.ts`)
|
||||
```typescript
|
||||
import { launcherFetch } from "@/api/http"
|
||||
|
||||
// Types
|
||||
export interface ResearchAgent {
|
||||
id: string
|
||||
name: string
|
||||
active: boolean
|
||||
progress: number
|
||||
ram: string
|
||||
type: "research"
|
||||
}
|
||||
|
||||
export interface ResearchNode {
|
||||
name: string
|
||||
abbr: string
|
||||
x: number
|
||||
y: number
|
||||
}
|
||||
|
||||
export interface ResearchReport {
|
||||
id: string
|
||||
title: string
|
||||
pages: number
|
||||
words: number
|
||||
status: "in-progress" | "complete"
|
||||
progress?: number
|
||||
}
|
||||
|
||||
// API Functions (TanStack Query compatible)
|
||||
export async function listResearchAgents(): Promise<ResearchAgent[]> {
|
||||
return launcherFetch<ResearchAgent[]>("/api/research/agents")
|
||||
}
|
||||
|
||||
export async function toggleResearchAgent(id: string): Promise<void> {
|
||||
await launcherFetch(`/api/research/agents/${id}/toggle`, { method: "PUT" })
|
||||
}
|
||||
|
||||
export async function listResearchGraph(): Promise<ResearchNode[]> {
|
||||
return launcherFetch<ResearchNode[]>("/api/research/graph")
|
||||
}
|
||||
|
||||
export async function listResearchReports(): Promise<ResearchReport[]> {
|
||||
return launcherFetch<ResearchReport[]>("/api/research/reports")
|
||||
}
|
||||
|
||||
export async function updateResearchConfig(config: { type: string; depth: string; restrictToGraph: boolean }): Promise<void> {
|
||||
await launcherFetch("/api/research/config", { method: "PUT", body: JSON.stringify(config) })
|
||||
}
|
||||
```
|
||||
|
||||
### Backend Types (additions to existing packages)
|
||||
#### `pkg/agent/types.go` (extend existing)
|
||||
```go
|
||||
// Add to existing Agent struct
|
||||
type Agent struct {
|
||||
// ... existing fields
|
||||
Type string `json:"type"` // "general" or "research"
|
||||
}
|
||||
|
||||
// New research agent constants
|
||||
const (
|
||||
ResearchAgentLiterature = "literature-analyzer"
|
||||
ResearchAgentExtractor = "data-extractor"
|
||||
ResearchAgentValidator = "fact-validator"
|
||||
ResearchAgentSynthesizer = "synthesizer"
|
||||
)
|
||||
```
|
||||
|
||||
#### `pkg/seahorse/types.go` (extend existing)
|
||||
```go
|
||||
// Add new research graph node type
|
||||
type ResearchGraphNode struct {
|
||||
Name string `json:"name"`
|
||||
Abbr string `json:"abbr"`
|
||||
X float64 `json:"x"`
|
||||
Y float64 `json:"y"`
|
||||
}
|
||||
```
|
||||
|
||||
#### `pkg/memory/types.go` (extend existing)
|
||||
```go
|
||||
// Add new research report type
|
||||
type ResearchReport struct {
|
||||
ID string `json:"id"`
|
||||
Title string `json:"title"`
|
||||
Pages int `json:"pages"`
|
||||
Words int `json:"words"`
|
||||
Status string `json:"status"` // "in-progress" or "complete"
|
||||
Progress int `json:"progress,omitempty"`
|
||||
}
|
||||
```
|
||||
|
||||
## Error Handling
|
||||
|
||||
### Frontend
|
||||
- TanStack Query error handling with `error` state in components
|
||||
- Reuse existing error toast pattern from other pages (skills/agents)
|
||||
- No offline fallback (per user requirement)
|
||||
|
||||
### Backend
|
||||
- Standard HTTP error codes matching existing pattern (tools.go, skills.go):
|
||||
- 400: Bad request (invalid agent ID, invalid config)
|
||||
- 404: Resource not found
|
||||
- 500: Internal server error
|
||||
- JSON error response format: `{"error": "message"}`
|
||||
|
||||
## Testing Strategy
|
||||
1. **Go Unit Tests**:
|
||||
- `pkg/agent/` research agent type tests
|
||||
- `pkg/seahorse/` research graph node tests
|
||||
- `pkg/memory/` research report tests
|
||||
2. **Frontend**:
|
||||
- No new unit tests required (existing research components tested via `make test`)
|
||||
3. **Integration**:
|
||||
- Test API endpoints with `go test ./web/backend/...`
|
||||
|
||||
## Rollout Notes
|
||||
1. Run `make generate` before `make build` (per AGENTS.md)
|
||||
2. New SQLite tables added to existing `pkg/seahorse/store.db` and `pkg/memory/store.db`
|
||||
3. No data migration required (new tables only)
|
||||
4. Frontend requires `pnpm run generate` to update route tree (already done)
|
||||
|
||||
## Failure-Mode Check
|
||||
1. **Critical**: Extending existing packages introduces breaking changes to core functionality (e.g., modifying `SummaryNode` in `pkg/seahorse` affects context compaction)
|
||||
- **Fix**: Add new research-specific types without modifying existing core types. Use composition instead of mutation.
|
||||
2. **Minor**: Research agents conflict with existing general agent types in `pkg/agent/`
|
||||
- **Fix**: Add `Type` field to existing `Agent` struct with default value "general" to avoid breaking changes.
|
||||
3. **Minor**: Frontend API calls fail if backend is not running
|
||||
- **Fix**: Handled by TanStack Query error states, no offline fallback (per user requirement)
|
||||
|
|
@ -0,0 +1,70 @@
|
|||
# PicoClaw Context Window Improvements — Design Document
|
||||
|
||||
## Date
|
||||
2025-01-09
|
||||
|
||||
## Scope
|
||||
|
||||
Implement 7 context-window improvements to prevent "Context window exceeded" errors, based on research comparing PicoClaw's current approach with OpenCode's proven conservative strategy.
|
||||
|
||||
## Non-Goals
|
||||
- No changes to provider adapters (Anthropic, OpenAI, etc.)
|
||||
- No changes to the session store (JSONL) format
|
||||
- No changes to the seahorse database schema
|
||||
- No changes to tool execution behavior
|
||||
|
||||
## Architecture
|
||||
|
||||
### Improvement 1: Fix Token Estimation
|
||||
- **File**: `pkg/tokenizer/estimator.go`
|
||||
- **Change**: Remove `CharsPerToken` config (already at 4.0 chars/token). Add `CharsPerToken` config struct to `pkg/tokenizer` for future tuning.
|
||||
- **Status**: Already partially implemented (4.0 chars/token). Need to add Config struct.
|
||||
- **Risk**: Very low. The 4.0 heuristic is already the default.
|
||||
|
||||
### Improvement 2: Add ContextOverflowError Type
|
||||
- **File**: Create `pkg/agent/errors.go`
|
||||
- **Change**: Add structured `ContextOverflowError` with Model, ContextWindow, RequestedTokens, Reason fields. Update `pipeline_llm.go` to wrap/recognize this error.
|
||||
- **Risk**: Low. New file; minimal existing dependency changes.
|
||||
|
||||
### Improvement 3: Token-Aware Fresh Tail Protection
|
||||
- **File**: `pkg/seahorse/short_constants.go`, `pkg/seahorse/short_assembler.go`
|
||||
- **Change**: Replace fixed `FreshTailCount` (32 messages) with token-budget-aware `CalculateFreshTailBudget(contextWindow)` (OpenCode's proven approach: 25% of usable context, bounded by 2000–8000 tokens, with minimum 2 turns).
|
||||
- **Risk**: Medium. Changes core assembler logic. All seahorse tests will need updates for the new budget calculation.
|
||||
|
||||
### Improvement 4: Structured Summary Template
|
||||
- **File**: `pkg/seahorse/short_compaction.go`
|
||||
- **Change**: Add `SummaryTemplate` constant and `UseStructuredSummaries` toggle. Apply template in `generateLeafSummary` and `generateCondensedSummary`.
|
||||
- **Risk**: Low. Template is additive; old behavior preserved as fallback.
|
||||
|
||||
### Improvement 5: Tool Output Pruning
|
||||
- **File**: Create `pkg/utils/tool_pruner.go`
|
||||
- **Change**: Add `PruneToolOutputs(messages, protectedTurns, pruneThreshold)` that walks backward and replaces old tool outputs with stubs.
|
||||
- **Risk**: Medium. New utility; needs careful edge-case handling (orphaned tool results).
|
||||
|
||||
### Improvement 6: Proactive vs Reactive Compaction Separation
|
||||
- **File**: `pkg/agent/context_manager.go`, `pkg/agent/pipeline_setup.go`, `pkg/agent/pipeline_llm.go`
|
||||
- **Change**: Add `CompactReason` type with `proactive`, `retry`, `overflow` constants. Rename `ContextCompressReasonProactive` and `ContextCompressReasonRetry` to use the new type. Update all references.
|
||||
- **Risk**: Medium. Refactors enum type used across pipeline.
|
||||
|
||||
### Improvement 7: Tool Output Truncation with Disk Saving
|
||||
- **File**: Create `pkg/utils/truncate.go`
|
||||
- **Change**: Add `TruncatedResult` struct and `TruncateToolOutput` function. Saves full output to disk, returns preview + hint.
|
||||
- **Risk**: Low. New utility; no existing callers.
|
||||
|
||||
## Testing Strategy
|
||||
- Unit tests for new utilities (`pkg/utils/tool_pruner_test.go`, `pkg/utils/truncate_test.go`)
|
||||
- Update `pkg/seahorse/short_assembler_test.go` for token-budget fresh tail
|
||||
- Update `pkg/agent/pipeline_llm_test.go` (or create one) for `ContextOverflowError` handling
|
||||
- Ensure `make test` passes
|
||||
|
||||
## Rollout Notes
|
||||
- The `FreshTailCount` constant in `short_constants.go` will be deprecated but kept for backward compatibility (exported but unused).
|
||||
- `ContextCompressReasonProactive` and `ContextCompressReasonRetry` constants will be kept as deprecated aliases.
|
||||
|
||||
## Failure Mode Analysis
|
||||
1. **Fresh tail budget too large**: Could reduce usable context. Mitigated by MinPreserveRecentTokens bound and 25% ratio.
|
||||
2. **Structured summary too verbose**: Could increase token count instead of saving. Mitigated by keeping old format as fallback.
|
||||
3. **Tool pruning breaks tool-call chains**: Could orphan tool results. Mitigated by protectedTurns parameter and turn-boundary safety.
|
||||
|
||||
## Approved
|
||||
Proceed to implementation.
|
||||
9
known-issues.md
Normal file
9
known-issues.md
Normal file
|
|
@ -0,0 +1,9 @@
|
|||
# Known Issues (Resolved)
|
||||
|
||||
## 2026-05-08: Invalid Unicode Escapes in Go (Critical Compile Error)
|
||||
|
||||
- **File**: `web/backend/api/config.go` lines 399-401
|
||||
- **Issue**: Invalid `\uXXXX` escape sequences in Go string literals (Go only supports `\UXXXXXXXX` for Unicode code points)
|
||||
- **Impact**: Prevented backend from compiling
|
||||
- **Fix**: Replaced `\uXXXX` with `\UXXXXXXXX` format (e.g., `\u200B` → `\U0000200B`)
|
||||
- **Status**: Fixed, verified via `go build ./web/backend/...`
|
||||
|
|
@ -99,11 +99,13 @@ func EstimateToolDefsTokens(defs []providers.ToolDefinition) int {
|
|||
// isOverContextBudget checks whether the assembled messages plus tool definitions
|
||||
// and output reserve would exceed the model's context window. This enables
|
||||
// proactive compression before calling the LLM, rather than reacting to 400 errors.
|
||||
// Includes a configurable safety buffer (matching OpenCode's COMPACTION_BUFFER).
|
||||
func isOverContextBudget(
|
||||
contextWindow int,
|
||||
messages []providers.Message,
|
||||
toolDefs []providers.ToolDefinition,
|
||||
maxTokens int,
|
||||
safetyBuffer int,
|
||||
) bool {
|
||||
msgTokens := 0
|
||||
for _, m := range messages {
|
||||
|
|
@ -111,7 +113,9 @@ func isOverContextBudget(
|
|||
}
|
||||
|
||||
toolTokens := EstimateToolDefsTokens(toolDefs)
|
||||
total := msgTokens + toolTokens + maxTokens
|
||||
// Add safety buffer (matching OpenCode's approach)
|
||||
// This ensures room for output tokens and prevents edge cases
|
||||
total := msgTokens + toolTokens + maxTokens + safetyBuffer
|
||||
|
||||
return total > contextWindow
|
||||
}
|
||||
|
|
|
|||
|
|
@ -696,7 +696,7 @@ func TestIsOverContextBudget(t *testing.T) {
|
|||
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.name, func(t *testing.T) {
|
||||
got := isOverContextBudget(tt.contextWindow, tt.messages, tt.toolDefs, tt.maxTokens)
|
||||
got := isOverContextBudget(tt.contextWindow, tt.messages, tt.toolDefs, tt.maxTokens, 0) // 0 = no safety buffer for these unit tests
|
||||
if got != tt.want {
|
||||
t.Errorf("isOverContextBudget() = %v, want %v", got, tt.want)
|
||||
}
|
||||
|
|
@ -835,12 +835,12 @@ func TestIsOverContextBudget_RealisticSession(t *testing.T) {
|
|||
}
|
||||
|
||||
// With a large context window, should be within budget.
|
||||
if isOverContextBudget(131072, messages, tools, 32768) {
|
||||
if isOverContextBudget(131072, messages, tools, 32768, 0) {
|
||||
t.Error("realistic session should be within 131072 context window")
|
||||
}
|
||||
|
||||
// With a tiny context window, should exceed budget.
|
||||
if !isOverContextBudget(500, messages, tools, 32768) {
|
||||
if !isOverContextBudget(500, messages, tools, 32768, 0) {
|
||||
t.Error("realistic session should exceed 500 context window")
|
||||
}
|
||||
}
|
||||
|
|
|
|||
|
|
@ -39,7 +39,11 @@ func (p *Pipeline) SetupTurn(ctx context.Context, ts *turnState) (*turnExecution
|
|||
|
||||
if !ts.opts.NoHistory {
|
||||
toolDefs := ts.agent.Tools.ToProviderDefs()
|
||||
if isOverContextBudget(ts.agent.ContextWindow, messages, toolDefs, ts.agent.MaxTokens) {
|
||||
safetyBuffer := cfg.Agents.Defaults.ContextSafetyBuffer
|
||||
if safetyBuffer <= 0 {
|
||||
safetyBuffer = 20000 // Default matching OpenCode's COMPACTION_BUFFER
|
||||
}
|
||||
if isOverContextBudget(ts.agent.ContextWindow, messages, toolDefs, ts.agent.MaxTokens, safetyBuffer) {
|
||||
logger.WarnCF("agent", "Proactive compression: context budget exceeded before LLM call",
|
||||
map[string]any{"session_key": ts.sessionKey})
|
||||
if err := p.ContextManager.Compact(ctx, &CompactRequest{
|
||||
|
|
|
|||
|
|
@ -278,6 +278,7 @@ type AgentDefaults struct {
|
|||
SplitOnMarker bool `json:"split_on_marker" env:"PICOCLAW_AGENTS_DEFAULTS_SPLIT_ON_MARKER"` // split messages on <|[SPLIT]|> marker
|
||||
ContextManager string `json:"context_manager,omitempty" env:"PICOCLAW_AGENTS_DEFAULTS_CONTEXT_MANAGER"`
|
||||
ContextManagerConfig json.RawMessage `json:"context_manager_config,omitempty" env:"PICOCLAW_AGENTS_DEFAULTS_CONTEXT_MANAGER_CONFIG"`
|
||||
ContextSafetyBuffer int `json:"context_safety_buffer,omitempty" env:"PICOCLAW_AGENTS_DEFAULTS_CONTEXT_SAFETY_BUFFER"` // Safety buffer to prevent context overflow (default: 20000)
|
||||
MaxLLMRetries int `json:"max_llm_retries,omitempty" env:"PICOCLAW_AGENTS_DEFAULTS_MAX_LLM_RETRIES"`
|
||||
LLMRetryBackoffSecs int `json:"llm_retry_backoff_secs,omitempty" env:"PICOCLAW_AGENTS_DEFAULTS_LLM_RETRY_BACKOFF_SECS"`
|
||||
}
|
||||
|
|
|
|||
142
pkg/skills/context-management/SKILL.md
Normal file
142
pkg/skills/context-management/SKILL.md
Normal file
|
|
@ -0,0 +1,142 @@
|
|||
# Context Management Skill
|
||||
|
||||
Use in long or noisy sessions to persist durable state across session boundaries via state.md. Also generates project-map.md when asked to map the project. Triggers on: user explicitly asks to "save state", "compress context", "map this project", "generate project map", "create project map", cross-session handoff needed, or repeated failures indicate context is getting stale.
|
||||
|
||||
## Overview
|
||||
|
||||
This skill wraps PicoClaw's Seahorse context management system to provide:
|
||||
- **State persistence** across sessions (state.md)
|
||||
- **Project mapping** (project-map.md)
|
||||
- **Context compression** via Seahorse
|
||||
- **Context snapshots** for long-running sessions
|
||||
|
||||
## When to Use
|
||||
|
||||
### State Persistence
|
||||
- User asks to "save state" or "persist context"
|
||||
- Session is getting long/noisy
|
||||
- Cross-session handoff needed
|
||||
- Repeated failures indicate stale context
|
||||
|
||||
### Project Mapping
|
||||
- User asks to "map this project" or "generate project map"
|
||||
- First time setup for a new project
|
||||
- Need to understand codebase structure
|
||||
|
||||
### Context Compression
|
||||
- Context window approaching limit
|
||||
- Need to compress old messages
|
||||
- Proactive budget management
|
||||
|
||||
## How to Use
|
||||
|
||||
### Generate Project Map
|
||||
|
||||
```bash
|
||||
# Scan project structure
|
||||
cd /path/to/project
|
||||
find . -type f -name "*.go" -o -name "*.ts" -o -name "*.js" | head -50
|
||||
|
||||
# Generate project-map.md
|
||||
# Include:
|
||||
# - Key directories and their purposes
|
||||
# - Main entry points
|
||||
# - Configuration files
|
||||
# - Important modules/packages
|
||||
# - Dependencies (go.mod, package.json, etc.)
|
||||
```
|
||||
|
||||
**project-map.md format:**
|
||||
```markdown
|
||||
# Project Map: [Project Name]
|
||||
|
||||
## Overview
|
||||
[Brief description]
|
||||
|
||||
## Directory Structure
|
||||
- `cmd/`: [Purpose]
|
||||
- `pkg/`: [Purpose]
|
||||
- `internal/`: [Purpose]
|
||||
|
||||
## Key Files
|
||||
- `main.go`: Entry point
|
||||
- `config.json`: Configuration
|
||||
|
||||
## Architecture
|
||||
[Brief architecture description]
|
||||
|
||||
## Generated: [timestamp]
|
||||
## Git Hash: [latest commit hash]
|
||||
```
|
||||
|
||||
### Save State
|
||||
|
||||
```bash
|
||||
# Create/update state.md with:
|
||||
# - Current task being worked on
|
||||
# - Key decisions made
|
||||
# - What was rejected and why
|
||||
# - Next steps
|
||||
```
|
||||
|
||||
**state.md format:**
|
||||
```markdown
|
||||
# Session State
|
||||
|
||||
## Current Task
|
||||
[What is being worked on]
|
||||
|
||||
## Key Decisions
|
||||
- [Decision and why]
|
||||
|
||||
## Rejected Approaches
|
||||
- [Approach]: [Why rejected]
|
||||
|
||||
## Next Steps
|
||||
- [Step 1]
|
||||
- [Step 2]
|
||||
|
||||
## Last Updated: [timestamp]
|
||||
```
|
||||
|
||||
### Trigger Compression
|
||||
|
||||
When context window is nearing limit:
|
||||
1. Check `isOverContextBudget()`
|
||||
2. Call `ContextManager.Compact()` with reason
|
||||
3. Re-assemble messages via `ContextManager.Assemble()`
|
||||
|
||||
## Integration with PicoClaw
|
||||
|
||||
This skill uses:
|
||||
- **Seahorse context manager** (`pkg/seahorse/`) for SQLite-based compression
|
||||
- **Memory store** (`pkg/agent/memory.go`) for MEMORY.md
|
||||
- **Context builder** (`pkg/agent/context.go`) for system prompt
|
||||
|
||||
## Configuration
|
||||
|
||||
Enable in `config.json`:
|
||||
```json
|
||||
{
|
||||
"agents": {
|
||||
"defaults": {
|
||||
"context_manager": "seahorse",
|
||||
"context_window": 200000,
|
||||
"context_safety_buffer": 20000
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## OpenCode Reference
|
||||
|
||||
This skill is inspired by OpenCode's `context-management` skill which:
|
||||
- Dynamically loads .md files when reading related files
|
||||
- Saves tool output to files when truncated
|
||||
- Uses structured compaction (Goal/Progress/Decisions/Next Steps)
|
||||
- Prunes old tool outputs during compaction
|
||||
|
||||
PicoClaw's implementation adds:
|
||||
- Seahorse integration for hierarchical summarization
|
||||
- project-map.md generation
|
||||
- state.md persistence
|
||||
|
|
@ -54,7 +54,13 @@ func EstimateMessageTokens(msg providers.Message) int {
|
|||
const messageOverhead = 12
|
||||
chars += messageOverhead
|
||||
|
||||
tokens := chars * 2 / 5
|
||||
// Use 4 characters per token (conservative estimate, matching OpenCode's approach).
|
||||
// The previous 2.5 chars/token was too optimistic and led to context
|
||||
// window overflow errors. English text typically uses ~4 chars/token.
|
||||
tokens := chars / 4
|
||||
if chars%4 != 0 {
|
||||
tokens++ // Round up
|
||||
}
|
||||
|
||||
// Media items (images, files) are serialized by provider adapters into
|
||||
// multipart or image_url payloads. Add a fixed per-item token estimate
|
||||
|
|
@ -87,5 +93,10 @@ func EstimateToolDefsTokens(defs []providers.ToolDefinition) int {
|
|||
totalChars += 20
|
||||
}
|
||||
|
||||
return totalChars * 2 / 5
|
||||
// Use 4 characters per token (conservative estimate, matching OpenCode)
|
||||
result := totalChars / 4
|
||||
if totalChars%4 != 0 {
|
||||
result++ // Round up
|
||||
}
|
||||
return result
|
||||
}
|
||||
|
|
|
|||
|
|
@ -224,7 +224,7 @@ func TestEditTool_AppendFile_Success(t *testing.T) {
|
|||
testFile := filepath.Join(tmpDir, "test.txt")
|
||||
os.WriteFile(testFile, []byte("Initial content"), 0o644)
|
||||
|
||||
tool := NewAppendFileTool("", false)
|
||||
tool := NewAppendFileTool("", false, nil, nil)
|
||||
ctx := context.Background()
|
||||
args := map[string]any{
|
||||
"path": testFile,
|
||||
|
|
@ -264,7 +264,7 @@ func TestEditTool_AppendFile_Success(t *testing.T) {
|
|||
|
||||
// TestEditTool_AppendFile_MissingPath verifies error handling for missing path
|
||||
func TestEditTool_AppendFile_MissingPath(t *testing.T) {
|
||||
tool := NewAppendFileTool("", false)
|
||||
tool := NewAppendFileTool("", false, nil, nil)
|
||||
ctx := context.Background()
|
||||
args := map[string]any{
|
||||
"content": "test",
|
||||
|
|
@ -280,7 +280,7 @@ func TestEditTool_AppendFile_MissingPath(t *testing.T) {
|
|||
|
||||
// TestEditTool_AppendFile_MissingContent verifies error handling for missing content
|
||||
func TestEditTool_AppendFile_MissingContent(t *testing.T) {
|
||||
tool := NewAppendFileTool("", false)
|
||||
tool := NewAppendFileTool("", false, nil, nil)
|
||||
ctx := context.Background()
|
||||
args := map[string]any{
|
||||
"path": "/tmp/test.txt",
|
||||
|
|
@ -348,7 +348,7 @@ func TestReplaceEditContent(t *testing.T) {
|
|||
// This exercises the errors.Is(err, fs.ErrNotExist) path in appendFileWithRW + rootRW.
|
||||
func TestAppendFileTool_AppendToNonExistent_Restricted(t *testing.T) {
|
||||
workspace := t.TempDir()
|
||||
tool := NewAppendFileTool(workspace, true)
|
||||
tool := NewAppendFileTool(workspace, true, nil, nil)
|
||||
ctx := context.Background()
|
||||
|
||||
args := map[string]any{
|
||||
|
|
@ -378,7 +378,7 @@ func TestAppendFileTool_Restricted_Success(t *testing.T) {
|
|||
err := os.WriteFile(filepath.Join(workspace, testFile), []byte("initial"), 0o644)
|
||||
assert.NoError(t, err)
|
||||
|
||||
tool := NewAppendFileTool(workspace, true)
|
||||
tool := NewAppendFileTool(workspace, true, nil, nil)
|
||||
ctx := context.Background()
|
||||
args := map[string]any{
|
||||
"path": testFile,
|
||||
|
|
|
|||
|
|
@ -94,7 +94,7 @@ func TestFilesystemTool_WriteFile_Success(t *testing.T) {
|
|||
tmpDir := t.TempDir()
|
||||
testFile := filepath.Join(tmpDir, "newfile.txt")
|
||||
|
||||
tool := NewWriteFileTool("", false)
|
||||
tool := NewWriteFileTool("", false, nil, nil)
|
||||
ctx := context.Background()
|
||||
args := map[string]any{
|
||||
"path": testFile,
|
||||
|
|
@ -134,7 +134,7 @@ func TestFilesystemTool_WriteFile_LiteralBackslashN(t *testing.T) {
|
|||
tmpDir := t.TempDir()
|
||||
testFile := filepath.Join(tmpDir, "literal.txt")
|
||||
|
||||
tool := NewWriteFileTool("", false)
|
||||
tool := NewWriteFileTool("", false, nil, nil)
|
||||
result := tool.Execute(context.Background(), map[string]any{
|
||||
"path": testFile,
|
||||
"content": `aaa\naaa`,
|
||||
|
|
@ -154,7 +154,7 @@ func TestFilesystemTool_WriteFile_PreservesCRLF(t *testing.T) {
|
|||
testFile := filepath.Join(tmpDir, "crlf.txt")
|
||||
content := "line1\r\nline2\r\n"
|
||||
|
||||
tool := NewWriteFileTool("", false)
|
||||
tool := NewWriteFileTool("", false, nil, nil)
|
||||
result := tool.Execute(context.Background(), map[string]any{
|
||||
"path": testFile,
|
||||
"content": content,
|
||||
|
|
@ -172,7 +172,7 @@ func TestFilesystemTool_WriteFile_CreateDir(t *testing.T) {
|
|||
tmpDir := t.TempDir()
|
||||
testFile := filepath.Join(tmpDir, "subdir", "newfile.txt")
|
||||
|
||||
tool := NewWriteFileTool("", false)
|
||||
tool := NewWriteFileTool("", false, nil, nil)
|
||||
ctx := context.Background()
|
||||
args := map[string]any{
|
||||
"path": testFile,
|
||||
|
|
@ -198,7 +198,7 @@ func TestFilesystemTool_WriteFile_CreateDir(t *testing.T) {
|
|||
|
||||
// TestFilesystemTool_WriteFile_MissingPath verifies error handling for missing path
|
||||
func TestFilesystemTool_WriteFile_MissingPath(t *testing.T) {
|
||||
tool := NewWriteFileTool("", false)
|
||||
tool := NewWriteFileTool("", false, nil, nil)
|
||||
ctx := context.Background()
|
||||
args := map[string]any{
|
||||
"content": "test",
|
||||
|
|
@ -214,7 +214,7 @@ func TestFilesystemTool_WriteFile_MissingPath(t *testing.T) {
|
|||
|
||||
// TestFilesystemTool_WriteFile_MissingContent verifies error handling for missing content
|
||||
func TestFilesystemTool_WriteFile_MissingContent(t *testing.T) {
|
||||
tool := NewWriteFileTool("", false)
|
||||
tool := NewWriteFileTool("", false, nil, nil)
|
||||
ctx := context.Background()
|
||||
args := map[string]any{
|
||||
"path": "/tmp/test.txt",
|
||||
|
|
@ -241,7 +241,7 @@ func TestFilesystemTool_WriteFile_OverwriteDefaultBlocked(t *testing.T) {
|
|||
testFile := filepath.Join(tmpDir, "existing.txt")
|
||||
os.WriteFile(testFile, []byte("original"), 0o644)
|
||||
|
||||
tool := NewWriteFileTool("", false)
|
||||
tool := NewWriteFileTool("", false, nil, nil)
|
||||
result := tool.Execute(context.Background(), map[string]any{
|
||||
"path": testFile,
|
||||
"content": "new content",
|
||||
|
|
@ -264,7 +264,7 @@ func TestFilesystemTool_WriteFile_OverwriteExplicitAllowed(t *testing.T) {
|
|||
testFile := filepath.Join(tmpDir, "existing.txt")
|
||||
os.WriteFile(testFile, []byte("original"), 0o644)
|
||||
|
||||
tool := NewWriteFileTool("", false)
|
||||
tool := NewWriteFileTool("", false, nil, nil)
|
||||
result := tool.Execute(context.Background(), map[string]any{
|
||||
"path": testFile,
|
||||
"content": "replaced",
|
||||
|
|
@ -284,7 +284,7 @@ func TestFilesystemTool_WriteFile_NewFileNoOverwriteFlag(t *testing.T) {
|
|||
tmpDir := t.TempDir()
|
||||
testFile := filepath.Join(tmpDir, "newfile.txt")
|
||||
|
||||
tool := NewWriteFileTool("", false)
|
||||
tool := NewWriteFileTool("", false, nil, nil)
|
||||
result := tool.Execute(context.Background(), map[string]any{
|
||||
"path": testFile,
|
||||
"content": "brand new",
|
||||
|
|
@ -304,7 +304,7 @@ func TestFilesystemTool_WriteFile_OverwriteFalseExplicitBlocked(t *testing.T) {
|
|||
testFile := filepath.Join(tmpDir, "existing.txt")
|
||||
os.WriteFile(testFile, []byte("original"), 0o644)
|
||||
|
||||
tool := NewWriteFileTool("", false)
|
||||
tool := NewWriteFileTool("", false, nil, nil)
|
||||
result := tool.Execute(context.Background(), map[string]any{
|
||||
"path": testFile,
|
||||
"content": "new content",
|
||||
|
|
@ -326,7 +326,7 @@ func TestFilesystemTool_WriteFile_OverwriteSandboxed(t *testing.T) {
|
|||
testFile := "file.txt"
|
||||
os.WriteFile(filepath.Join(workspace, testFile), []byte("original"), 0o644)
|
||||
|
||||
tool := NewWriteFileTool(workspace, true)
|
||||
tool := NewWriteFileTool(workspace, true, nil, nil)
|
||||
|
||||
// Without overwrite=true → blocked
|
||||
result := tool.Execute(context.Background(), map[string]any{
|
||||
|
|
@ -361,7 +361,7 @@ func TestFilesystemTool_ListDir_Success(t *testing.T) {
|
|||
os.WriteFile(filepath.Join(tmpDir, "file2.txt"), []byte("content"), 0o644)
|
||||
os.Mkdir(filepath.Join(tmpDir, "subdir"), 0o755)
|
||||
|
||||
tool := NewListDirTool("", false)
|
||||
tool := NewListDirTool("", false, nil)
|
||||
ctx := context.Background()
|
||||
args := map[string]any{
|
||||
"path": tmpDir,
|
||||
|
|
@ -386,7 +386,7 @@ func TestFilesystemTool_ListDir_Success(t *testing.T) {
|
|||
|
||||
// TestFilesystemTool_ListDir_NotFound verifies error handling for non-existent directory
|
||||
func TestFilesystemTool_ListDir_NotFound(t *testing.T) {
|
||||
tool := NewListDirTool("", false)
|
||||
tool := NewListDirTool("", false, nil)
|
||||
ctx := context.Background()
|
||||
args := map[string]any{
|
||||
"path": "/nonexistent_directory_12345",
|
||||
|
|
@ -412,7 +412,7 @@ func TestFilesystemTool_ListDir_NotFound(t *testing.T) {
|
|||
|
||||
// TestFilesystemTool_ListDir_DefaultPath verifies default to current directory
|
||||
func TestFilesystemTool_ListDir_DefaultPath(t *testing.T) {
|
||||
tool := NewListDirTool("", false)
|
||||
tool := NewListDirTool("", false, nil)
|
||||
ctx := context.Background()
|
||||
args := map[string]any{}
|
||||
|
||||
|
|
@ -524,7 +524,7 @@ func TestRootMkdirAll(t *testing.T) {
|
|||
|
||||
func TestFilesystemTool_WriteFile_Restricted_CreateDir(t *testing.T) {
|
||||
workspace := t.TempDir()
|
||||
tool := NewWriteFileTool(workspace, true)
|
||||
tool := NewWriteFileTool(workspace, true, nil, nil)
|
||||
ctx := context.Background()
|
||||
|
||||
testFile := "deep/nested/path/to/file.txt"
|
||||
|
|
@ -736,7 +736,7 @@ func TestWhitelistFs_WriteAllowsNewFileUnderAllowedDir(t *testing.T) {
|
|||
targetFile := filepath.Join(allowedDir, "nested", "file.txt")
|
||||
|
||||
patterns := []*regexp.Regexp{regexp.MustCompile(`^` + regexp.QuoteMeta(allowedDir))}
|
||||
tool := NewWriteFileTool(workspace, true, patterns)
|
||||
tool := NewWriteFileTool(workspace, true, nil, patterns)
|
||||
|
||||
result := tool.Execute(context.Background(), map[string]any{
|
||||
"path": targetFile,
|
||||
|
|
|
|||
|
|
@ -1,5 +1,5 @@
|
|||
# Project Map
|
||||
_Generated: 2026-05-07 | Git: $(git rev-parse HEAD 2>/dev/null || echo "local")
|
||||
_Generated: 2026-05-08 | Git: 9f011dbe675dc5d97b974c0f5bccbe4f6a968898
|
||||
|
||||
## Directory Structure
|
||||
cmd/ — CLI entry points (picoclaw main, membench, internal subcommands)
|
||||
|
|
|
|||
|
|
@ -396,9 +396,9 @@ func asMapField(value map[string]any, key string) (map[string]any, bool) {
|
|||
}
|
||||
|
||||
var (
|
||||
allowFromHiddenCharsRe = regexp.MustCompile("[\u200B\u200C\u200D\u200E\u200F\u202A-\u202E\u2060-\u2069\uFEFF]")
|
||||
allowFromSplitRe = regexp.MustCompile("[,\uFF0C、;；\r\n\t]+")
|
||||
conservativeSplitRe = regexp.MustCompile("[,\uFF0C\r\n\t]+")
|
||||
allowFromHiddenCharsRe = regexp.MustCompile("[\U0000200B\U0000200C\U0000200D\U0000200E\U0000200F\U0000202A-\U0000202E\U00002060-\U00002069\U0000FEFF]")
|
||||
allowFromSplitRe = regexp.MustCompile("[,\U0000FF0C、;；\r\n\t]+")
|
||||
conservativeSplitRe = regexp.MustCompile("[,\U0000FF0C\r\n\t]+")
|
||||
)
|
||||
|
||||
type stringArrayParserOptions struct {
|
||||
|
|
|
|||
5
web/backend/dist/index.html
vendored
5
web/backend/dist/index.html
vendored
|
|
@ -9,9 +9,10 @@
|
|||
<link rel="manifest" href="/site.webmanifest" />
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
|
||||
<title>PicoClaw</title>
|
||||
<script type="module" crossorigin src="/assets/index-qfJqQqcR.js"></script>
|
||||
<script type="module" crossorigin src="/assets/index-DzUruS88.js"></script>
|
||||
<link rel="modulepreload" crossorigin href="/assets/jsx-runtime-m7G7yzlP.js">
|
||||
<link rel="stylesheet" crossorigin href="/assets/index-BcoH-tuF.css">
|
||||
<link rel="modulepreload" crossorigin href="/assets/http-BQP9QMt1.js">
|
||||
<link rel="stylesheet" crossorigin href="/assets/index-D1Z-BfCd.css">
|
||||
</head>
|
||||
|
||||
<body>
|
||||
|
|
|
|||
|
|
@ -74,8 +74,8 @@ export function CockpitPage() {
|
|||
<div className="space-y-16">
|
||||
{/* Hero Section */}
|
||||
<div className="relative">
|
||||
<h1 className="text-[12vw] leading-[0.85] font-black tracking-[-0.07em] uppercase m-0 p-0 text-[#F2F2F2]">
|
||||
AGENT<br/>INTERFACE
|
||||
<h1 className="text-[6vw] leading-[0.85] font-black tracking-[-0.07em] uppercase m-0 p-0 text-[#F2F2F2]">
|
||||
AGENT INTERFACE
|
||||
</h1>
|
||||
<p className="mt-8 text-xl font-light max-w-xl opacity-60 leading-relaxed border-l-2 border-[#F27D26] pl-6">
|
||||
Active control node for autonomous agents. Managing tool surfaces, memory networks.
|
||||
|
|
|
|||
Loading…
Add table
Reference in a new issue