CLI overrides, workspace config, MagicForm channel, and security hardening (#1)

* feat(cli): add workspace, config-dir, tools, and skills override flags

Add --workspace, --config-dir, --tools, and --skills flags to
`picoclaw agent` for single-shot invocations. Supports workspace
override with bootstrap file injection (AGENTS.md, IDENTITY.md,
SOUL.md, USER.md), tool allowlisting, and skills filtering.

Also extracts FormatSkillsSummary from SkillsLoader for reuse,
adds SetSkillsFilter to ContextBuilder, and wires SkillsFilter
from agent config into the context builder.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat(channels): add MagicForm webhook channel plugin

Implement MagicForm channel for webhook-based agentic task delegation.
Supports Bearer token auth, async processing with HTTP callback,
workspace path validation against configurable workspace_root,
bootstrap file injection, tool/skill filtering via metadata, and
per-conversation session key isolation.

Includes Bus() accessor on BaseChannel for direct message publishing,
MagicFormConfig with workspace_root for path traversal prevention,
request body size limits, and TTL cleanup for stale request contexts.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat(agent): support per-request workspace, tool, and skill overrides

Add workspace override, tool allowlisting, and skills filtering to the
agent loop via processOptions metadata. When a workspace override is
active (e.g. from MagicForm channel), creates isolated SessionManager
and ContextBuilder instances per request.

Threads effSessions/effContextBuilder through all downstream paths
including runLLMIteration, forceCompression, maybeSummarize, and
summarizeSession to ensure full workspace isolation. Adds defense-in-
depth tool execution guard alongside LLM-facing tool definition filter.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* security(auth): add response body size limits to OAuth HTTP calls

Wrap all io.ReadAll(resp.Body) calls in the OAuth flow with
io.LimitReader capped at 1 MB to prevent memory exhaustion from
malicious or unexpectedly large server responses.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* security(tools): add response body size limits to search providers

Add 2 MB LimitReader to BraveSearch, Tavily, DuckDuckGo, and
Perplexity search providers to prevent memory exhaustion from
unexpectedly large responses. Matches existing pattern used by
GLMSearchProvider.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* security(tools): add 20 MB file write size limit

Reject writes exceeding 20 MB in WriteFileTool to prevent disk
exhaustion from unexpectedly large LLM-generated file content.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* security(tools): use crypto/rand for temp file naming

Replace predictable PID+nanosecond temp file names with
cryptographically random hex strings to prevent symlink
pre-placement attacks.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* security(skills): validate GitHub repo format and limit download size

Add regex validation for 'owner/repo' format to prevent URL injection
in InstallFromGitHub. Add 5 MB LimitReader on skill file downloads
to prevent memory exhaustion.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* security(auth): allow OAuth credentials to be overridden via env vars

Add PICOCLAW_OPENAI_CLIENT_ID, PICOCLAW_GOOGLE_CLIENT_ID, and
PICOCLAW_GOOGLE_CLIENT_SECRET env var overrides. Hardcoded values
remain as fallback defaults for backward compatibility.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* security(auth): sanitize OAuth error messages

Replace raw HTTP response bodies in error messages with generic
status-code-only messages. Full response details are logged at
debug level for troubleshooting.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* security(launcher): HTML-escape error messages in auth callback

Use html.EscapeString on all error text rendered in HTML responses
to prevent XSS via crafted OAuth error parameters or error messages.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* security(launcher): remove internal details from HTTP error responses

Log detailed errors server-side and return generic messages to
clients to prevent information disclosure of file paths and
internal state.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* security(launcher): add HTTP security headers middleware

Add SecurityHeaders middleware setting X-Content-Type-Options,
X-Frame-Options, and Content-Security-Policy on all responses
to mitigate MIME sniffing, clickjacking, and content injection.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* security(skills): validate ClawHub registry base URL scheme

Reject non-HTTPS registry URLs (unless localhost) to prevent
config-based redirection to malicious skill servers. Falls back
to the default https://clawhub.ai with a warning.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* security(skills): prevent workspace skills from shadowing builtins

Builtin skill names are now reserved — workspace and global skills
with the same name as a builtin are skipped with a warning log.
LoadSkill also checks builtins first. This prevents supply chain
attacks where a malicious skill replaces a trusted builtin.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* security(skills): warn when safety metadata is unavailable

Add MetadataAvailable flag to InstallResult. When the registry
metadata fetch fails (silent fallback), the user now sees a
warning that safety checks could not be completed.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* security(shell): add opt-in environment variable filtering

When filter_env is enabled in config, shell commands run with a minimal
allowlist of environment variables (PATH, HOME, LANG, TERM, etc. plus
PICOCLAW_* prefix), preventing accidental leakage of sensitive env vars
like API keys to spawned processes. Disabled by default for backward compat.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat(config): add workspace-local config.json overlay for per-tenant isolation

Support per-workspace config overrides via --config-dir (CLI) or configDir
(webhook), enabling different API keys, models, and agent settings per tenant.

- Add LoadWorkspaceConfig/MergeWorkspaceConfig with raw JSON overlay to
  preserve unmentioned bool fields (prevents clobbering tool enabled flags)
- Add Config.Clone() for safe per-request config copies in gateway mode
- Gateway: per-request provider creation from workspace config, with
  effProvider/effModel threaded through all LLM call sites and summarization
- CLI: workspace config merged before provider creation, CLI flags win
- MagicForm: replace inline bootstrap fields with configDir path;
  agent loop copies bootstrap files and loads config.json from configDir
- Fix --session flag: format as agent:main:cli:{key} so router honors it
- Fix cron session key: format as agent:main:cron:{id} for isolation
- Add shared CopyBootstrapFiles helper in pkg/agent/bootstrap.go
- Add config/workspace.config.example.json

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(agent): honor workspace config overrides in model routing

selectCandidates uses agent.Model/Candidates baked in at startup, which
ignores per-request workspace config overrides (effProvider/effModel).
When the workspace overrides the provider, skip routing and fallback
candidates entirely to avoid cross-provider credential issues. When only
the model is overridden, allow routing but use the effective model.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* style: fix linter errors (gci, gofumpt, golines, govet, unused)

- gci: alphabetize magicform import in gateway/helpers.go
- gofumpt: reformat multi-line function signature in agent/helpers.go
- golines: wrap lines exceeding 120 chars across 5 files
- govet: rename shadowed variables (err → wcErr, s → trimmed)
- unused: remove maybeSummarize, forceCompression, summarizeSession
  wrappers superseded by their *With variants

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: nuestraai <nuestraai@users.noreply.github.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: admin-mf <admin@magicform.ai>
This commit is contained in:
nuestraai 2026-03-06 10:16:36 -06:00 committed by GitHub
parent 1945436dd4
commit ac6e77d7c9
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
28 changed files with 1322 additions and 134 deletions

View file

@ -3,6 +3,7 @@ package server
import (
"encoding/json"
"fmt"
"html"
"io"
"log"
"net/http"
@ -237,7 +238,7 @@ func handleOAuthCallback(w http.ResponseWriter, r *http.Request) {
fmt.Fprintf(
w,
`<html><body><h2>Authentication failed</h2><p>%s</p><p>You can close this window.</p></body></html>`,
errMsg,
html.EscapeString(errMsg),
)
return
}
@ -248,7 +249,7 @@ func handleOAuthCallback(w http.ResponseWriter, r *http.Request) {
fmt.Fprintf(
w,
`<html><body><h2>Authentication failed</h2><p>%s</p><p>You can close this window.</p></body></html>`,
err.Error(),
html.EscapeString(err.Error()),
)
return
}
@ -267,7 +268,11 @@ func handleOAuthCallback(w http.ResponseWriter, r *http.Request) {
if err := auth.SetCredential(session.Provider, cred); err != nil {
w.Header().Set("Content-Type", "text/html")
fmt.Fprintf(w, `<html><body><h2>Failed to save credentials</h2><p>%s</p></body></html>`, err.Error())
fmt.Fprintf(
w,
`<html><body><h2>Failed to save credentials</h2><p>%s</p></body></html>`,
html.EscapeString(err.Error()),
)
return
}

View file

@ -32,7 +32,8 @@ func RegisterConfigAPI(mux *http.ServeMux, absPath string) {
mux.HandleFunc("GET /api/config", func(w http.ResponseWriter, r *http.Request) {
cfg, err := config.LoadConfig(absPath)
if err != nil {
http.Error(w, fmt.Sprintf("Failed to load config: %v", err), http.StatusInternalServerError)
log.Printf("Failed to load config: %v", err)
http.Error(w, "Failed to load config", http.StatusInternalServerError)
return
}
w.Header().Set("Content-Type", "application/json")
@ -63,7 +64,8 @@ func RegisterConfigAPI(mux *http.ServeMux, absPath string) {
}
if err := config.SaveConfig(absPath, &cfg); err != nil {
http.Error(w, fmt.Sprintf("Failed to save config: %v", err), http.StatusInternalServerError)
log.Printf("Failed to save config: %v", err)
http.Error(w, "Failed to save config", http.StatusInternalServerError)
return
}
@ -77,7 +79,8 @@ func RegisterAuthAPI(mux *http.ServeMux, absPath string) {
mux.HandleFunc("GET /api/auth/status", func(w http.ResponseWriter, r *http.Request) {
store, err := auth.LoadStore()
if err != nil {
http.Error(w, fmt.Sprintf("Failed to load auth store: %v", err), http.StatusInternalServerError)
log.Printf("Failed to load auth store: %v", err)
http.Error(w, "Failed to load auth store", http.StatusInternalServerError)
return
}
@ -194,3 +197,14 @@ func RegisterAuthAPI(mux *http.ServeMux, absPath string) {
// GET /auth/callback — OAuth browser callback for Google Antigravity
mux.HandleFunc("GET /auth/callback", handleOAuthCallback)
}
// SecurityHeaders wraps an http.Handler to add standard security headers.
func SecurityHeaders(next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.Header().Set("X-Content-Type-Options", "nosniff")
w.Header().Set("X-Frame-Options", "DENY")
w.Header().
Set("Content-Security-Policy", "default-src 'self'; script-src 'self' 'unsafe-inline'; style-src 'self' 'unsafe-inline'")
next.ServeHTTP(w, r)
})
}

View file

@ -105,7 +105,7 @@ func main() {
}
}()
if err := http.ListenAndServe(addr, mux); err != nil {
if err := http.ListenAndServe(addr, server.SecurityHeaders(mux)); err != nil {
log.Fatalf("Server failed: %v", err)
}
}

View file

@ -10,6 +10,11 @@ func NewAgentCommand() *cobra.Command {
sessionKey string
model string
debug bool
// Workspace and config overrides
workspace string
configDir string
tools string
skills string
)
cmd := &cobra.Command{
@ -17,14 +22,23 @@ func NewAgentCommand() *cobra.Command {
Short: "Interact with the agent directly",
Args: cobra.NoArgs,
RunE: func(cmd *cobra.Command, _ []string) error {
return agentCmd(message, sessionKey, model, debug)
return agentCmd(message, sessionKey, model, debug,
workspace, configDir, tools, skills)
},
}
cmd.Flags().BoolVarP(&debug, "debug", "d", false, "Enable debug logging")
cmd.Flags().StringVarP(&message, "message", "m", "", "Send a single message (non-interactive mode)")
cmd.Flags().StringVarP(&sessionKey, "session", "s", "cli:default", "Session key")
cmd.Flags().
StringVarP(&sessionKey, "session", "s", "", "Session key for conversation isolation (e.g. stackId:conversationId)")
cmd.Flags().StringVarP(&model, "model", "", "", "Model to use")
// Workspace and config overrides
cmd.Flags().StringVar(&workspace, "workspace", "", "Override agent workspace directory")
cmd.Flags().
StringVar(&configDir, "config-dir", "", "Directory containing config.json (model/agent/tool overrides) and bootstrap files (AGENTS.md, IDENTITY.md, SOUL.md, USER.md)")
cmd.Flags().StringVar(&tools, "tools", "", "Comma-separated tool allowlist (only these tools enabled)")
cmd.Flags().StringVar(&skills, "skills", "", "Comma-separated skill filter (only these skills loaded)")
return cmd
}

View file

@ -14,13 +14,18 @@ import (
"github.com/sipeed/picoclaw/cmd/picoclaw/internal"
"github.com/sipeed/picoclaw/pkg/agent"
"github.com/sipeed/picoclaw/pkg/bus"
"github.com/sipeed/picoclaw/pkg/config"
"github.com/sipeed/picoclaw/pkg/logger"
"github.com/sipeed/picoclaw/pkg/providers"
)
func agentCmd(message, sessionKey, model string, debug bool) error {
func agentCmd(message, sessionKey, model string, debug bool,
workspace, configDir, toolsFlag, skillsFlag string,
) error {
if sessionKey == "" {
sessionKey = "cli:default"
sessionKey = "agent:main:cli:default"
} else if !strings.HasPrefix(sessionKey, "agent:") {
sessionKey = "agent:main:cli:" + sessionKey
}
if debug {
@ -33,10 +38,44 @@ func agentCmd(message, sessionKey, model string, debug bool) error {
return fmt.Errorf("error loading config: %w", err)
}
// Apply workspace-local config overrides from config-dir
if configDir != "" {
wc, wcErr := config.LoadWorkspaceConfig(configDir)
if wcErr != nil {
return fmt.Errorf("error loading workspace config from %s: %w", configDir, wcErr)
}
cfg.MergeWorkspaceConfig(wc)
}
// CLI flags win over workspace config
if model != "" {
cfg.Agents.Defaults.ModelName = model
}
// Workspace override
if workspace != "" {
cfg.Agents.Defaults.Workspace = workspace
os.MkdirAll(workspace, 0o755)
}
// Tool allowlist: disable all tools, then enable only the listed ones
if toolsFlag != "" {
toolList := strings.Split(toolsFlag, ",")
for i := range toolList {
toolList[i] = strings.TrimSpace(toolList[i])
}
applyToolAllowlist(cfg, toolList)
}
// Skills filter: inject into agent config so NewAgentInstance picks it up
if skillsFlag != "" {
skillList := strings.Split(skillsFlag, ",")
for i := range skillList {
skillList[i] = strings.TrimSpace(skillList[i])
}
applySkillsFilter(cfg, skillList)
}
provider, modelID, err := providers.CreateProvider(cfg)
if err != nil {
return fmt.Errorf("error creating provider: %w", err)
@ -51,6 +90,11 @@ func agentCmd(message, sessionKey, model string, debug bool) error {
defer msgBus.Close()
agentLoop := agent.NewAgentLoop(cfg, msgBus, provider)
// Copy bootstrap files from config-dir to workspace
if configDir != "" {
copyBootstrapFiles(configDir, cfg.Agents.Defaults.Workspace)
}
// Print agent startup info (only for interactive mode)
startupInfo := agentLoop.GetStartupInfo()
logger.InfoCF("agent", "Agent initialized",
@ -76,6 +120,65 @@ func agentCmd(message, sessionKey, model string, debug bool) error {
return nil
}
// applyToolAllowlist disables all tools, then enables only the listed ones.
func applyToolAllowlist(cfg *config.Config, allowed []string) {
allowSet := make(map[string]bool, len(allowed))
for _, t := range allowed {
allowSet[t] = true
}
cfg.Tools.ReadFile.Enabled = allowSet["read_file"]
cfg.Tools.WriteFile.Enabled = allowSet["write_file"]
cfg.Tools.EditFile.Enabled = allowSet["edit_file"]
cfg.Tools.AppendFile.Enabled = allowSet["append_file"]
cfg.Tools.ListDir.Enabled = allowSet["list_dir"]
cfg.Tools.Exec.Enabled = allowSet["exec"]
cfg.Tools.Spawn.Enabled = allowSet["spawn"]
cfg.Tools.Cron.Enabled = allowSet["cron"]
cfg.Tools.Web.Enabled = allowSet["web"] || allowSet["web_search"]
cfg.Tools.WebFetch.Enabled = allowSet["web_fetch"]
cfg.Tools.Skills.Enabled = allowSet["skills"]
cfg.Tools.FindSkills.Enabled = allowSet["find_skills"]
cfg.Tools.InstallSkill.Enabled = allowSet["install_skill"]
cfg.Tools.Subagent.Enabled = allowSet["subagent"]
cfg.Tools.Message.Enabled = allowSet["message"]
cfg.Tools.MCP.Enabled = allowSet["mcp"]
cfg.Tools.I2C.Enabled = allowSet["i2c"]
cfg.Tools.SPI.Enabled = allowSet["spi"]
}
// applySkillsFilter injects a skills filter into the agent config.
func applySkillsFilter(cfg *config.Config, skills []string) {
if len(cfg.Agents.List) == 0 {
// Create an implicit main agent with skills filter
cfg.Agents.List = []config.AgentConfig{
{ID: "main", Default: true, Skills: skills},
}
} else {
// Apply to all agents
for i := range cfg.Agents.List {
cfg.Agents.List[i].Skills = skills
}
}
}
// copyBootstrapFiles copies recognized bootstrap files (AGENTS.md, IDENTITY.md,
// SOUL.md, USER.md) from srcDir into the workspace directory.
func copyBootstrapFiles(srcDir, workspace string) {
bootstrapFiles := []string{"AGENTS.md", "IDENTITY.md", "SOUL.md", "USER.md"}
for _, filename := range bootstrapFiles {
srcPath := filepath.Join(srcDir, filename)
data, err := os.ReadFile(srcPath)
if err != nil {
continue // file not present in config-dir, skip
}
dstPath := filepath.Join(workspace, filename)
if err := os.WriteFile(dstPath, data, 0o644); err != nil {
fmt.Fprintf(os.Stderr, "Warning: failed to write %s: %v\n", dstPath, err)
}
}
}
func interactiveMode(agentLoop *agent.AgentLoop, sessionKey string) {
prompt := fmt.Sprintf("%s You: ", internal.Logo)

View file

@ -17,6 +17,7 @@ import (
_ "github.com/sipeed/picoclaw/pkg/channels/discord"
_ "github.com/sipeed/picoclaw/pkg/channels/feishu"
_ "github.com/sipeed/picoclaw/pkg/channels/line"
_ "github.com/sipeed/picoclaw/pkg/channels/magicform"
_ "github.com/sipeed/picoclaw/pkg/channels/maixcam"
_ "github.com/sipeed/picoclaw/pkg/channels/onebot"
_ "github.com/sipeed/picoclaw/pkg/channels/pico"

View file

@ -164,6 +164,15 @@
"max_steps": 10,
"welcome_message": "Hello! I'm your AI assistant. How can I help you today?",
"reasoning_channel_id": ""
},
"magicform": {
"_comment": "MagicForm - Webhook-based channel for MagicForm agentic task delegation",
"enabled": false,
"token": "",
"backend_url": "",
"webhook_path": "/hooks/magicform",
"workspace_root": "/data/workspaces",
"allow_from": []
}
},
"providers": {

View file

@ -0,0 +1,30 @@
{
"_comment": "Workspace-local config override. Place as config.json in a config directory passed via --config-dir (CLI) or configDir (webhook). Only the fields below are honored; gateway, heartbeat, devices, and providers (legacy) are ignored.",
"model_list": [
{
"model_name": "main",
"model": "anthropic/claude-sonnet-4.6",
"api_key": "sk-ant-your-key",
"api_base": "https://api.anthropic.com/v1"
}
],
"agents": {
"defaults": {
"model_name": "main",
"max_tokens": 4096,
"temperature": 0.7,
"max_tool_iterations": 20
}
},
"session": {
"dm_scope": "per-channel-peer"
},
"tools": {
"exec": {
"enabled": false
},
"web": {
"enabled": true
}
}
}

29
pkg/agent/bootstrap.go Normal file
View file

@ -0,0 +1,29 @@
package agent
import (
"os"
"path/filepath"
"github.com/sipeed/picoclaw/pkg/logger"
)
// bootstrapFiles are the recognized bootstrap file names copied from a config
// directory into the agent workspace.
var bootstrapFiles = []string{"AGENTS.md", "IDENTITY.md", "SOUL.md", "USER.md"}
// CopyBootstrapFiles copies recognized bootstrap files from srcDir into dstDir.
// Missing files in srcDir are silently skipped.
func CopyBootstrapFiles(srcDir, dstDir string) {
for _, filename := range bootstrapFiles {
srcPath := filepath.Join(srcDir, filename)
data, err := os.ReadFile(srcPath)
if err != nil {
continue // file not present, skip
}
dstPath := filepath.Join(dstDir, filename)
if err := os.WriteFile(dstPath, data, 0o644); err != nil {
logger.WarnCF("agent", "Failed to write bootstrap file",
map[string]any{"path": dstPath, "error": err.Error()})
}
}
}

View file

@ -39,6 +39,54 @@ type ContextBuilder struct {
// build time. This catches nested file creations/deletions/mtime changes
// that may not update the top-level skill root directory mtime.
skillFilesAtCache map[string]time.Time
// skillsFilter limits which skills are included in the system prompt.
// When non-empty, only skills whose name matches an entry are included.
// ["*"] means include all skills.
skillsFilter []string
}
// SetSkillsFilter sets the skills filter on the context builder.
// When non-empty, only skills matching these names are included in the system prompt.
// Use ["*"] to include all skills.
func (cb *ContextBuilder) SetSkillsFilter(filter []string) {
cb.skillsFilter = filter
cb.InvalidateCache()
}
// buildFilteredSkillsSummary returns the skills summary, filtered by skillsFilter
// if one is set. When no filter is set, all skills are included.
func (cb *ContextBuilder) buildFilteredSkillsSummary() string {
if len(cb.skillsFilter) == 0 {
return cb.skillsLoader.BuildSkillsSummary()
}
// Check for wildcard
for _, f := range cb.skillsFilter {
if f == "*" {
return cb.skillsLoader.BuildSkillsSummary()
}
}
// Build filter set
filterSet := make(map[string]bool, len(cb.skillsFilter))
for _, f := range cb.skillsFilter {
filterSet[f] = true
}
allSkills := cb.skillsLoader.ListSkills()
var filtered []skills.SkillInfo
for _, s := range allSkills {
if filterSet[s.Name] {
filtered = append(filtered, s)
}
}
if len(filtered) == 0 {
return ""
}
return skills.FormatSkillsSummary(filtered)
}
func getGlobalConfigDir() string {
@ -107,7 +155,7 @@ func (cb *ContextBuilder) BuildSystemPrompt() string {
}
// Skills - show summary, AI can read full content with read_file tool
skillsSummary := cb.skillsLoader.BuildSkillsSummary()
skillsSummary := cb.buildFilteredSkillsSummary()
if skillsSummary != "" {
parts = append(parts, fmt.Sprintf(`# Skills

View file

@ -98,6 +98,7 @@ func NewAgentInstance(
contextBuilder := NewContextBuilder(workspace)
// SkillsFilter will be applied after we know the agent config
agentID := routing.DefaultAgentID
agentName := ""
var subagents *config.SubagentsConfig
@ -110,6 +111,11 @@ func NewAgentInstance(
skillsFilter = agentCfg.Skills
}
// Apply skills filter to context builder so only matching skills appear in the prompt
if len(skillsFilter) > 0 {
contextBuilder.SetSkillsFilter(skillsFilter)
}
maxIter := defaults.MaxToolIterations
if maxIter == 0 {
maxIter = 20

View file

@ -11,6 +11,7 @@ import (
"encoding/json"
"errors"
"fmt"
"os"
"path/filepath"
"regexp"
"strings"
@ -29,6 +30,7 @@ import (
"github.com/sipeed/picoclaw/pkg/media"
"github.com/sipeed/picoclaw/pkg/providers"
"github.com/sipeed/picoclaw/pkg/routing"
"github.com/sipeed/picoclaw/pkg/session"
"github.com/sipeed/picoclaw/pkg/skills"
"github.com/sipeed/picoclaw/pkg/state"
"github.com/sipeed/picoclaw/pkg/tools"
@ -52,15 +54,31 @@ type AgentLoop struct {
// processOptions configures how a message is processed
type processOptions struct {
SessionKey string // Session identifier for history/context
Channel string // Target channel for tool execution
ChatID string // Target chat ID for tool execution
UserMessage string // User message content (may include prefix)
Media []string // media:// refs from inbound message
DefaultResponse string // Response when LLM returns empty
EnableSummary bool // Whether to trigger summarization
SendResponse bool // Whether to send response via bus
NoHistory bool // If true, don't load session history (for heartbeat)
SessionKey string // Session identifier for history/context
Channel string // Target channel for tool execution
ChatID string // Target chat ID for tool execution
UserMessage string // User message content (may include prefix)
Media []string // media:// refs from inbound message
DefaultResponse string // Response when LLM returns empty
EnableSummary bool // Whether to trigger summarization
SendResponse bool // Whether to send response via bus
NoHistory bool // If true, don't load session history (for heartbeat)
WorkspaceOverride string // If set, use this workspace instead of agent.Workspace
ConfigDir string // If set, config directory for workspace-local overrides
AllowedTools []string // If non-empty, only these tools are active for this request
AllowedSkills []string // If non-empty, only these skills are loaded for this request
// effSessions and effContextBuilder are set by runAgentLoop when a workspace
// override is active. All downstream code (runLLMIteration, forceCompressionWith
// retry) MUST use these instead of agent.Sessions / agent.ContextBuilder.
effSessions *session.SessionManager
effContextBuilder *ContextBuilder
// effProvider and effModel are set by runAgentLoop when a workspace config
// provides per-request provider overrides. All LLM call sites (runLLMIteration,
// summarization) MUST use these instead of agent.Provider / agent.Model.
effProvider providers.LLMProvider
effModel string
}
const (
@ -615,15 +633,39 @@ func (al *AgentLoop) processMessage(ctx context.Context, msg bus.InboundMessage)
"route_channel": route.Channel,
})
// Extract overrides from metadata (used by magicform channel / gateway mode)
workspaceOverride := msg.Metadata["workspace_override"]
configDir := msg.Metadata["config_dir"]
var allowedTools, allowedSkills []string
if v := msg.Metadata["allowed_tools"]; v != "" {
for _, t := range strings.Split(v, ",") {
if s := strings.TrimSpace(t); s != "" {
allowedTools = append(allowedTools, s)
}
}
}
if v := msg.Metadata["allowed_skills"]; v != "" {
for _, s := range strings.Split(v, ",") {
if trimmed := strings.TrimSpace(s); trimmed != "" {
allowedSkills = append(allowedSkills, trimmed)
}
}
}
return al.runAgentLoop(ctx, agent, processOptions{
SessionKey: sessionKey,
Channel: msg.Channel,
ChatID: msg.ChatID,
UserMessage: msg.Content,
Media: msg.Media,
DefaultResponse: defaultResponse,
EnableSummary: true,
SendResponse: false,
SessionKey: sessionKey,
Channel: msg.Channel,
ChatID: msg.ChatID,
UserMessage: msg.Content,
Media: msg.Media,
DefaultResponse: defaultResponse,
EnableSummary: true,
SendResponse: false,
WorkspaceOverride: workspaceOverride,
ConfigDir: configDir,
AllowedTools: allowedTools,
AllowedSkills: allowedSkills,
})
}
@ -741,14 +783,65 @@ func (al *AgentLoop) runAgentLoop(
}
}
// Resolve effective sessions and context builder.
// When a workspace override is provided (e.g. from magicform channel),
// create temporary instances pointing at the override path for full isolation.
effSessions := agent.Sessions
effContextBuilder := agent.ContextBuilder
if opts.WorkspaceOverride != "" {
wp := opts.WorkspaceOverride
os.MkdirAll(wp, 0o755)
effSessions = session.NewSessionManager(filepath.Join(wp, "sessions"))
effContextBuilder = NewContextBuilder(wp)
}
// Copy bootstrap files from config-dir to workspace
if opts.ConfigDir != "" && opts.WorkspaceOverride != "" {
CopyBootstrapFiles(opts.ConfigDir, opts.WorkspaceOverride)
}
// Load workspace-local config.json for per-request overrides
configSource := opts.ConfigDir
if configSource == "" {
configSource = opts.WorkspaceOverride // fallback: check workspace itself
}
if configSource != "" {
if wc, err := config.LoadWorkspaceConfig(configSource); err != nil {
logger.WarnCF("agent", "Failed to load workspace config",
map[string]any{"path": configSource, "error": err.Error()})
} else if wc != nil {
tmpCfg := al.cfg.Clone()
tmpCfg.MergeWorkspaceConfig(wc)
if tmpCfg.Agents.Defaults.GetModelName() == "" {
tmpCfg.Agents.Defaults.ModelName = agent.Model
}
if provider, modelID, err := providers.CreateProvider(tmpCfg); err != nil {
logger.ErrorCF("agent", "Failed to create workspace provider",
map[string]any{"path": configSource, "error": err.Error()})
} else {
opts.effProvider = provider
opts.effModel = modelID
if sp, ok := provider.(providers.StatefulProvider); ok {
defer sp.Close()
}
}
}
}
// Apply skills filter unconditionally — works with or without workspace override
if len(opts.AllowedSkills) > 0 {
effContextBuilder.SetSkillsFilter(opts.AllowedSkills)
}
// 1. Build messages (skip history for heartbeat)
var history []providers.Message
var summary string
if !opts.NoHistory {
history = agent.Sessions.GetHistory(opts.SessionKey)
summary = agent.Sessions.GetSummary(opts.SessionKey)
history = effSessions.GetHistory(opts.SessionKey)
summary = effSessions.GetSummary(opts.SessionKey)
}
messages := agent.ContextBuilder.BuildMessages(
messages := effContextBuilder.BuildMessages(
history,
summary,
opts.UserMessage,
@ -762,7 +855,11 @@ func (al *AgentLoop) runAgentLoop(
messages = resolveMediaRefs(messages, al.mediaStore, maxMediaSize)
// 2. Save user message to session
agent.Sessions.AddMessage(opts.SessionKey, "user", opts.UserMessage)
effSessions.AddMessage(opts.SessionKey, "user", opts.UserMessage)
// Store effective sessions/context on opts so runLLMIteration can use them
opts.effSessions = effSessions
opts.effContextBuilder = effContextBuilder
// 3. Run LLM iteration loop
finalContent, iteration, err := al.runLLMIteration(ctx, agent, messages, opts)
@ -779,12 +876,20 @@ func (al *AgentLoop) runAgentLoop(
}
// 5. Save final assistant message to session
agent.Sessions.AddMessage(opts.SessionKey, "assistant", finalContent)
agent.Sessions.Save(opts.SessionKey)
effSessions.AddMessage(opts.SessionKey, "assistant", finalContent)
effSessions.Save(opts.SessionKey)
// 6. Optional: summarization
if opts.EnableSummary {
al.maybeSummarize(agent, opts.SessionKey, opts.Channel, opts.ChatID)
al.maybeSummarizeWith(
effSessions,
agent,
opts.SessionKey,
opts.Channel,
opts.ChatID,
opts.effProvider,
opts.effModel,
)
}
// 7. Optional: send response via bus
@ -875,11 +980,32 @@ func (al *AgentLoop) runLLMIteration(
iteration := 0
var finalContent string
// Resolve effective provider/model — workspace config overrides win
effProvider := agent.Provider
effModel := agent.Model
if opts.effProvider != nil {
effProvider = opts.effProvider
}
if opts.effModel != "" {
effModel = opts.effModel
}
// Determine effective model tier for this conversation turn.
// selectCandidates evaluates routing once and the decision is sticky for
// all tool-follow-up iterations within the same turn so that a multi-step
// tool chain doesn't switch models mid-way through.
activeCandidates, activeModel := al.selectCandidates(agent, opts.UserMessage, messages)
var activeCandidates []providers.FallbackCandidate
var activeModel string
if opts.effProvider != nil {
// Workspace overrides the provider — skip routing and fallback
// candidates since they may reference different provider credentials.
activeModel = effModel
} else {
activeCandidates, activeModel = al.selectCandidates(agent, opts.UserMessage, messages)
if opts.effModel != "" {
activeModel = effModel
}
}
for iteration < agent.MaxIterations {
iteration++
@ -891,8 +1017,21 @@ func (al *AgentLoop) runLLMIteration(
"max": agent.MaxIterations,
})
// Build tool definitions
// Build tool definitions, filtered by AllowedTools if set
providerToolDefs := agent.Tools.ToProviderDefs()
if len(opts.AllowedTools) > 0 {
allowSet := make(map[string]bool, len(opts.AllowedTools))
for _, t := range opts.AllowedTools {
allowSet[t] = true
}
filtered := providerToolDefs[:0]
for _, td := range providerToolDefs {
if allowSet[td.Function.Name] {
filtered = append(filtered, td)
}
}
providerToolDefs = filtered
}
// Log LLM request details
logger.DebugCF("agent", "LLM request",
@ -927,7 +1066,7 @@ func (al *AgentLoop) runLLMIteration(
// parseThinkingLevel guarantees ThinkingOff for empty/unknown values,
// so checking != ThinkingOff is sufficient.
if agent.ThinkingLevel != ThinkingOff {
if tc, ok := agent.Provider.(providers.ThinkingCapable); ok && tc.SupportsThinking() {
if tc, ok := effProvider.(providers.ThinkingCapable); ok && tc.SupportsThinking() {
llmOpts["thinking_level"] = string(agent.ThinkingLevel)
} else {
logger.WarnCF("agent", "thinking_level is set but current provider does not support it, ignoring",
@ -941,7 +1080,7 @@ func (al *AgentLoop) runLLMIteration(
ctx,
activeCandidates,
func(ctx context.Context, provider, model string) (*providers.LLMResponse, error) {
return agent.Provider.Chat(ctx, messages, providerToolDefs, model, llmOpts)
return effProvider.Chat(ctx, messages, providerToolDefs, model, llmOpts)
},
)
if fbErr != nil {
@ -957,7 +1096,7 @@ func (al *AgentLoop) runLLMIteration(
}
return fbResult.Response, nil
}
return agent.Provider.Chat(ctx, messages, providerToolDefs, activeModel, llmOpts)
return effProvider.Chat(ctx, messages, providerToolDefs, activeModel, llmOpts)
}
// Retry loop for context/token errors
@ -1017,10 +1156,10 @@ func (al *AgentLoop) runLLMIteration(
})
}
al.forceCompression(agent, opts.SessionKey)
newHistory := agent.Sessions.GetHistory(opts.SessionKey)
newSummary := agent.Sessions.GetSummary(opts.SessionKey)
messages = agent.ContextBuilder.BuildMessages(
al.forceCompressionWith(opts.effSessions, agent, opts.SessionKey)
newHistory := opts.effSessions.GetHistory(opts.SessionKey)
newSummary := opts.effSessions.GetSummary(opts.SessionKey)
messages = opts.effContextBuilder.BuildMessages(
newHistory, newSummary, "",
nil, opts.Channel, opts.ChatID,
)
@ -1117,7 +1256,7 @@ func (al *AgentLoop) runLLMIteration(
messages = append(messages, assistantMsg)
// Save assistant message with tool calls to session
agent.Sessions.AddFullMessage(opts.SessionKey, assistantMsg)
opts.effSessions.AddFullMessage(opts.SessionKey, assistantMsg)
// Execute tool calls in parallel
type indexedAgentResult struct {
@ -1144,6 +1283,24 @@ func (al *AgentLoop) runLLMIteration(
"iteration": iteration,
})
// Enforce tool allowlist at execution time (defense-in-depth)
if len(opts.AllowedTools) > 0 {
allowed := false
for _, t := range opts.AllowedTools {
if t == tc.Name {
allowed = true
break
}
}
if !allowed {
agentResults[idx].result = &tools.ToolResult{
ForLLM: fmt.Sprintf("Tool %q is not allowed for this request", tc.Name),
IsError: true,
}
return
}
}
// Create async callback for tools that implement AsyncExecutor
asyncCallback := func(callbackCtx context.Context, result *tools.ToolResult) {
if !result.Silent && result.ForUser != "" {
@ -1219,7 +1376,7 @@ func (al *AgentLoop) runLLMIteration(
messages = append(messages, toolResultMsg)
// Save tool result message to session
agent.Sessions.AddFullMessage(opts.SessionKey, toolResultMsg)
opts.effSessions.AddFullMessage(opts.SessionKey, toolResultMsg)
}
}
@ -1264,9 +1421,16 @@ func (al *AgentLoop) selectCandidates(
return agent.LightCandidates, agent.Router.LightModel()
}
// maybeSummarize triggers summarization if the session history exceeds thresholds.
func (al *AgentLoop) maybeSummarize(agent *AgentInstance, sessionKey, channel, chatID string) {
newHistory := agent.Sessions.GetHistory(sessionKey)
// maybeSummarizeWith triggers summarization if the session history exceeds thresholds.
// and optional per-request provider/model overrides.
func (al *AgentLoop) maybeSummarizeWith(
sessions *session.SessionManager,
agent *AgentInstance,
sessionKey, channel, chatID string,
effProvider providers.LLMProvider,
effModel string,
) {
newHistory := sessions.GetHistory(sessionKey)
tokenEstimate := al.estimateTokens(newHistory)
threshold := agent.ContextWindow * agent.SummarizeTokenPercent / 100
@ -1276,16 +1440,16 @@ func (al *AgentLoop) maybeSummarize(agent *AgentInstance, sessionKey, channel, c
go func() {
defer al.summarizing.Delete(summarizeKey)
logger.Debug("Memory threshold reached. Optimizing conversation history...")
al.summarizeSession(agent, sessionKey)
al.summarizeSessionWith(sessions, agent, sessionKey, effProvider, effModel)
}()
}
}
}
// forceCompression aggressively reduces context when the limit is hit.
// forceCompressionWith aggressively reduces context when the limit is hit.
// It drops the oldest 50% of messages (keeping system prompt and last user message).
func (al *AgentLoop) forceCompression(agent *AgentInstance, sessionKey string) {
history := agent.Sessions.GetHistory(sessionKey)
func (al *AgentLoop) forceCompressionWith(sessions *session.SessionManager, agent *AgentInstance, sessionKey string) {
history := sessions.GetHistory(sessionKey)
if len(history) <= 4 {
return
}
@ -1325,8 +1489,8 @@ func (al *AgentLoop) forceCompression(agent *AgentInstance, sessionKey string) {
newHistory = append(newHistory, history[len(history)-1]) // Last message
// Update session
agent.Sessions.SetHistory(sessionKey, newHistory)
agent.Sessions.Save(sessionKey)
sessions.SetHistory(sessionKey, newHistory)
sessions.Save(sessionKey)
logger.WarnCF("agent", "Forced compression executed", map[string]any{
"session_key": sessionKey,
@ -1422,13 +1586,29 @@ func formatToolsForLog(toolDefs []providers.ToolDefinition) string {
return sb.String()
}
// summarizeSession summarizes the conversation history for a session.
func (al *AgentLoop) summarizeSession(agent *AgentInstance, sessionKey string) {
// summarizeSessionWith summarizes the conversation history for a session.
func (al *AgentLoop) summarizeSessionWith(
sessions *session.SessionManager,
agent *AgentInstance,
sessionKey string,
effProvider providers.LLMProvider,
effModel string,
) {
ctx, cancel := context.WithTimeout(context.Background(), 120*time.Second)
defer cancel()
history := agent.Sessions.GetHistory(sessionKey)
summary := agent.Sessions.GetSummary(sessionKey)
// Resolve effective provider/model
sumProvider := agent.Provider
sumModel := agent.Model
if effProvider != nil {
sumProvider = effProvider
}
if effModel != "" {
sumModel = effModel
}
history := sessions.GetHistory(sessionKey)
summary := sessions.GetSummary(sessionKey)
// Keep last 4 messages for continuity
if len(history) <= 4 {
@ -1465,19 +1645,19 @@ func (al *AgentLoop) summarizeSession(agent *AgentInstance, sessionKey string) {
part1 := validMessages[:mid]
part2 := validMessages[mid:]
s1, _ := al.summarizeBatch(ctx, agent, part1, "")
s2, _ := al.summarizeBatch(ctx, agent, part2, "")
s1, _ := al.summarizeBatch(ctx, sumProvider, sumModel, agent.ID, part1, "")
s2, _ := al.summarizeBatch(ctx, sumProvider, sumModel, agent.ID, part2, "")
mergePrompt := fmt.Sprintf(
"Merge these two conversation summaries into one cohesive summary:\n\n1: %s\n\n2: %s",
s1,
s2,
)
resp, err := agent.Provider.Chat(
resp, err := sumProvider.Chat(
ctx,
[]providers.Message{{Role: "user", Content: mergePrompt}},
nil,
agent.Model,
sumModel,
map[string]any{
"max_tokens": 1024,
"temperature": 0.3,
@ -1490,7 +1670,7 @@ func (al *AgentLoop) summarizeSession(agent *AgentInstance, sessionKey string) {
finalSummary = s1 + " " + s2
}
} else {
finalSummary, _ = al.summarizeBatch(ctx, agent, validMessages, summary)
finalSummary, _ = al.summarizeBatch(ctx, sumProvider, sumModel, agent.ID, validMessages, summary)
}
if omitted && finalSummary != "" {
@ -1498,16 +1678,18 @@ func (al *AgentLoop) summarizeSession(agent *AgentInstance, sessionKey string) {
}
if finalSummary != "" {
agent.Sessions.SetSummary(sessionKey, finalSummary)
agent.Sessions.TruncateHistory(sessionKey, 4)
agent.Sessions.Save(sessionKey)
sessions.SetSummary(sessionKey, finalSummary)
sessions.TruncateHistory(sessionKey, 4)
sessions.Save(sessionKey)
}
}
// summarizeBatch summarizes a batch of messages.
// summarizeBatch summarizes a batch of messages using the given provider/model.
func (al *AgentLoop) summarizeBatch(
ctx context.Context,
agent *AgentInstance,
provider providers.LLMProvider,
model string,
agentID string,
batch []providers.Message,
existingSummary string,
) (string, error) {
@ -1526,15 +1708,15 @@ func (al *AgentLoop) summarizeBatch(
}
prompt := sb.String()
response, err := agent.Provider.Chat(
response, err := provider.Chat(
ctx,
[]providers.Message{{Role: "user", Content: prompt}},
nil,
agent.Model,
model,
map[string]any{
"max_tokens": 1024,
"temperature": 0.3,
"prompt_cache_key": agent.ID,
"prompt_cache_key": agentID,
},
)
if err != nil {

View file

@ -18,8 +18,12 @@ import (
"strconv"
"strings"
"time"
"github.com/sipeed/picoclaw/pkg/logger"
)
const oauthMaxResponseSize int64 = 1 << 20 // 1 MB — more than sufficient for any OAuth response
type OAuthProviderConfig struct {
Issuer string
ClientID string
@ -33,7 +37,7 @@ type OAuthProviderConfig struct {
func OpenAIOAuthConfig() OAuthProviderConfig {
return OAuthProviderConfig{
Issuer: "https://auth.openai.com",
ClientID: "app_EMoamEEZ73f0CkXaXp7hrann",
ClientID: getEnvOrDefault("PICOCLAW_OPENAI_CLIENT_ID", "app_EMoamEEZ73f0CkXaXp7hrann"),
Scopes: "openid profile email offline_access",
Originator: "codex_cli_rs",
Port: 1455,
@ -43,11 +47,16 @@ func OpenAIOAuthConfig() OAuthProviderConfig {
// GoogleAntigravityOAuthConfig returns the OAuth configuration for Google Cloud Code Assist (Antigravity).
// Client credentials are the same ones used by OpenCode/pi-ai for Cloud Code Assist access.
func GoogleAntigravityOAuthConfig() OAuthProviderConfig {
// These are the same client credentials used by the OpenCode antigravity plugin.
clientID := decodeBase64(
"MTA3MTAwNjA2MDU5MS10bWhzc2luMmgyMWxjcmUyMzV2dG9sb2poNGc0MDNlcC5hcHBzLmdvb2dsZXVzZXJjb250ZW50LmNvbQ==",
clientID := getEnvOrDefault(
"PICOCLAW_GOOGLE_CLIENT_ID",
decodeBase64(
"MTA3MTAwNjA2MDU5MS10bWhzc2luMmgyMWxjcmUyMzV2dG9sb2poNGc0MDNlcC5hcHBzLmdvb2dsZXVzZXJjb250ZW50LmNvbQ==",
),
)
clientSecret := getEnvOrDefault(
"PICOCLAW_GOOGLE_CLIENT_SECRET",
decodeBase64("R09DU1BYLUs1OEZXUjQ4NkxkTEoxbUxCOHNYQzR6NnFEQWY="),
)
clientSecret := decodeBase64("R09DU1BYLUs1OEZXUjQ4NkxkTEoxbUxCOHNYQzR6NnFEQWY=")
return OAuthProviderConfig{
Issuer: "https://accounts.google.com/o/oauth2/v2",
TokenURL: "https://oauth2.googleapis.com/token",
@ -58,6 +67,13 @@ func GoogleAntigravityOAuthConfig() OAuthProviderConfig {
}
}
func getEnvOrDefault(key, fallback string) string {
if v := os.Getenv(key); v != "" {
return v
}
return fallback
}
func decodeBase64(s string) string {
data, err := base64.StdEncoding.DecodeString(s)
if err != nil {
@ -212,12 +228,14 @@ func RequestDeviceCode(cfg OAuthProviderConfig) (*DeviceCodeInfo, error) {
}
defer resp.Body.Close()
body, err := io.ReadAll(resp.Body)
if err != nil {
return nil, fmt.Errorf("reading device code response: %w", err)
}
body, _ := io.ReadAll(io.LimitReader(resp.Body, oauthMaxResponseSize))
if resp.StatusCode != http.StatusOK {
return nil, fmt.Errorf("device code request failed: %s", string(body))
logger.DebugCF(
"auth",
"device code request failed",
map[string]any{"status": resp.StatusCode, "body": string(body)},
)
return nil, fmt.Errorf("device code request failed (HTTP %d)", resp.StatusCode)
}
deviceResp, err := parseDeviceCodeResponse(body)
@ -303,12 +321,14 @@ func LoginDeviceCode(cfg OAuthProviderConfig) (*AuthCredential, error) {
}
defer resp.Body.Close()
body, err := io.ReadAll(resp.Body)
if err != nil {
return nil, fmt.Errorf("reading device code response: %w", err)
}
body, _ := io.ReadAll(io.LimitReader(resp.Body, oauthMaxResponseSize))
if resp.StatusCode != http.StatusOK {
return nil, fmt.Errorf("device code request failed: %s", string(body))
logger.DebugCF(
"auth",
"device code request failed",
map[string]any{"status": resp.StatusCode, "body": string(body)},
)
return nil, fmt.Errorf("device code request failed (HTTP %d)", resp.StatusCode)
}
deviceResp, err := parseDeviceCodeResponse(body)
@ -366,10 +386,7 @@ func pollDeviceCode(cfg OAuthProviderConfig, deviceAuthID, userCode string) (*Au
return nil, fmt.Errorf("pending")
}
body, err := io.ReadAll(resp.Body)
if err != nil {
return nil, fmt.Errorf("reading device token response: %w", err)
}
body, _ := io.ReadAll(io.LimitReader(resp.Body, oauthMaxResponseSize))
var tokenResp struct {
AuthorizationCode string `json:"authorization_code"`
@ -410,12 +427,10 @@ func RefreshAccessToken(cred *AuthCredential, cfg OAuthProviderConfig) (*AuthCre
}
defer resp.Body.Close()
body, err := io.ReadAll(resp.Body)
if err != nil {
return nil, fmt.Errorf("reading token refresh response: %w", err)
}
body, _ := io.ReadAll(io.LimitReader(resp.Body, oauthMaxResponseSize))
if resp.StatusCode != http.StatusOK {
return nil, fmt.Errorf("token refresh failed: %s", string(body))
logger.DebugCF("auth", "token refresh failed", map[string]any{"status": resp.StatusCode, "body": string(body)})
return nil, fmt.Errorf("token refresh failed (HTTP %d)", resp.StatusCode)
}
refreshed, err := parseTokenResponse(body, cred.Provider)
@ -506,12 +521,10 @@ func ExchangeCodeForTokens(cfg OAuthProviderConfig, code, codeVerifier, redirect
}
defer resp.Body.Close()
body, err := io.ReadAll(resp.Body)
if err != nil {
return nil, fmt.Errorf("reading token exchange response: %w", err)
}
body, _ := io.ReadAll(io.LimitReader(resp.Body, oauthMaxResponseSize))
if resp.StatusCode != http.StatusOK {
return nil, fmt.Errorf("token exchange failed: %s", string(body))
logger.DebugCF("auth", "token exchange failed", map[string]any{"status": resp.StatusCode, "body": string(body)})
return nil, fmt.Errorf("token exchange failed (HTTP %d)", resp.StatusCode)
}
return parseTokenResponse(body, provider)

View file

@ -301,6 +301,10 @@ func (c *BaseChannel) HandleMessage(
}
}
// Bus returns the underlying MessageBus. This is used by channels that need to
// publish directly (e.g. to set SessionKey on InboundMessage).
func (c *BaseChannel) Bus() *bus.MessageBus { return c.bus }
func (c *BaseChannel) SetRunning(running bool) {
c.running.Store(running)
}

View file

@ -0,0 +1,13 @@
package magicform
import (
"github.com/sipeed/picoclaw/pkg/bus"
"github.com/sipeed/picoclaw/pkg/channels"
"github.com/sipeed/picoclaw/pkg/config"
)
func init() {
channels.RegisterFactory("magicform", func(cfg *config.Config, b *bus.MessageBus) (channels.Channel, error) {
return NewMagicFormChannel(cfg.Channels.MagicForm, b)
})
}

View file

@ -0,0 +1,420 @@
package magicform
import (
"bytes"
"context"
"crypto/subtle"
"encoding/json"
"fmt"
"io"
"net/http"
"path/filepath"
"strings"
"sync"
"time"
"github.com/sipeed/picoclaw/pkg/bus"
"github.com/sipeed/picoclaw/pkg/channels"
"github.com/sipeed/picoclaw/pkg/config"
"github.com/sipeed/picoclaw/pkg/logger"
)
// WebhookPayload is the inbound payload from MagicForm.
type WebhookPayload struct {
StackID string `json:"stackId"`
ConversationID string `json:"conversationId"`
UserID string `json:"userId"`
Message string `json:"message"`
Workspace string `json:"workspace"` // e.g. "s1/c1" — agent working directory (relative to workspace_root)
ConfigDir string `json:"configDir,omitempty"` // e.g. "s1/config" — pre-provisioned config directory (relative to workspace_root)
CallbackURL string `json:"callbackUrl"`
// Tool/skill filtering
AllowedTools []string `json:"allowedTools,omitempty"` // Tool allowlist (empty = all)
AllowedSkills []string `json:"allowedSkills,omitempty"` // Skill filter (empty = all)
}
// CallbackPayload is the outbound payload sent back to MagicForm.
type CallbackPayload struct {
StackID string `json:"stackId"`
ConversationID string `json:"conversationId"`
Response string `json:"response"`
Type string `json:"type"` // "final"
}
// requestContext stores per-request state so Send() can resolve callback info.
type requestContext struct {
stackID string
conversationID string
userID string
callbackURL string
createdAt time.Time
}
// MagicFormChannel implements the MagicForm channel plugin.
type MagicFormChannel struct {
*channels.BaseChannel
config config.MagicFormConfig
httpClient *http.Client
requests sync.Map // chatID → *requestContext
ctx context.Context
cancel context.CancelFunc
}
// NewMagicFormChannel creates a new MagicForm channel.
func NewMagicFormChannel(cfg config.MagicFormConfig, msgBus *bus.MessageBus) (*MagicFormChannel, error) {
base := channels.NewBaseChannel(
"magicform",
cfg,
msgBus,
cfg.AllowFrom,
)
ctx, cancel := context.WithCancel(context.Background())
ch := &MagicFormChannel{
BaseChannel: base,
config: cfg,
httpClient: &http.Client{
Timeout: 30 * time.Second,
},
ctx: ctx,
cancel: cancel,
}
base.SetOwner(ch)
return ch, nil
}
// Start begins the channel and starts the TTL cleanup goroutine.
func (c *MagicFormChannel) Start(_ context.Context) error {
c.SetRunning(true)
// Background goroutine to clean up stale request contexts
go c.cleanupLoop()
logger.InfoCF("magicform", "MagicForm channel started", nil)
return nil
}
// Stop shuts down the channel.
func (c *MagicFormChannel) Stop(_ context.Context) error {
c.cancel()
c.SetRunning(false)
logger.InfoCF("magicform", "MagicForm channel stopped", nil)
return nil
}
// WebhookPath returns the HTTP path for the inbound webhook.
func (c *MagicFormChannel) WebhookPath() string {
if c.config.WebhookPath != "" {
return c.config.WebhookPath
}
return "/hooks/magicform"
}
// HealthPath returns the HTTP path for the health check endpoint.
func (c *MagicFormChannel) HealthPath() string {
return "/health/magicform"
}
// HealthHandler handles the health check HTTP request.
func (c *MagicFormChannel) HealthHandler(w http.ResponseWriter, _ *http.Request) {
w.Header().Set("Content-Type", "application/json")
w.WriteHeader(http.StatusOK)
json.NewEncoder(w).Encode(map[string]string{
"status": "ok",
"channel": "magicform",
})
}
// maxWebhookBodySize is the maximum allowed size for inbound webhook payloads (1 MB).
const maxWebhookBodySize = 1 << 20
// ServeHTTP handles inbound webhook requests from MagicForm.
func (c *MagicFormChannel) ServeHTTP(w http.ResponseWriter, r *http.Request) {
if r.Method != http.MethodPost {
http.Error(w, "Method not allowed", http.StatusMethodNotAllowed)
return
}
// Limit request body size to prevent memory exhaustion
r.Body = http.MaxBytesReader(w, r.Body, maxWebhookBodySize)
body, err := io.ReadAll(r.Body)
if err != nil {
http.Error(w, "Request body too large", http.StatusRequestEntityTooLarge)
return
}
defer r.Body.Close()
// Verify Bearer token
if !c.verifyToken(r) {
http.Error(w, "Unauthorized", http.StatusUnauthorized)
return
}
var payload WebhookPayload
if err := json.Unmarshal(body, &payload); err != nil {
http.Error(w, "Invalid JSON", http.StatusBadRequest)
return
}
// Validate required fields
if payload.StackID == "" || payload.ConversationID == "" || payload.Message == "" {
http.Error(w, "Missing required fields: stackId, conversationId, message", http.StatusBadRequest)
return
}
// Validate workspace path: must be relative and resolve under workspace_root
if payload.Workspace != "" {
resolved, err := c.resolveWorkspace(payload.Workspace)
if err != nil {
http.Error(w, fmt.Sprintf("Invalid workspace: %v", err), http.StatusBadRequest)
return
}
payload.Workspace = resolved
}
// Validate configDir path against workspace_root
if payload.ConfigDir != "" {
resolved, err := c.resolveWorkspace(payload.ConfigDir)
if err != nil {
http.Error(w, fmt.Sprintf("Invalid configDir: %v", err), http.StatusBadRequest)
return
}
payload.ConfigDir = resolved
}
// Return 200 immediately, process asynchronously
w.WriteHeader(http.StatusOK)
go c.processWebhook(c.ctx, payload)
}
// resolveWorkspace validates and resolves the workspace path.
// If workspace_root is configured, the workspace must be a relative path that
// resolves under the root. If workspace_root is not configured, workspace is
// rejected (no arbitrary path writes allowed).
func (c *MagicFormChannel) resolveWorkspace(workspace string) (string, error) {
root := c.config.WorkspaceRoot
if root == "" {
return "", fmt.Errorf("workspace_root not configured; workspace overrides are not allowed")
}
absRoot, err := filepath.Abs(root)
if err != nil {
return "", fmt.Errorf("invalid workspace_root: %w", err)
}
// Join the root with the provided workspace (which may be relative)
resolved := filepath.Join(absRoot, workspace)
resolved = filepath.Clean(resolved)
// Ensure the resolved path is under the root (prevents ../../../etc traversal)
if !strings.HasPrefix(resolved, absRoot+string(filepath.Separator)) && resolved != absRoot {
return "", fmt.Errorf("workspace path escapes workspace_root")
}
return resolved, nil
}
// verifyToken checks the Authorization Bearer token using constant-time comparison.
func (c *MagicFormChannel) verifyToken(r *http.Request) bool {
if c.config.Token == "" {
return true // No token configured = allow all (dev mode)
}
auth := r.Header.Get("Authorization")
if !strings.HasPrefix(auth, "Bearer ") {
return false
}
token := strings.TrimPrefix(auth, "Bearer ")
return subtle.ConstantTimeCompare([]byte(token), []byte(c.config.Token)) == 1
}
// processWebhook handles an inbound webhook payload asynchronously.
func (c *MagicFormChannel) processWebhook(ctx context.Context, p WebhookPayload) {
// Check for shutdown before doing any work
if ctx.Err() != nil {
logger.WarnCF("magicform", "Skipping webhook processing: channel shutting down",
map[string]any{"stack_id": p.StackID, "conversation_id": p.ConversationID})
return
}
chatID := "magicform:" + p.ConversationID
senderID := p.UserID
if senderID == "" {
senderID = "anonymous"
}
// Store request context for Send() to look up later
c.requests.Store(chatID, &requestContext{
stackID: p.StackID,
conversationID: p.ConversationID,
userID: p.UserID,
callbackURL: p.CallbackURL,
createdAt: time.Now(),
})
peer := bus.Peer{Kind: "direct", ID: p.ConversationID}
sender := bus.SenderInfo{
Platform: "magicform",
PlatformID: senderID,
CanonicalID: "magicform:" + senderID,
}
// Session key: per-stack per-conversation isolation
sessionKey := fmt.Sprintf("agent:main:magicform:%s:%s", p.StackID, p.ConversationID)
metadata := map[string]string{
"platform": "magicform",
"stack_id": p.StackID,
"conversation_id": p.ConversationID,
}
if p.CallbackURL != "" {
metadata["callback_url"] = p.CallbackURL
}
// Workspace override — agent loop will pick this up
if p.Workspace != "" {
metadata["workspace_override"] = p.Workspace
}
// Config directory — agent loop reads config.json and copies bootstrap files
if p.ConfigDir != "" {
metadata["config_dir"] = p.ConfigDir
}
// Tool/skill filtering — passed via metadata, picked up by agent loop
if len(p.AllowedTools) > 0 {
metadata["allowed_tools"] = strings.Join(trimSlice(p.AllowedTools), ",")
}
if len(p.AllowedSkills) > 0 {
metadata["allowed_skills"] = strings.Join(trimSlice(p.AllowedSkills), ",")
}
messageID := fmt.Sprintf("mf-%s-%d", p.ConversationID, time.Now().UnixMilli())
// Build InboundMessage directly (not via HandleMessage) to set SessionKey.
// MagicForm is API-to-API, so typing/reaction/placeholder don't apply.
msg := bus.InboundMessage{
Channel: "magicform",
SenderID: sender.CanonicalID,
Sender: sender,
ChatID: chatID,
Content: p.Message,
Peer: peer,
MessageID: messageID,
SessionKey: sessionKey,
Metadata: metadata,
}
if err := c.Bus().PublishInbound(ctx, msg); err != nil {
logger.ErrorCF("magicform", "Failed to publish inbound message",
map[string]any{
"chat_id": chatID,
"stack_id": p.StackID,
"conversation_id": p.ConversationID,
"error": err.Error(),
})
}
}
// trimSlice trims whitespace from each element in the slice.
func trimSlice(s []string) []string {
out := make([]string, len(s))
for i, v := range s {
out[i] = strings.TrimSpace(v)
}
return out
}
// Send delivers the agent response back to MagicForm via HTTP callback.
func (c *MagicFormChannel) Send(ctx context.Context, msg bus.OutboundMessage) error {
if !c.IsRunning() {
return channels.ErrNotRunning
}
// Look up request context
val, ok := c.requests.LoadAndDelete(msg.ChatID)
if !ok {
return fmt.Errorf("%w: no request context for chatID %s", channels.ErrSendFailed, msg.ChatID)
}
reqCtx := val.(*requestContext)
// Resolve callback URL
callbackURL := reqCtx.callbackURL
if callbackURL == "" {
callbackURL = c.config.BackendURL + "/claw-agent/callback"
}
if callbackURL == "" {
return fmt.Errorf("%w: no callback URL available", channels.ErrSendFailed)
}
// Build callback payload
payload := CallbackPayload{
StackID: reqCtx.stackID,
ConversationID: reqCtx.conversationID,
Response: msg.Content,
Type: "final",
}
body, err := json.Marshal(payload)
if err != nil {
return fmt.Errorf("%w: marshal callback payload: %v", channels.ErrSendFailed, err)
}
req, err := http.NewRequestWithContext(ctx, http.MethodPost, callbackURL, bytes.NewReader(body))
if err != nil {
return fmt.Errorf("%w: create callback request: %v", channels.ErrSendFailed, err)
}
req.Header.Set("Content-Type", "application/json")
if c.config.Token != "" {
req.Header.Set("Authorization", "Bearer "+c.config.Token)
}
resp, err := c.httpClient.Do(req)
if err != nil {
return channels.ClassifyNetError(err)
}
defer resp.Body.Close()
if resp.StatusCode >= 400 {
respBody, _ := io.ReadAll(resp.Body)
return channels.ClassifySendError(resp.StatusCode, fmt.Errorf("callback error: %s", respBody))
}
logger.InfoCF("magicform", "Callback sent",
map[string]any{
"conversation_id": reqCtx.conversationID,
"status": resp.StatusCode,
})
return nil
}
// cleanupLoop periodically removes stale request contexts.
func (c *MagicFormChannel) cleanupLoop() {
ticker := time.NewTicker(60 * time.Second)
defer ticker.Stop()
for {
select {
case <-c.ctx.Done():
return
case <-ticker.C:
c.requests.Range(func(key, value any) bool {
rc := value.(*requestContext)
if time.Since(rc.createdAt) > 10*time.Minute {
c.requests.Delete(key)
logger.DebugCF("magicform", "Cleaned up stale request context",
map[string]any{"chat_id": key})
}
return true
})
}
}
}

View file

@ -267,6 +267,10 @@ func (m *Manager) initChannels() error {
m.initChannel("pico", "Pico")
}
if m.config.Channels.MagicForm.Enabled && m.config.Channels.MagicForm.Token != "" {
m.initChannel("magicform", "MagicForm")
}
logger.InfoCF("channels", "Channel initialization completed", map[string]any{
"enabled_channels": len(m.channels),
})

View file

@ -4,6 +4,7 @@ import (
"encoding/json"
"fmt"
"os"
"path/filepath"
"sync/atomic"
"github.com/caarlos0/env/v11"
@ -231,6 +232,7 @@ type ChannelsConfig struct {
WeComApp WeComAppConfig `json:"wecom_app"`
WeComAIBot WeComAIBotConfig `json:"wecom_aibot"`
Pico PicoConfig `json:"pico"`
MagicForm MagicFormConfig `json:"magicform"`
}
// GroupTriggerConfig controls when the bot responds in group chats.
@ -414,6 +416,15 @@ type PicoConfig struct {
Placeholder PlaceholderConfig `json:"placeholder,omitempty"`
}
type MagicFormConfig struct {
Enabled bool `json:"enabled" env:"PICOCLAW_CHANNELS_MAGICFORM_ENABLED"`
Token string `json:"token" env:"PICOCLAW_CHANNELS_MAGICFORM_TOKEN"`
BackendURL string `json:"backend_url" env:"PICOCLAW_CHANNELS_MAGICFORM_BACKEND_URL"`
WebhookPath string `json:"webhook_path" env:"PICOCLAW_CHANNELS_MAGICFORM_WEBHOOK_PATH"`
WorkspaceRoot string `json:"workspace_root" env:"PICOCLAW_CHANNELS_MAGICFORM_WORKSPACE_ROOT"`
AllowFrom FlexibleStringSlice `json:"allow_from" env:"PICOCLAW_CHANNELS_MAGICFORM_ALLOW_FROM"`
}
type HeartbeatConfig struct {
Enabled bool `json:"enabled" env:"PICOCLAW_HEARTBEAT_ENABLED"`
Interval int `json:"interval" env:"PICOCLAW_HEARTBEAT_INTERVAL"` // minutes, min 5
@ -608,6 +619,7 @@ type ExecConfig struct {
CustomDenyPatterns []string ` env:"PICOCLAW_TOOLS_EXEC_CUSTOM_DENY_PATTERNS" json:"custom_deny_patterns"`
CustomAllowPatterns []string ` env:"PICOCLAW_TOOLS_EXEC_CUSTOM_ALLOW_PATTERNS" json:"custom_allow_patterns"`
TimeoutSeconds int ` env:"PICOCLAW_TOOLS_EXEC_TIMEOUT_SECONDS" json:"timeout_seconds"` // 0 means use default (60s)
FilterEnv bool ` env:"PICOCLAW_TOOLS_EXEC_FILTER_ENV" json:"filter_env"`
}
type SkillsToolsConfig struct {
@ -924,3 +936,160 @@ func (t *ToolsConfig) IsToolEnabled(name string) bool {
return true
}
}
// WorkspaceConfig wraps a parsed Config along with the raw JSON bytes so that
// merging can distinguish "field not present" from "field is zero-valued".
type WorkspaceConfig struct {
Config *Config
rawJSON json.RawMessage
}
// LoadWorkspaceConfig loads a workspace-local config.json from the given directory.
// Returns nil, nil if the file does not exist.
func LoadWorkspaceConfig(dir string) (*WorkspaceConfig, error) {
path := filepath.Join(dir, "config.json")
data, err := os.ReadFile(path)
if err != nil {
if os.IsNotExist(err) {
return nil, nil
}
return nil, fmt.Errorf("reading workspace config %s: %w", path, err)
}
var wc Config
if err := json.Unmarshal(data, &wc); err != nil {
return nil, fmt.Errorf("parsing workspace config %s: %w", path, err)
}
for i := range wc.ModelList {
if err := wc.ModelList[i].Validate(); err != nil {
return nil, fmt.Errorf("workspace config model_list[%d]: %w", i, err)
}
}
return &WorkspaceConfig{Config: &wc, rawJSON: data}, nil
}
// Clone returns a deep copy of the config via JSON round-trip.
func (c *Config) Clone() *Config {
data, err := json.Marshal(c)
if err != nil {
// Should never happen with a valid Config
return DefaultConfig()
}
var clone Config
if err := json.Unmarshal(data, &clone); err != nil {
return DefaultConfig()
}
return &clone
}
// MergeWorkspaceConfig overlays allowed fields from a workspace config onto this config.
// Fields NOT honored (infrastructure-level): Gateway, Heartbeat, Devices, Providers.
func (c *Config) MergeWorkspaceConfig(wc *WorkspaceConfig) {
if wc == nil || wc.Config == nil {
return
}
src := wc.Config
// model_list: replace if workspace has entries
if len(src.ModelList) > 0 {
c.ModelList = src.ModelList
}
// agents.defaults: merge non-zero fields
mergeAgentDefaults(&c.Agents.Defaults, &src.Agents.Defaults)
// agents.list: replace if workspace has entries
if len(src.Agents.List) > 0 {
c.Agents.List = src.Agents.List
}
// tools & channels: use raw JSON overlay so that only keys actually present
// in the workspace file are applied (avoids clobbering bool fields with false).
mergeRawJSONField(wc.rawJSON, "tools", &c.Tools)
mergeRawJSONField(wc.rawJSON, "channels", &c.Channels)
// bindings: replace if workspace has entries
if len(src.Bindings) > 0 {
c.Bindings = src.Bindings
}
// session: merge non-zero fields (prevents cross-tenant identity leakage)
mergeSessionConfig(&c.Session, &src.Session)
}
// mergeAgentDefaults copies non-zero fields from src into dst.
func mergeAgentDefaults(dst, src *AgentDefaults) {
if src.Workspace != "" {
dst.Workspace = src.Workspace
}
if src.RestrictToWorkspace {
dst.RestrictToWorkspace = true
}
if src.AllowReadOutsideWorkspace {
dst.AllowReadOutsideWorkspace = true
}
if src.Provider != "" {
dst.Provider = src.Provider
}
if src.ModelName != "" {
dst.ModelName = src.ModelName
}
if src.Model != "" {
dst.Model = src.Model
}
if len(src.ModelFallbacks) > 0 {
dst.ModelFallbacks = src.ModelFallbacks
}
if src.ImageModel != "" {
dst.ImageModel = src.ImageModel
}
if len(src.ImageModelFallbacks) > 0 {
dst.ImageModelFallbacks = src.ImageModelFallbacks
}
if src.MaxTokens > 0 {
dst.MaxTokens = src.MaxTokens
}
if src.Temperature != nil {
dst.Temperature = src.Temperature
}
if src.MaxToolIterations > 0 {
dst.MaxToolIterations = src.MaxToolIterations
}
if src.SummarizeMessageThreshold > 0 {
dst.SummarizeMessageThreshold = src.SummarizeMessageThreshold
}
if src.SummarizeTokenPercent > 0 {
dst.SummarizeTokenPercent = src.SummarizeTokenPercent
}
if src.MaxMediaSize > 0 {
dst.MaxMediaSize = src.MaxMediaSize
}
}
// mergeSessionConfig copies non-zero fields from src into dst.
func mergeSessionConfig(dst, src *SessionConfig) {
if src.DMScope != "" {
dst.DMScope = src.DMScope
}
if len(src.IdentityLinks) > 0 {
dst.IdentityLinks = src.IdentityLinks
}
}
// mergeRawJSONField extracts a top-level key from raw JSON and unmarshals it
// onto dst. Because we use the original JSON bytes, only keys actually present
// in the workspace file are applied — zero-valued fields (e.g. bool false) that
// were never in the file are not included.
func mergeRawJSONField[T any](rawJSON json.RawMessage, key string, dst *T) {
var top map[string]json.RawMessage
if err := json.Unmarshal(rawJSON, &top); err != nil {
return
}
fieldData, ok := top[key]
if !ok || string(fieldData) == "null" {
return
}
json.Unmarshal(fieldData, dst) //nolint:errcheck
}

View file

@ -5,9 +5,11 @@ import (
"encoding/json"
"fmt"
"io"
"log/slog"
"net/http"
"net/url"
"os"
"strings"
"time"
"github.com/sipeed/picoclaw/pkg/utils"
@ -31,12 +33,24 @@ type ClawHubRegistry struct {
client *http.Client
}
const defaultRegistryURL = "https://clawhub.ai"
// NewClawHubRegistry creates a new ClawHub registry client from config.
func NewClawHubRegistry(cfg ClawHubConfig) *ClawHubRegistry {
baseURL := cfg.BaseURL
if baseURL == "" {
baseURL = "https://clawhub.ai"
baseURL = defaultRegistryURL
}
// Validate base URL: require https unless targeting localhost.
if parsedURL, err := url.Parse(baseURL); err != nil || parsedURL.Host == "" {
slog.Warn("invalid clawhub base_url, falling back to default", "url", baseURL)
baseURL = defaultRegistryURL
} else if parsedURL.Scheme != "https" && !strings.HasPrefix(parsedURL.Host, "localhost") && !strings.HasPrefix(parsedURL.Host, "127.0.0.1") {
slog.Warn("clawhub base_url must use https (unless localhost), falling back to default", "url", baseURL)
baseURL = defaultRegistryURL
}
searchPath := cfg.SearchPath
if searchPath == "" {
searchPath = "/api/v1/search"
@ -234,6 +248,7 @@ func (c *ClawHubRegistry) DownloadAndInstall(
result.IsMalwareBlocked = meta.IsMalwareBlocked
result.IsSuspicious = meta.IsSuspicious
result.Summary = meta.Summary
result.MetadataAvailable = true
}
// Step 2: Resolve version.

View file

@ -7,12 +7,17 @@ import (
"net/http"
"os"
"path/filepath"
"regexp"
"time"
"github.com/sipeed/picoclaw/pkg/fileutil"
"github.com/sipeed/picoclaw/pkg/utils"
)
const maxSkillFileSize int64 = 5 << 20 // 5 MB — SKILL.md files should never approach this
var repoPattern = regexp.MustCompile(`^[a-zA-Z0-9_.-]+/[a-zA-Z0-9_.-]+$`)
type SkillInstaller struct {
workspace string
}
@ -24,6 +29,10 @@ func NewSkillInstaller(workspace string) *SkillInstaller {
}
func (si *SkillInstaller) InstallFromGitHub(ctx context.Context, repo string) error {
if !repoPattern.MatchString(repo) {
return fmt.Errorf("invalid repository format %q: must be 'owner/repo'", repo)
}
skillDir := filepath.Join(si.workspace, "skills", filepath.Base(repo))
if _, err := os.Stat(skillDir); err == nil {
@ -48,7 +57,7 @@ func (si *SkillInstaller) InstallFromGitHub(ctx context.Context, repo string) er
return fmt.Errorf("failed to fetch skill: HTTP %d", resp.StatusCode)
}
body, err := io.ReadAll(resp.Body)
body, err := io.ReadAll(io.LimitReader(resp.Body, maxSkillFileSize))
if err != nil {
return fmt.Errorf("failed to read response: %w", err)
}

View file

@ -96,9 +96,37 @@ func NewSkillsLoader(workspace string, globalSkills string, builtinSkills string
}
}
// builtinSkillNames returns the set of validated skill names from the builtin directory.
func (sl *SkillsLoader) builtinSkillNames() map[string]bool {
names := make(map[string]bool)
if sl.builtinSkills == "" {
return names
}
dirs, err := os.ReadDir(sl.builtinSkills)
if err != nil {
return names
}
for _, d := range dirs {
if !d.IsDir() {
continue
}
skillFile := filepath.Join(sl.builtinSkills, d.Name(), "SKILL.md")
if _, err := os.Stat(skillFile); err != nil {
continue
}
name := d.Name()
if metadata := sl.getSkillMetadata(skillFile); metadata != nil && metadata.Name != "" {
name = metadata.Name
}
names[name] = true
}
return names
}
func (sl *SkillsLoader) ListSkills() []SkillInfo {
skills := make([]SkillInfo, 0)
seen := make(map[string]bool)
builtinNames := sl.builtinSkillNames()
addSkills := func(dir, source string) {
if dir == "" {
@ -133,6 +161,12 @@ func (sl *SkillsLoader) ListSkills() []SkillInfo {
if seen[info.Name] {
continue
}
// Block workspace/global skills from shadowing builtins.
if source != "builtin" && builtinNames[info.Name] {
slog.Warn("skill shadows a builtin and will be skipped",
"name", info.Name, "source", source)
continue
}
seen[info.Name] = true
skills = append(skills, info)
}
@ -147,7 +181,15 @@ func (sl *SkillsLoader) ListSkills() []SkillInfo {
}
func (sl *SkillsLoader) LoadSkill(name string) (string, bool) {
// 1. load from workspace skills first (project-level)
// If this is a builtin skill, always load from builtin to prevent shadowing.
if sl.builtinSkills != "" {
skillFile := filepath.Join(sl.builtinSkills, name, "SKILL.md")
if content, err := os.ReadFile(skillFile); err == nil {
return sl.stripFrontmatter(string(content)), true
}
}
// 1. load from workspace skills (project-level)
if sl.workspaceSkills != "" {
skillFile := filepath.Join(sl.workspaceSkills, name, "SKILL.md")
if content, err := os.ReadFile(skillFile); err == nil {
@ -163,14 +205,6 @@ func (sl *SkillsLoader) LoadSkill(name string) (string, bool) {
}
}
// 3. finally load from builtin skills
if sl.builtinSkills != "" {
skillFile := filepath.Join(sl.builtinSkills, name, "SKILL.md")
if content, err := os.ReadFile(skillFile); err == nil {
return sl.stripFrontmatter(string(content)), true
}
}
return "", false
}
@ -191,14 +225,18 @@ func (sl *SkillsLoader) LoadSkillsForContext(skillNames []string) string {
}
func (sl *SkillsLoader) BuildSkillsSummary() string {
allSkills := sl.ListSkills()
if len(allSkills) == 0 {
return FormatSkillsSummary(sl.ListSkills())
}
// FormatSkillsSummary renders a list of SkillInfo entries as an XML summary.
func FormatSkillsSummary(skillList []SkillInfo) string {
if len(skillList) == 0 {
return ""
}
var lines []string
lines = append(lines, "<skills>")
for _, s := range allSkills {
for _, s := range skillList {
escapedName := escapeXML(s.Name)
escapedDesc := escapeXML(s.Description)
escapedPath := escapeXML(s.Path)

View file

@ -163,7 +163,7 @@ func TestListSkillsWorkspaceOverridesGlobal(t *testing.T) {
assert.Equal(t, "workspace version", skills[0].Description)
}
func TestListSkillsGlobalOverridesBuiltin(t *testing.T) {
func TestListSkillsBuiltinCannotBeShadowed(t *testing.T) {
tmp := t.TempDir()
ws := filepath.Join(tmp, "workspace")
global := filepath.Join(tmp, "global")
@ -176,8 +176,8 @@ func TestListSkillsGlobalOverridesBuiltin(t *testing.T) {
skills := sl.ListSkills()
assert.Len(t, skills, 1)
assert.Equal(t, "global", skills[0].Source)
assert.Equal(t, "global version", skills[0].Description)
assert.Equal(t, "builtin", skills[0].Source)
assert.Equal(t, "builtin version", skills[0].Description)
}
func TestListSkillsMetadataNameDedup(t *testing.T) {

View file

@ -36,10 +36,11 @@ type SkillMeta struct {
// InstallResult is returned by DownloadAndInstall to carry metadata
// back to the caller for moderation and user messaging.
type InstallResult struct {
Version string
IsMalwareBlocked bool
IsSuspicious bool
Summary string
Version string
IsMalwareBlocked bool
IsSuspicious bool
Summary string
MetadataAvailable bool
}
// SkillRegistry is the interface that all skill registries must implement.

View file

@ -315,7 +315,7 @@ func (t *CronTool) ExecuteJob(ctx context.Context, job *cron.CronJob) string {
}
// For deliver=false, process through agent (for complex tasks)
sessionKey := fmt.Sprintf("cron-%s", job.ID)
sessionKey := fmt.Sprintf("agent:main:cron:%s", job.ID)
// Call agent with job's message
response, err := t.executor.ProcessDirectWithChannel(

View file

@ -2,17 +2,20 @@ package tools
import (
"context"
"crypto/rand"
"encoding/hex"
"fmt"
"io/fs"
"os"
"path/filepath"
"regexp"
"strings"
"time"
"github.com/sipeed/picoclaw/pkg/fileutil"
)
const maxWriteSize = 20 * 1024 * 1024 // 20 MB — limit for file writes via the write tool
// validatePath ensures the given path is within the workspace if restrict is true.
func validatePath(path, workspace string, restrict bool) (string, error) {
if workspace == "" {
@ -178,6 +181,10 @@ func (t *WriteFileTool) Execute(ctx context.Context, args map[string]any) *ToolR
return ErrorResult("content is required")
}
if len(content) > maxWriteSize {
return ErrorResult(fmt.Sprintf("content too large: %d bytes exceeds %d byte limit", len(content), maxWriteSize))
}
if err := t.fs.WriteFile(path, []byte(content)); err != nil {
return ErrorResult(err.Error())
}
@ -334,7 +341,11 @@ func (r *sandboxFs) WriteFile(path string, data []byte) error {
// Use atomic write pattern with explicit sync for flash storage reliability.
// Using 0o600 (owner read/write only) for secure default permissions.
tmpRelPath := fmt.Sprintf(".tmp-%d-%d", os.Getpid(), time.Now().UnixNano())
randBytes := make([]byte, 8)
if _, err := rand.Read(randBytes); err != nil {
return fmt.Errorf("failed to generate random bytes for temp file: %w", err)
}
tmpRelPath := fmt.Sprintf(".tmp-%s", hex.EncodeToString(randBytes))
tmpFile, err := root.OpenFile(tmpRelPath, os.O_WRONLY|os.O_CREATE|os.O_EXCL, 0o600)
if err != nil {

View file

@ -23,6 +23,7 @@ type ExecTool struct {
allowPatterns []*regexp.Regexp
customAllowPatterns []*regexp.Regexp
restrictToWorkspace bool
filterEnv bool
}
var (
@ -136,6 +137,11 @@ func NewExecToolWithConfig(workingDir string, restrict bool, config *config.Conf
timeout = time.Duration(config.Tools.Exec.TimeoutSeconds) * time.Second
}
filterEnv := false
if config != nil {
filterEnv = config.Tools.Exec.FilterEnv
}
return &ExecTool{
workingDir: workingDir,
timeout: timeout,
@ -143,6 +149,7 @@ func NewExecToolWithConfig(workingDir string, restrict bool, config *config.Conf
allowPatterns: nil,
customAllowPatterns: customAllowPatterns,
restrictToWorkspace: restrict,
filterEnv: filterEnv,
}, nil
}
@ -221,6 +228,10 @@ func (t *ExecTool) Execute(ctx context.Context, args map[string]any) *ToolResult
cmd.Dir = cwd
}
if t.filterEnv {
cmd.Env = filterEnvironment(os.Environ())
}
prepareCommandForTermination(cmd)
var stdout, stderr bytes.Buffer
@ -381,3 +392,38 @@ func (t *ExecTool) SetAllowPatterns(patterns []string) error {
}
return nil
}
// safeEnvVars is the allowlist of environment variable names preserved when
// filterEnv is enabled. Variables with a PICOCLAW_ prefix are always kept.
var safeEnvVars = map[string]bool{
"PATH": true,
"HOME": true,
"USER": true,
"LANG": true,
"TERM": true,
"SHELL": true,
"TMPDIR": true,
"TMP": true,
"TEMP": true,
"SystemRoot": true,
"COMSPEC": true,
"USERPROFILE": true,
"APPDATA": true,
"LOCALAPPDATA": true,
}
// filterEnvironment returns a filtered copy of environ keeping only safe vars.
func filterEnvironment(environ []string) []string {
filtered := make([]string, 0, len(safeEnvVars)+4)
for _, entry := range environ {
eqIdx := strings.IndexByte(entry, '=')
if eqIdx < 0 {
continue
}
name := entry[:eqIdx]
if safeEnvVars[name] || strings.HasPrefix(name, "PICOCLAW_") {
filtered = append(filtered, entry)
}
}
return filtered
}

View file

@ -159,10 +159,13 @@ func (t *InstallSkillTool) Execute(ctx context.Context, args map[string]any) *To
_ = err
}
// Build result with moderation warning if suspicious.
// Build result with moderation warnings.
var output string
if !result.MetadataAvailable {
output += fmt.Sprintf("Warning: safety metadata was not available for skill %q. Exercise caution.\n\n", slug)
}
if result.IsSuspicious {
output = fmt.Sprintf("⚠️ Warning: skill %q is flagged as suspicious (may contain risky patterns).\n\n", slug)
output += fmt.Sprintf("⚠️ Warning: skill %q is flagged as suspicious (may contain risky patterns).\n\n", slug)
}
output += fmt.Sprintf("Successfully installed skill %q v%s from %s registry.\nLocation: %s\n",
slug, result.Version, registry.Name(), targetDir)

View file

@ -22,8 +22,9 @@ const (
perplexityTimeout = 30 * time.Second // Perplexity (LLM-based, slower)
fetchTimeout = 60 * time.Second // WebFetchTool
defaultMaxChars = 50000
maxRedirects = 5
defaultMaxChars = 50000
maxRedirects = 5
searchMaxResponseSize int64 = 2 << 20 // 2 MB — limit for search provider responses
)
// Pre-compiled regexes for HTML text extraction
@ -104,7 +105,7 @@ func (p *BraveSearchProvider) Search(ctx context.Context, query string, count in
}
defer resp.Body.Close()
body, err := io.ReadAll(resp.Body)
body, err := io.ReadAll(io.LimitReader(resp.Body, searchMaxResponseSize))
if err != nil {
return "", fmt.Errorf("failed to read response: %w", err)
}
@ -191,7 +192,7 @@ func (p *TavilySearchProvider) Search(ctx context.Context, query string, count i
}
defer resp.Body.Close()
body, err := io.ReadAll(resp.Body)
body, err := io.ReadAll(io.LimitReader(resp.Body, searchMaxResponseSize))
if err != nil {
return "", fmt.Errorf("failed to read response: %w", err)
}
@ -253,7 +254,7 @@ func (p *DuckDuckGoSearchProvider) Search(ctx context.Context, query string, cou
}
defer resp.Body.Close()
body, err := io.ReadAll(resp.Body)
body, err := io.ReadAll(io.LimitReader(resp.Body, searchMaxResponseSize))
if err != nil {
return "", fmt.Errorf("failed to read response: %w", err)
}
@ -367,7 +368,7 @@ func (p *PerplexitySearchProvider) Search(ctx context.Context, query string, cou
}
defer resp.Body.Close()
body, err := io.ReadAll(resp.Body)
body, err := io.ReadAll(io.LimitReader(resp.Body, searchMaxResponseSize))
if err != nil {
return "", fmt.Errorf("failed to read response: %w", err)
}