CLI overrides, workspace config, MagicForm channel, and security hardening (#1)
* feat(cli): add workspace, config-dir, tools, and skills override flags Add --workspace, --config-dir, --tools, and --skills flags to `picoclaw agent` for single-shot invocations. Supports workspace override with bootstrap file injection (AGENTS.md, IDENTITY.md, SOUL.md, USER.md), tool allowlisting, and skills filtering. Also extracts FormatSkillsSummary from SkillsLoader for reuse, adds SetSkillsFilter to ContextBuilder, and wires SkillsFilter from agent config into the context builder. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat(channels): add MagicForm webhook channel plugin Implement MagicForm channel for webhook-based agentic task delegation. Supports Bearer token auth, async processing with HTTP callback, workspace path validation against configurable workspace_root, bootstrap file injection, tool/skill filtering via metadata, and per-conversation session key isolation. Includes Bus() accessor on BaseChannel for direct message publishing, MagicFormConfig with workspace_root for path traversal prevention, request body size limits, and TTL cleanup for stale request contexts. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat(agent): support per-request workspace, tool, and skill overrides Add workspace override, tool allowlisting, and skills filtering to the agent loop via processOptions metadata. When a workspace override is active (e.g. from MagicForm channel), creates isolated SessionManager and ContextBuilder instances per request. Threads effSessions/effContextBuilder through all downstream paths including runLLMIteration, forceCompression, maybeSummarize, and summarizeSession to ensure full workspace isolation. Adds defense-in- depth tool execution guard alongside LLM-facing tool definition filter. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * security(auth): add response body size limits to OAuth HTTP calls Wrap all io.ReadAll(resp.Body) calls in the OAuth flow with io.LimitReader capped at 1 MB to prevent memory exhaustion from malicious or unexpectedly large server responses. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * security(tools): add response body size limits to search providers Add 2 MB LimitReader to BraveSearch, Tavily, DuckDuckGo, and Perplexity search providers to prevent memory exhaustion from unexpectedly large responses. Matches existing pattern used by GLMSearchProvider. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * security(tools): add 20 MB file write size limit Reject writes exceeding 20 MB in WriteFileTool to prevent disk exhaustion from unexpectedly large LLM-generated file content. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * security(tools): use crypto/rand for temp file naming Replace predictable PID+nanosecond temp file names with cryptographically random hex strings to prevent symlink pre-placement attacks. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * security(skills): validate GitHub repo format and limit download size Add regex validation for 'owner/repo' format to prevent URL injection in InstallFromGitHub. Add 5 MB LimitReader on skill file downloads to prevent memory exhaustion. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * security(auth): allow OAuth credentials to be overridden via env vars Add PICOCLAW_OPENAI_CLIENT_ID, PICOCLAW_GOOGLE_CLIENT_ID, and PICOCLAW_GOOGLE_CLIENT_SECRET env var overrides. Hardcoded values remain as fallback defaults for backward compatibility. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * security(auth): sanitize OAuth error messages Replace raw HTTP response bodies in error messages with generic status-code-only messages. Full response details are logged at debug level for troubleshooting. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * security(launcher): HTML-escape error messages in auth callback Use html.EscapeString on all error text rendered in HTML responses to prevent XSS via crafted OAuth error parameters or error messages. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * security(launcher): remove internal details from HTTP error responses Log detailed errors server-side and return generic messages to clients to prevent information disclosure of file paths and internal state. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * security(launcher): add HTTP security headers middleware Add SecurityHeaders middleware setting X-Content-Type-Options, X-Frame-Options, and Content-Security-Policy on all responses to mitigate MIME sniffing, clickjacking, and content injection. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * security(skills): validate ClawHub registry base URL scheme Reject non-HTTPS registry URLs (unless localhost) to prevent config-based redirection to malicious skill servers. Falls back to the default https://clawhub.ai with a warning. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * security(skills): prevent workspace skills from shadowing builtins Builtin skill names are now reserved — workspace and global skills with the same name as a builtin are skipped with a warning log. LoadSkill also checks builtins first. This prevents supply chain attacks where a malicious skill replaces a trusted builtin. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * security(skills): warn when safety metadata is unavailable Add MetadataAvailable flag to InstallResult. When the registry metadata fetch fails (silent fallback), the user now sees a warning that safety checks could not be completed. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * security(shell): add opt-in environment variable filtering When filter_env is enabled in config, shell commands run with a minimal allowlist of environment variables (PATH, HOME, LANG, TERM, etc. plus PICOCLAW_* prefix), preventing accidental leakage of sensitive env vars like API keys to spawned processes. Disabled by default for backward compat. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat(config): add workspace-local config.json overlay for per-tenant isolation Support per-workspace config overrides via --config-dir (CLI) or configDir (webhook), enabling different API keys, models, and agent settings per tenant. - Add LoadWorkspaceConfig/MergeWorkspaceConfig with raw JSON overlay to preserve unmentioned bool fields (prevents clobbering tool enabled flags) - Add Config.Clone() for safe per-request config copies in gateway mode - Gateway: per-request provider creation from workspace config, with effProvider/effModel threaded through all LLM call sites and summarization - CLI: workspace config merged before provider creation, CLI flags win - MagicForm: replace inline bootstrap fields with configDir path; agent loop copies bootstrap files and loads config.json from configDir - Fix --session flag: format as agent:main:cli:{key} so router honors it - Fix cron session key: format as agent:main:cron:{id} for isolation - Add shared CopyBootstrapFiles helper in pkg/agent/bootstrap.go - Add config/workspace.config.example.json Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(agent): honor workspace config overrides in model routing selectCandidates uses agent.Model/Candidates baked in at startup, which ignores per-request workspace config overrides (effProvider/effModel). When the workspace overrides the provider, skip routing and fallback candidates entirely to avoid cross-provider credential issues. When only the model is overridden, allow routing but use the effective model. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * style: fix linter errors (gci, gofumpt, golines, govet, unused) - gci: alphabetize magicform import in gateway/helpers.go - gofumpt: reformat multi-line function signature in agent/helpers.go - golines: wrap lines exceeding 120 chars across 5 files - govet: rename shadowed variables (err → wcErr, s → trimmed) - unused: remove maybeSummarize, forceCompression, summarizeSession wrappers superseded by their *With variants Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: nuestraai <nuestraai@users.noreply.github.com> Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> Co-authored-by: admin-mf <admin@magicform.ai>
This commit is contained in:
parent
1945436dd4
commit
ac6e77d7c9
28 changed files with 1322 additions and 134 deletions
|
|
@ -3,6 +3,7 @@ package server
|
|||
import (
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"html"
|
||||
"io"
|
||||
"log"
|
||||
"net/http"
|
||||
|
|
@ -237,7 +238,7 @@ func handleOAuthCallback(w http.ResponseWriter, r *http.Request) {
|
|||
fmt.Fprintf(
|
||||
w,
|
||||
`<html><body><h2>Authentication failed</h2><p>%s</p><p>You can close this window.</p></body></html>`,
|
||||
errMsg,
|
||||
html.EscapeString(errMsg),
|
||||
)
|
||||
return
|
||||
}
|
||||
|
|
@ -248,7 +249,7 @@ func handleOAuthCallback(w http.ResponseWriter, r *http.Request) {
|
|||
fmt.Fprintf(
|
||||
w,
|
||||
`<html><body><h2>Authentication failed</h2><p>%s</p><p>You can close this window.</p></body></html>`,
|
||||
err.Error(),
|
||||
html.EscapeString(err.Error()),
|
||||
)
|
||||
return
|
||||
}
|
||||
|
|
@ -267,7 +268,11 @@ func handleOAuthCallback(w http.ResponseWriter, r *http.Request) {
|
|||
|
||||
if err := auth.SetCredential(session.Provider, cred); err != nil {
|
||||
w.Header().Set("Content-Type", "text/html")
|
||||
fmt.Fprintf(w, `<html><body><h2>Failed to save credentials</h2><p>%s</p></body></html>`, err.Error())
|
||||
fmt.Fprintf(
|
||||
w,
|
||||
`<html><body><h2>Failed to save credentials</h2><p>%s</p></body></html>`,
|
||||
html.EscapeString(err.Error()),
|
||||
)
|
||||
return
|
||||
}
|
||||
|
||||
|
|
|
|||
|
|
@ -32,7 +32,8 @@ func RegisterConfigAPI(mux *http.ServeMux, absPath string) {
|
|||
mux.HandleFunc("GET /api/config", func(w http.ResponseWriter, r *http.Request) {
|
||||
cfg, err := config.LoadConfig(absPath)
|
||||
if err != nil {
|
||||
http.Error(w, fmt.Sprintf("Failed to load config: %v", err), http.StatusInternalServerError)
|
||||
log.Printf("Failed to load config: %v", err)
|
||||
http.Error(w, "Failed to load config", http.StatusInternalServerError)
|
||||
return
|
||||
}
|
||||
w.Header().Set("Content-Type", "application/json")
|
||||
|
|
@ -63,7 +64,8 @@ func RegisterConfigAPI(mux *http.ServeMux, absPath string) {
|
|||
}
|
||||
|
||||
if err := config.SaveConfig(absPath, &cfg); err != nil {
|
||||
http.Error(w, fmt.Sprintf("Failed to save config: %v", err), http.StatusInternalServerError)
|
||||
log.Printf("Failed to save config: %v", err)
|
||||
http.Error(w, "Failed to save config", http.StatusInternalServerError)
|
||||
return
|
||||
}
|
||||
|
||||
|
|
@ -77,7 +79,8 @@ func RegisterAuthAPI(mux *http.ServeMux, absPath string) {
|
|||
mux.HandleFunc("GET /api/auth/status", func(w http.ResponseWriter, r *http.Request) {
|
||||
store, err := auth.LoadStore()
|
||||
if err != nil {
|
||||
http.Error(w, fmt.Sprintf("Failed to load auth store: %v", err), http.StatusInternalServerError)
|
||||
log.Printf("Failed to load auth store: %v", err)
|
||||
http.Error(w, "Failed to load auth store", http.StatusInternalServerError)
|
||||
return
|
||||
}
|
||||
|
||||
|
|
@ -194,3 +197,14 @@ func RegisterAuthAPI(mux *http.ServeMux, absPath string) {
|
|||
// GET /auth/callback — OAuth browser callback for Google Antigravity
|
||||
mux.HandleFunc("GET /auth/callback", handleOAuthCallback)
|
||||
}
|
||||
|
||||
// SecurityHeaders wraps an http.Handler to add standard security headers.
|
||||
func SecurityHeaders(next http.Handler) http.Handler {
|
||||
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
w.Header().Set("X-Content-Type-Options", "nosniff")
|
||||
w.Header().Set("X-Frame-Options", "DENY")
|
||||
w.Header().
|
||||
Set("Content-Security-Policy", "default-src 'self'; script-src 'self' 'unsafe-inline'; style-src 'self' 'unsafe-inline'")
|
||||
next.ServeHTTP(w, r)
|
||||
})
|
||||
}
|
||||
|
|
|
|||
|
|
@ -105,7 +105,7 @@ func main() {
|
|||
}
|
||||
}()
|
||||
|
||||
if err := http.ListenAndServe(addr, mux); err != nil {
|
||||
if err := http.ListenAndServe(addr, server.SecurityHeaders(mux)); err != nil {
|
||||
log.Fatalf("Server failed: %v", err)
|
||||
}
|
||||
}
|
||||
|
|
|
|||
|
|
@ -10,6 +10,11 @@ func NewAgentCommand() *cobra.Command {
|
|||
sessionKey string
|
||||
model string
|
||||
debug bool
|
||||
// Workspace and config overrides
|
||||
workspace string
|
||||
configDir string
|
||||
tools string
|
||||
skills string
|
||||
)
|
||||
|
||||
cmd := &cobra.Command{
|
||||
|
|
@ -17,14 +22,23 @@ func NewAgentCommand() *cobra.Command {
|
|||
Short: "Interact with the agent directly",
|
||||
Args: cobra.NoArgs,
|
||||
RunE: func(cmd *cobra.Command, _ []string) error {
|
||||
return agentCmd(message, sessionKey, model, debug)
|
||||
return agentCmd(message, sessionKey, model, debug,
|
||||
workspace, configDir, tools, skills)
|
||||
},
|
||||
}
|
||||
|
||||
cmd.Flags().BoolVarP(&debug, "debug", "d", false, "Enable debug logging")
|
||||
cmd.Flags().StringVarP(&message, "message", "m", "", "Send a single message (non-interactive mode)")
|
||||
cmd.Flags().StringVarP(&sessionKey, "session", "s", "cli:default", "Session key")
|
||||
cmd.Flags().
|
||||
StringVarP(&sessionKey, "session", "s", "", "Session key for conversation isolation (e.g. stackId:conversationId)")
|
||||
cmd.Flags().StringVarP(&model, "model", "", "", "Model to use")
|
||||
|
||||
// Workspace and config overrides
|
||||
cmd.Flags().StringVar(&workspace, "workspace", "", "Override agent workspace directory")
|
||||
cmd.Flags().
|
||||
StringVar(&configDir, "config-dir", "", "Directory containing config.json (model/agent/tool overrides) and bootstrap files (AGENTS.md, IDENTITY.md, SOUL.md, USER.md)")
|
||||
cmd.Flags().StringVar(&tools, "tools", "", "Comma-separated tool allowlist (only these tools enabled)")
|
||||
cmd.Flags().StringVar(&skills, "skills", "", "Comma-separated skill filter (only these skills loaded)")
|
||||
|
||||
return cmd
|
||||
}
|
||||
|
|
|
|||
|
|
@ -14,13 +14,18 @@ import (
|
|||
"github.com/sipeed/picoclaw/cmd/picoclaw/internal"
|
||||
"github.com/sipeed/picoclaw/pkg/agent"
|
||||
"github.com/sipeed/picoclaw/pkg/bus"
|
||||
"github.com/sipeed/picoclaw/pkg/config"
|
||||
"github.com/sipeed/picoclaw/pkg/logger"
|
||||
"github.com/sipeed/picoclaw/pkg/providers"
|
||||
)
|
||||
|
||||
func agentCmd(message, sessionKey, model string, debug bool) error {
|
||||
func agentCmd(message, sessionKey, model string, debug bool,
|
||||
workspace, configDir, toolsFlag, skillsFlag string,
|
||||
) error {
|
||||
if sessionKey == "" {
|
||||
sessionKey = "cli:default"
|
||||
sessionKey = "agent:main:cli:default"
|
||||
} else if !strings.HasPrefix(sessionKey, "agent:") {
|
||||
sessionKey = "agent:main:cli:" + sessionKey
|
||||
}
|
||||
|
||||
if debug {
|
||||
|
|
@ -33,10 +38,44 @@ func agentCmd(message, sessionKey, model string, debug bool) error {
|
|||
return fmt.Errorf("error loading config: %w", err)
|
||||
}
|
||||
|
||||
// Apply workspace-local config overrides from config-dir
|
||||
if configDir != "" {
|
||||
wc, wcErr := config.LoadWorkspaceConfig(configDir)
|
||||
if wcErr != nil {
|
||||
return fmt.Errorf("error loading workspace config from %s: %w", configDir, wcErr)
|
||||
}
|
||||
cfg.MergeWorkspaceConfig(wc)
|
||||
}
|
||||
|
||||
// CLI flags win over workspace config
|
||||
if model != "" {
|
||||
cfg.Agents.Defaults.ModelName = model
|
||||
}
|
||||
|
||||
// Workspace override
|
||||
if workspace != "" {
|
||||
cfg.Agents.Defaults.Workspace = workspace
|
||||
os.MkdirAll(workspace, 0o755)
|
||||
}
|
||||
|
||||
// Tool allowlist: disable all tools, then enable only the listed ones
|
||||
if toolsFlag != "" {
|
||||
toolList := strings.Split(toolsFlag, ",")
|
||||
for i := range toolList {
|
||||
toolList[i] = strings.TrimSpace(toolList[i])
|
||||
}
|
||||
applyToolAllowlist(cfg, toolList)
|
||||
}
|
||||
|
||||
// Skills filter: inject into agent config so NewAgentInstance picks it up
|
||||
if skillsFlag != "" {
|
||||
skillList := strings.Split(skillsFlag, ",")
|
||||
for i := range skillList {
|
||||
skillList[i] = strings.TrimSpace(skillList[i])
|
||||
}
|
||||
applySkillsFilter(cfg, skillList)
|
||||
}
|
||||
|
||||
provider, modelID, err := providers.CreateProvider(cfg)
|
||||
if err != nil {
|
||||
return fmt.Errorf("error creating provider: %w", err)
|
||||
|
|
@ -51,6 +90,11 @@ func agentCmd(message, sessionKey, model string, debug bool) error {
|
|||
defer msgBus.Close()
|
||||
agentLoop := agent.NewAgentLoop(cfg, msgBus, provider)
|
||||
|
||||
// Copy bootstrap files from config-dir to workspace
|
||||
if configDir != "" {
|
||||
copyBootstrapFiles(configDir, cfg.Agents.Defaults.Workspace)
|
||||
}
|
||||
|
||||
// Print agent startup info (only for interactive mode)
|
||||
startupInfo := agentLoop.GetStartupInfo()
|
||||
logger.InfoCF("agent", "Agent initialized",
|
||||
|
|
@ -76,6 +120,65 @@ func agentCmd(message, sessionKey, model string, debug bool) error {
|
|||
return nil
|
||||
}
|
||||
|
||||
// applyToolAllowlist disables all tools, then enables only the listed ones.
|
||||
func applyToolAllowlist(cfg *config.Config, allowed []string) {
|
||||
allowSet := make(map[string]bool, len(allowed))
|
||||
for _, t := range allowed {
|
||||
allowSet[t] = true
|
||||
}
|
||||
|
||||
cfg.Tools.ReadFile.Enabled = allowSet["read_file"]
|
||||
cfg.Tools.WriteFile.Enabled = allowSet["write_file"]
|
||||
cfg.Tools.EditFile.Enabled = allowSet["edit_file"]
|
||||
cfg.Tools.AppendFile.Enabled = allowSet["append_file"]
|
||||
cfg.Tools.ListDir.Enabled = allowSet["list_dir"]
|
||||
cfg.Tools.Exec.Enabled = allowSet["exec"]
|
||||
cfg.Tools.Spawn.Enabled = allowSet["spawn"]
|
||||
cfg.Tools.Cron.Enabled = allowSet["cron"]
|
||||
cfg.Tools.Web.Enabled = allowSet["web"] || allowSet["web_search"]
|
||||
cfg.Tools.WebFetch.Enabled = allowSet["web_fetch"]
|
||||
cfg.Tools.Skills.Enabled = allowSet["skills"]
|
||||
cfg.Tools.FindSkills.Enabled = allowSet["find_skills"]
|
||||
cfg.Tools.InstallSkill.Enabled = allowSet["install_skill"]
|
||||
cfg.Tools.Subagent.Enabled = allowSet["subagent"]
|
||||
cfg.Tools.Message.Enabled = allowSet["message"]
|
||||
cfg.Tools.MCP.Enabled = allowSet["mcp"]
|
||||
cfg.Tools.I2C.Enabled = allowSet["i2c"]
|
||||
cfg.Tools.SPI.Enabled = allowSet["spi"]
|
||||
}
|
||||
|
||||
// applySkillsFilter injects a skills filter into the agent config.
|
||||
func applySkillsFilter(cfg *config.Config, skills []string) {
|
||||
if len(cfg.Agents.List) == 0 {
|
||||
// Create an implicit main agent with skills filter
|
||||
cfg.Agents.List = []config.AgentConfig{
|
||||
{ID: "main", Default: true, Skills: skills},
|
||||
}
|
||||
} else {
|
||||
// Apply to all agents
|
||||
for i := range cfg.Agents.List {
|
||||
cfg.Agents.List[i].Skills = skills
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// copyBootstrapFiles copies recognized bootstrap files (AGENTS.md, IDENTITY.md,
|
||||
// SOUL.md, USER.md) from srcDir into the workspace directory.
|
||||
func copyBootstrapFiles(srcDir, workspace string) {
|
||||
bootstrapFiles := []string{"AGENTS.md", "IDENTITY.md", "SOUL.md", "USER.md"}
|
||||
for _, filename := range bootstrapFiles {
|
||||
srcPath := filepath.Join(srcDir, filename)
|
||||
data, err := os.ReadFile(srcPath)
|
||||
if err != nil {
|
||||
continue // file not present in config-dir, skip
|
||||
}
|
||||
dstPath := filepath.Join(workspace, filename)
|
||||
if err := os.WriteFile(dstPath, data, 0o644); err != nil {
|
||||
fmt.Fprintf(os.Stderr, "Warning: failed to write %s: %v\n", dstPath, err)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func interactiveMode(agentLoop *agent.AgentLoop, sessionKey string) {
|
||||
prompt := fmt.Sprintf("%s You: ", internal.Logo)
|
||||
|
||||
|
|
|
|||
|
|
@ -17,6 +17,7 @@ import (
|
|||
_ "github.com/sipeed/picoclaw/pkg/channels/discord"
|
||||
_ "github.com/sipeed/picoclaw/pkg/channels/feishu"
|
||||
_ "github.com/sipeed/picoclaw/pkg/channels/line"
|
||||
_ "github.com/sipeed/picoclaw/pkg/channels/magicform"
|
||||
_ "github.com/sipeed/picoclaw/pkg/channels/maixcam"
|
||||
_ "github.com/sipeed/picoclaw/pkg/channels/onebot"
|
||||
_ "github.com/sipeed/picoclaw/pkg/channels/pico"
|
||||
|
|
|
|||
|
|
@ -164,6 +164,15 @@
|
|||
"max_steps": 10,
|
||||
"welcome_message": "Hello! I'm your AI assistant. How can I help you today?",
|
||||
"reasoning_channel_id": ""
|
||||
},
|
||||
"magicform": {
|
||||
"_comment": "MagicForm - Webhook-based channel for MagicForm agentic task delegation",
|
||||
"enabled": false,
|
||||
"token": "",
|
||||
"backend_url": "",
|
||||
"webhook_path": "/hooks/magicform",
|
||||
"workspace_root": "/data/workspaces",
|
||||
"allow_from": []
|
||||
}
|
||||
},
|
||||
"providers": {
|
||||
|
|
|
|||
30
config/workspace.config.example.json
Normal file
30
config/workspace.config.example.json
Normal file
|
|
@ -0,0 +1,30 @@
|
|||
{
|
||||
"_comment": "Workspace-local config override. Place as config.json in a config directory passed via --config-dir (CLI) or configDir (webhook). Only the fields below are honored; gateway, heartbeat, devices, and providers (legacy) are ignored.",
|
||||
"model_list": [
|
||||
{
|
||||
"model_name": "main",
|
||||
"model": "anthropic/claude-sonnet-4.6",
|
||||
"api_key": "sk-ant-your-key",
|
||||
"api_base": "https://api.anthropic.com/v1"
|
||||
}
|
||||
],
|
||||
"agents": {
|
||||
"defaults": {
|
||||
"model_name": "main",
|
||||
"max_tokens": 4096,
|
||||
"temperature": 0.7,
|
||||
"max_tool_iterations": 20
|
||||
}
|
||||
},
|
||||
"session": {
|
||||
"dm_scope": "per-channel-peer"
|
||||
},
|
||||
"tools": {
|
||||
"exec": {
|
||||
"enabled": false
|
||||
},
|
||||
"web": {
|
||||
"enabled": true
|
||||
}
|
||||
}
|
||||
}
|
||||
29
pkg/agent/bootstrap.go
Normal file
29
pkg/agent/bootstrap.go
Normal file
|
|
@ -0,0 +1,29 @@
|
|||
package agent
|
||||
|
||||
import (
|
||||
"os"
|
||||
"path/filepath"
|
||||
|
||||
"github.com/sipeed/picoclaw/pkg/logger"
|
||||
)
|
||||
|
||||
// bootstrapFiles are the recognized bootstrap file names copied from a config
|
||||
// directory into the agent workspace.
|
||||
var bootstrapFiles = []string{"AGENTS.md", "IDENTITY.md", "SOUL.md", "USER.md"}
|
||||
|
||||
// CopyBootstrapFiles copies recognized bootstrap files from srcDir into dstDir.
|
||||
// Missing files in srcDir are silently skipped.
|
||||
func CopyBootstrapFiles(srcDir, dstDir string) {
|
||||
for _, filename := range bootstrapFiles {
|
||||
srcPath := filepath.Join(srcDir, filename)
|
||||
data, err := os.ReadFile(srcPath)
|
||||
if err != nil {
|
||||
continue // file not present, skip
|
||||
}
|
||||
dstPath := filepath.Join(dstDir, filename)
|
||||
if err := os.WriteFile(dstPath, data, 0o644); err != nil {
|
||||
logger.WarnCF("agent", "Failed to write bootstrap file",
|
||||
map[string]any{"path": dstPath, "error": err.Error()})
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
@ -39,6 +39,54 @@ type ContextBuilder struct {
|
|||
// build time. This catches nested file creations/deletions/mtime changes
|
||||
// that may not update the top-level skill root directory mtime.
|
||||
skillFilesAtCache map[string]time.Time
|
||||
|
||||
// skillsFilter limits which skills are included in the system prompt.
|
||||
// When non-empty, only skills whose name matches an entry are included.
|
||||
// ["*"] means include all skills.
|
||||
skillsFilter []string
|
||||
}
|
||||
|
||||
// SetSkillsFilter sets the skills filter on the context builder.
|
||||
// When non-empty, only skills matching these names are included in the system prompt.
|
||||
// Use ["*"] to include all skills.
|
||||
func (cb *ContextBuilder) SetSkillsFilter(filter []string) {
|
||||
cb.skillsFilter = filter
|
||||
cb.InvalidateCache()
|
||||
}
|
||||
|
||||
// buildFilteredSkillsSummary returns the skills summary, filtered by skillsFilter
|
||||
// if one is set. When no filter is set, all skills are included.
|
||||
func (cb *ContextBuilder) buildFilteredSkillsSummary() string {
|
||||
if len(cb.skillsFilter) == 0 {
|
||||
return cb.skillsLoader.BuildSkillsSummary()
|
||||
}
|
||||
|
||||
// Check for wildcard
|
||||
for _, f := range cb.skillsFilter {
|
||||
if f == "*" {
|
||||
return cb.skillsLoader.BuildSkillsSummary()
|
||||
}
|
||||
}
|
||||
|
||||
// Build filter set
|
||||
filterSet := make(map[string]bool, len(cb.skillsFilter))
|
||||
for _, f := range cb.skillsFilter {
|
||||
filterSet[f] = true
|
||||
}
|
||||
|
||||
allSkills := cb.skillsLoader.ListSkills()
|
||||
var filtered []skills.SkillInfo
|
||||
for _, s := range allSkills {
|
||||
if filterSet[s.Name] {
|
||||
filtered = append(filtered, s)
|
||||
}
|
||||
}
|
||||
|
||||
if len(filtered) == 0 {
|
||||
return ""
|
||||
}
|
||||
|
||||
return skills.FormatSkillsSummary(filtered)
|
||||
}
|
||||
|
||||
func getGlobalConfigDir() string {
|
||||
|
|
@ -107,7 +155,7 @@ func (cb *ContextBuilder) BuildSystemPrompt() string {
|
|||
}
|
||||
|
||||
// Skills - show summary, AI can read full content with read_file tool
|
||||
skillsSummary := cb.skillsLoader.BuildSkillsSummary()
|
||||
skillsSummary := cb.buildFilteredSkillsSummary()
|
||||
if skillsSummary != "" {
|
||||
parts = append(parts, fmt.Sprintf(`# Skills
|
||||
|
||||
|
|
|
|||
|
|
@ -98,6 +98,7 @@ func NewAgentInstance(
|
|||
|
||||
contextBuilder := NewContextBuilder(workspace)
|
||||
|
||||
// SkillsFilter will be applied after we know the agent config
|
||||
agentID := routing.DefaultAgentID
|
||||
agentName := ""
|
||||
var subagents *config.SubagentsConfig
|
||||
|
|
@ -110,6 +111,11 @@ func NewAgentInstance(
|
|||
skillsFilter = agentCfg.Skills
|
||||
}
|
||||
|
||||
// Apply skills filter to context builder so only matching skills appear in the prompt
|
||||
if len(skillsFilter) > 0 {
|
||||
contextBuilder.SetSkillsFilter(skillsFilter)
|
||||
}
|
||||
|
||||
maxIter := defaults.MaxToolIterations
|
||||
if maxIter == 0 {
|
||||
maxIter = 20
|
||||
|
|
|
|||
|
|
@ -11,6 +11,7 @@ import (
|
|||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"regexp"
|
||||
"strings"
|
||||
|
|
@ -29,6 +30,7 @@ import (
|
|||
"github.com/sipeed/picoclaw/pkg/media"
|
||||
"github.com/sipeed/picoclaw/pkg/providers"
|
||||
"github.com/sipeed/picoclaw/pkg/routing"
|
||||
"github.com/sipeed/picoclaw/pkg/session"
|
||||
"github.com/sipeed/picoclaw/pkg/skills"
|
||||
"github.com/sipeed/picoclaw/pkg/state"
|
||||
"github.com/sipeed/picoclaw/pkg/tools"
|
||||
|
|
@ -52,15 +54,31 @@ type AgentLoop struct {
|
|||
|
||||
// processOptions configures how a message is processed
|
||||
type processOptions struct {
|
||||
SessionKey string // Session identifier for history/context
|
||||
Channel string // Target channel for tool execution
|
||||
ChatID string // Target chat ID for tool execution
|
||||
UserMessage string // User message content (may include prefix)
|
||||
Media []string // media:// refs from inbound message
|
||||
DefaultResponse string // Response when LLM returns empty
|
||||
EnableSummary bool // Whether to trigger summarization
|
||||
SendResponse bool // Whether to send response via bus
|
||||
NoHistory bool // If true, don't load session history (for heartbeat)
|
||||
SessionKey string // Session identifier for history/context
|
||||
Channel string // Target channel for tool execution
|
||||
ChatID string // Target chat ID for tool execution
|
||||
UserMessage string // User message content (may include prefix)
|
||||
Media []string // media:// refs from inbound message
|
||||
DefaultResponse string // Response when LLM returns empty
|
||||
EnableSummary bool // Whether to trigger summarization
|
||||
SendResponse bool // Whether to send response via bus
|
||||
NoHistory bool // If true, don't load session history (for heartbeat)
|
||||
WorkspaceOverride string // If set, use this workspace instead of agent.Workspace
|
||||
ConfigDir string // If set, config directory for workspace-local overrides
|
||||
AllowedTools []string // If non-empty, only these tools are active for this request
|
||||
AllowedSkills []string // If non-empty, only these skills are loaded for this request
|
||||
|
||||
// effSessions and effContextBuilder are set by runAgentLoop when a workspace
|
||||
// override is active. All downstream code (runLLMIteration, forceCompressionWith
|
||||
// retry) MUST use these instead of agent.Sessions / agent.ContextBuilder.
|
||||
effSessions *session.SessionManager
|
||||
effContextBuilder *ContextBuilder
|
||||
|
||||
// effProvider and effModel are set by runAgentLoop when a workspace config
|
||||
// provides per-request provider overrides. All LLM call sites (runLLMIteration,
|
||||
// summarization) MUST use these instead of agent.Provider / agent.Model.
|
||||
effProvider providers.LLMProvider
|
||||
effModel string
|
||||
}
|
||||
|
||||
const (
|
||||
|
|
@ -615,15 +633,39 @@ func (al *AgentLoop) processMessage(ctx context.Context, msg bus.InboundMessage)
|
|||
"route_channel": route.Channel,
|
||||
})
|
||||
|
||||
// Extract overrides from metadata (used by magicform channel / gateway mode)
|
||||
workspaceOverride := msg.Metadata["workspace_override"]
|
||||
configDir := msg.Metadata["config_dir"]
|
||||
|
||||
var allowedTools, allowedSkills []string
|
||||
if v := msg.Metadata["allowed_tools"]; v != "" {
|
||||
for _, t := range strings.Split(v, ",") {
|
||||
if s := strings.TrimSpace(t); s != "" {
|
||||
allowedTools = append(allowedTools, s)
|
||||
}
|
||||
}
|
||||
}
|
||||
if v := msg.Metadata["allowed_skills"]; v != "" {
|
||||
for _, s := range strings.Split(v, ",") {
|
||||
if trimmed := strings.TrimSpace(s); trimmed != "" {
|
||||
allowedSkills = append(allowedSkills, trimmed)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
return al.runAgentLoop(ctx, agent, processOptions{
|
||||
SessionKey: sessionKey,
|
||||
Channel: msg.Channel,
|
||||
ChatID: msg.ChatID,
|
||||
UserMessage: msg.Content,
|
||||
Media: msg.Media,
|
||||
DefaultResponse: defaultResponse,
|
||||
EnableSummary: true,
|
||||
SendResponse: false,
|
||||
SessionKey: sessionKey,
|
||||
Channel: msg.Channel,
|
||||
ChatID: msg.ChatID,
|
||||
UserMessage: msg.Content,
|
||||
Media: msg.Media,
|
||||
DefaultResponse: defaultResponse,
|
||||
EnableSummary: true,
|
||||
SendResponse: false,
|
||||
WorkspaceOverride: workspaceOverride,
|
||||
ConfigDir: configDir,
|
||||
AllowedTools: allowedTools,
|
||||
AllowedSkills: allowedSkills,
|
||||
})
|
||||
}
|
||||
|
||||
|
|
@ -741,14 +783,65 @@ func (al *AgentLoop) runAgentLoop(
|
|||
}
|
||||
}
|
||||
|
||||
// Resolve effective sessions and context builder.
|
||||
// When a workspace override is provided (e.g. from magicform channel),
|
||||
// create temporary instances pointing at the override path for full isolation.
|
||||
effSessions := agent.Sessions
|
||||
effContextBuilder := agent.ContextBuilder
|
||||
|
||||
if opts.WorkspaceOverride != "" {
|
||||
wp := opts.WorkspaceOverride
|
||||
os.MkdirAll(wp, 0o755)
|
||||
effSessions = session.NewSessionManager(filepath.Join(wp, "sessions"))
|
||||
effContextBuilder = NewContextBuilder(wp)
|
||||
}
|
||||
|
||||
// Copy bootstrap files from config-dir to workspace
|
||||
if opts.ConfigDir != "" && opts.WorkspaceOverride != "" {
|
||||
CopyBootstrapFiles(opts.ConfigDir, opts.WorkspaceOverride)
|
||||
}
|
||||
|
||||
// Load workspace-local config.json for per-request overrides
|
||||
configSource := opts.ConfigDir
|
||||
if configSource == "" {
|
||||
configSource = opts.WorkspaceOverride // fallback: check workspace itself
|
||||
}
|
||||
if configSource != "" {
|
||||
if wc, err := config.LoadWorkspaceConfig(configSource); err != nil {
|
||||
logger.WarnCF("agent", "Failed to load workspace config",
|
||||
map[string]any{"path": configSource, "error": err.Error()})
|
||||
} else if wc != nil {
|
||||
tmpCfg := al.cfg.Clone()
|
||||
tmpCfg.MergeWorkspaceConfig(wc)
|
||||
if tmpCfg.Agents.Defaults.GetModelName() == "" {
|
||||
tmpCfg.Agents.Defaults.ModelName = agent.Model
|
||||
}
|
||||
if provider, modelID, err := providers.CreateProvider(tmpCfg); err != nil {
|
||||
logger.ErrorCF("agent", "Failed to create workspace provider",
|
||||
map[string]any{"path": configSource, "error": err.Error()})
|
||||
} else {
|
||||
opts.effProvider = provider
|
||||
opts.effModel = modelID
|
||||
if sp, ok := provider.(providers.StatefulProvider); ok {
|
||||
defer sp.Close()
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Apply skills filter unconditionally — works with or without workspace override
|
||||
if len(opts.AllowedSkills) > 0 {
|
||||
effContextBuilder.SetSkillsFilter(opts.AllowedSkills)
|
||||
}
|
||||
|
||||
// 1. Build messages (skip history for heartbeat)
|
||||
var history []providers.Message
|
||||
var summary string
|
||||
if !opts.NoHistory {
|
||||
history = agent.Sessions.GetHistory(opts.SessionKey)
|
||||
summary = agent.Sessions.GetSummary(opts.SessionKey)
|
||||
history = effSessions.GetHistory(opts.SessionKey)
|
||||
summary = effSessions.GetSummary(opts.SessionKey)
|
||||
}
|
||||
messages := agent.ContextBuilder.BuildMessages(
|
||||
messages := effContextBuilder.BuildMessages(
|
||||
history,
|
||||
summary,
|
||||
opts.UserMessage,
|
||||
|
|
@ -762,7 +855,11 @@ func (al *AgentLoop) runAgentLoop(
|
|||
messages = resolveMediaRefs(messages, al.mediaStore, maxMediaSize)
|
||||
|
||||
// 2. Save user message to session
|
||||
agent.Sessions.AddMessage(opts.SessionKey, "user", opts.UserMessage)
|
||||
effSessions.AddMessage(opts.SessionKey, "user", opts.UserMessage)
|
||||
|
||||
// Store effective sessions/context on opts so runLLMIteration can use them
|
||||
opts.effSessions = effSessions
|
||||
opts.effContextBuilder = effContextBuilder
|
||||
|
||||
// 3. Run LLM iteration loop
|
||||
finalContent, iteration, err := al.runLLMIteration(ctx, agent, messages, opts)
|
||||
|
|
@ -779,12 +876,20 @@ func (al *AgentLoop) runAgentLoop(
|
|||
}
|
||||
|
||||
// 5. Save final assistant message to session
|
||||
agent.Sessions.AddMessage(opts.SessionKey, "assistant", finalContent)
|
||||
agent.Sessions.Save(opts.SessionKey)
|
||||
effSessions.AddMessage(opts.SessionKey, "assistant", finalContent)
|
||||
effSessions.Save(opts.SessionKey)
|
||||
|
||||
// 6. Optional: summarization
|
||||
if opts.EnableSummary {
|
||||
al.maybeSummarize(agent, opts.SessionKey, opts.Channel, opts.ChatID)
|
||||
al.maybeSummarizeWith(
|
||||
effSessions,
|
||||
agent,
|
||||
opts.SessionKey,
|
||||
opts.Channel,
|
||||
opts.ChatID,
|
||||
opts.effProvider,
|
||||
opts.effModel,
|
||||
)
|
||||
}
|
||||
|
||||
// 7. Optional: send response via bus
|
||||
|
|
@ -875,11 +980,32 @@ func (al *AgentLoop) runLLMIteration(
|
|||
iteration := 0
|
||||
var finalContent string
|
||||
|
||||
// Resolve effective provider/model — workspace config overrides win
|
||||
effProvider := agent.Provider
|
||||
effModel := agent.Model
|
||||
if opts.effProvider != nil {
|
||||
effProvider = opts.effProvider
|
||||
}
|
||||
if opts.effModel != "" {
|
||||
effModel = opts.effModel
|
||||
}
|
||||
|
||||
// Determine effective model tier for this conversation turn.
|
||||
// selectCandidates evaluates routing once and the decision is sticky for
|
||||
// all tool-follow-up iterations within the same turn so that a multi-step
|
||||
// tool chain doesn't switch models mid-way through.
|
||||
activeCandidates, activeModel := al.selectCandidates(agent, opts.UserMessage, messages)
|
||||
var activeCandidates []providers.FallbackCandidate
|
||||
var activeModel string
|
||||
if opts.effProvider != nil {
|
||||
// Workspace overrides the provider — skip routing and fallback
|
||||
// candidates since they may reference different provider credentials.
|
||||
activeModel = effModel
|
||||
} else {
|
||||
activeCandidates, activeModel = al.selectCandidates(agent, opts.UserMessage, messages)
|
||||
if opts.effModel != "" {
|
||||
activeModel = effModel
|
||||
}
|
||||
}
|
||||
|
||||
for iteration < agent.MaxIterations {
|
||||
iteration++
|
||||
|
|
@ -891,8 +1017,21 @@ func (al *AgentLoop) runLLMIteration(
|
|||
"max": agent.MaxIterations,
|
||||
})
|
||||
|
||||
// Build tool definitions
|
||||
// Build tool definitions, filtered by AllowedTools if set
|
||||
providerToolDefs := agent.Tools.ToProviderDefs()
|
||||
if len(opts.AllowedTools) > 0 {
|
||||
allowSet := make(map[string]bool, len(opts.AllowedTools))
|
||||
for _, t := range opts.AllowedTools {
|
||||
allowSet[t] = true
|
||||
}
|
||||
filtered := providerToolDefs[:0]
|
||||
for _, td := range providerToolDefs {
|
||||
if allowSet[td.Function.Name] {
|
||||
filtered = append(filtered, td)
|
||||
}
|
||||
}
|
||||
providerToolDefs = filtered
|
||||
}
|
||||
|
||||
// Log LLM request details
|
||||
logger.DebugCF("agent", "LLM request",
|
||||
|
|
@ -927,7 +1066,7 @@ func (al *AgentLoop) runLLMIteration(
|
|||
// parseThinkingLevel guarantees ThinkingOff for empty/unknown values,
|
||||
// so checking != ThinkingOff is sufficient.
|
||||
if agent.ThinkingLevel != ThinkingOff {
|
||||
if tc, ok := agent.Provider.(providers.ThinkingCapable); ok && tc.SupportsThinking() {
|
||||
if tc, ok := effProvider.(providers.ThinkingCapable); ok && tc.SupportsThinking() {
|
||||
llmOpts["thinking_level"] = string(agent.ThinkingLevel)
|
||||
} else {
|
||||
logger.WarnCF("agent", "thinking_level is set but current provider does not support it, ignoring",
|
||||
|
|
@ -941,7 +1080,7 @@ func (al *AgentLoop) runLLMIteration(
|
|||
ctx,
|
||||
activeCandidates,
|
||||
func(ctx context.Context, provider, model string) (*providers.LLMResponse, error) {
|
||||
return agent.Provider.Chat(ctx, messages, providerToolDefs, model, llmOpts)
|
||||
return effProvider.Chat(ctx, messages, providerToolDefs, model, llmOpts)
|
||||
},
|
||||
)
|
||||
if fbErr != nil {
|
||||
|
|
@ -957,7 +1096,7 @@ func (al *AgentLoop) runLLMIteration(
|
|||
}
|
||||
return fbResult.Response, nil
|
||||
}
|
||||
return agent.Provider.Chat(ctx, messages, providerToolDefs, activeModel, llmOpts)
|
||||
return effProvider.Chat(ctx, messages, providerToolDefs, activeModel, llmOpts)
|
||||
}
|
||||
|
||||
// Retry loop for context/token errors
|
||||
|
|
@ -1017,10 +1156,10 @@ func (al *AgentLoop) runLLMIteration(
|
|||
})
|
||||
}
|
||||
|
||||
al.forceCompression(agent, opts.SessionKey)
|
||||
newHistory := agent.Sessions.GetHistory(opts.SessionKey)
|
||||
newSummary := agent.Sessions.GetSummary(opts.SessionKey)
|
||||
messages = agent.ContextBuilder.BuildMessages(
|
||||
al.forceCompressionWith(opts.effSessions, agent, opts.SessionKey)
|
||||
newHistory := opts.effSessions.GetHistory(opts.SessionKey)
|
||||
newSummary := opts.effSessions.GetSummary(opts.SessionKey)
|
||||
messages = opts.effContextBuilder.BuildMessages(
|
||||
newHistory, newSummary, "",
|
||||
nil, opts.Channel, opts.ChatID,
|
||||
)
|
||||
|
|
@ -1117,7 +1256,7 @@ func (al *AgentLoop) runLLMIteration(
|
|||
messages = append(messages, assistantMsg)
|
||||
|
||||
// Save assistant message with tool calls to session
|
||||
agent.Sessions.AddFullMessage(opts.SessionKey, assistantMsg)
|
||||
opts.effSessions.AddFullMessage(opts.SessionKey, assistantMsg)
|
||||
|
||||
// Execute tool calls in parallel
|
||||
type indexedAgentResult struct {
|
||||
|
|
@ -1144,6 +1283,24 @@ func (al *AgentLoop) runLLMIteration(
|
|||
"iteration": iteration,
|
||||
})
|
||||
|
||||
// Enforce tool allowlist at execution time (defense-in-depth)
|
||||
if len(opts.AllowedTools) > 0 {
|
||||
allowed := false
|
||||
for _, t := range opts.AllowedTools {
|
||||
if t == tc.Name {
|
||||
allowed = true
|
||||
break
|
||||
}
|
||||
}
|
||||
if !allowed {
|
||||
agentResults[idx].result = &tools.ToolResult{
|
||||
ForLLM: fmt.Sprintf("Tool %q is not allowed for this request", tc.Name),
|
||||
IsError: true,
|
||||
}
|
||||
return
|
||||
}
|
||||
}
|
||||
|
||||
// Create async callback for tools that implement AsyncExecutor
|
||||
asyncCallback := func(callbackCtx context.Context, result *tools.ToolResult) {
|
||||
if !result.Silent && result.ForUser != "" {
|
||||
|
|
@ -1219,7 +1376,7 @@ func (al *AgentLoop) runLLMIteration(
|
|||
messages = append(messages, toolResultMsg)
|
||||
|
||||
// Save tool result message to session
|
||||
agent.Sessions.AddFullMessage(opts.SessionKey, toolResultMsg)
|
||||
opts.effSessions.AddFullMessage(opts.SessionKey, toolResultMsg)
|
||||
}
|
||||
}
|
||||
|
||||
|
|
@ -1264,9 +1421,16 @@ func (al *AgentLoop) selectCandidates(
|
|||
return agent.LightCandidates, agent.Router.LightModel()
|
||||
}
|
||||
|
||||
// maybeSummarize triggers summarization if the session history exceeds thresholds.
|
||||
func (al *AgentLoop) maybeSummarize(agent *AgentInstance, sessionKey, channel, chatID string) {
|
||||
newHistory := agent.Sessions.GetHistory(sessionKey)
|
||||
// maybeSummarizeWith triggers summarization if the session history exceeds thresholds.
|
||||
// and optional per-request provider/model overrides.
|
||||
func (al *AgentLoop) maybeSummarizeWith(
|
||||
sessions *session.SessionManager,
|
||||
agent *AgentInstance,
|
||||
sessionKey, channel, chatID string,
|
||||
effProvider providers.LLMProvider,
|
||||
effModel string,
|
||||
) {
|
||||
newHistory := sessions.GetHistory(sessionKey)
|
||||
tokenEstimate := al.estimateTokens(newHistory)
|
||||
threshold := agent.ContextWindow * agent.SummarizeTokenPercent / 100
|
||||
|
||||
|
|
@ -1276,16 +1440,16 @@ func (al *AgentLoop) maybeSummarize(agent *AgentInstance, sessionKey, channel, c
|
|||
go func() {
|
||||
defer al.summarizing.Delete(summarizeKey)
|
||||
logger.Debug("Memory threshold reached. Optimizing conversation history...")
|
||||
al.summarizeSession(agent, sessionKey)
|
||||
al.summarizeSessionWith(sessions, agent, sessionKey, effProvider, effModel)
|
||||
}()
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// forceCompression aggressively reduces context when the limit is hit.
|
||||
// forceCompressionWith aggressively reduces context when the limit is hit.
|
||||
// It drops the oldest 50% of messages (keeping system prompt and last user message).
|
||||
func (al *AgentLoop) forceCompression(agent *AgentInstance, sessionKey string) {
|
||||
history := agent.Sessions.GetHistory(sessionKey)
|
||||
func (al *AgentLoop) forceCompressionWith(sessions *session.SessionManager, agent *AgentInstance, sessionKey string) {
|
||||
history := sessions.GetHistory(sessionKey)
|
||||
if len(history) <= 4 {
|
||||
return
|
||||
}
|
||||
|
|
@ -1325,8 +1489,8 @@ func (al *AgentLoop) forceCompression(agent *AgentInstance, sessionKey string) {
|
|||
newHistory = append(newHistory, history[len(history)-1]) // Last message
|
||||
|
||||
// Update session
|
||||
agent.Sessions.SetHistory(sessionKey, newHistory)
|
||||
agent.Sessions.Save(sessionKey)
|
||||
sessions.SetHistory(sessionKey, newHistory)
|
||||
sessions.Save(sessionKey)
|
||||
|
||||
logger.WarnCF("agent", "Forced compression executed", map[string]any{
|
||||
"session_key": sessionKey,
|
||||
|
|
@ -1422,13 +1586,29 @@ func formatToolsForLog(toolDefs []providers.ToolDefinition) string {
|
|||
return sb.String()
|
||||
}
|
||||
|
||||
// summarizeSession summarizes the conversation history for a session.
|
||||
func (al *AgentLoop) summarizeSession(agent *AgentInstance, sessionKey string) {
|
||||
// summarizeSessionWith summarizes the conversation history for a session.
|
||||
func (al *AgentLoop) summarizeSessionWith(
|
||||
sessions *session.SessionManager,
|
||||
agent *AgentInstance,
|
||||
sessionKey string,
|
||||
effProvider providers.LLMProvider,
|
||||
effModel string,
|
||||
) {
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 120*time.Second)
|
||||
defer cancel()
|
||||
|
||||
history := agent.Sessions.GetHistory(sessionKey)
|
||||
summary := agent.Sessions.GetSummary(sessionKey)
|
||||
// Resolve effective provider/model
|
||||
sumProvider := agent.Provider
|
||||
sumModel := agent.Model
|
||||
if effProvider != nil {
|
||||
sumProvider = effProvider
|
||||
}
|
||||
if effModel != "" {
|
||||
sumModel = effModel
|
||||
}
|
||||
|
||||
history := sessions.GetHistory(sessionKey)
|
||||
summary := sessions.GetSummary(sessionKey)
|
||||
|
||||
// Keep last 4 messages for continuity
|
||||
if len(history) <= 4 {
|
||||
|
|
@ -1465,19 +1645,19 @@ func (al *AgentLoop) summarizeSession(agent *AgentInstance, sessionKey string) {
|
|||
part1 := validMessages[:mid]
|
||||
part2 := validMessages[mid:]
|
||||
|
||||
s1, _ := al.summarizeBatch(ctx, agent, part1, "")
|
||||
s2, _ := al.summarizeBatch(ctx, agent, part2, "")
|
||||
s1, _ := al.summarizeBatch(ctx, sumProvider, sumModel, agent.ID, part1, "")
|
||||
s2, _ := al.summarizeBatch(ctx, sumProvider, sumModel, agent.ID, part2, "")
|
||||
|
||||
mergePrompt := fmt.Sprintf(
|
||||
"Merge these two conversation summaries into one cohesive summary:\n\n1: %s\n\n2: %s",
|
||||
s1,
|
||||
s2,
|
||||
)
|
||||
resp, err := agent.Provider.Chat(
|
||||
resp, err := sumProvider.Chat(
|
||||
ctx,
|
||||
[]providers.Message{{Role: "user", Content: mergePrompt}},
|
||||
nil,
|
||||
agent.Model,
|
||||
sumModel,
|
||||
map[string]any{
|
||||
"max_tokens": 1024,
|
||||
"temperature": 0.3,
|
||||
|
|
@ -1490,7 +1670,7 @@ func (al *AgentLoop) summarizeSession(agent *AgentInstance, sessionKey string) {
|
|||
finalSummary = s1 + " " + s2
|
||||
}
|
||||
} else {
|
||||
finalSummary, _ = al.summarizeBatch(ctx, agent, validMessages, summary)
|
||||
finalSummary, _ = al.summarizeBatch(ctx, sumProvider, sumModel, agent.ID, validMessages, summary)
|
||||
}
|
||||
|
||||
if omitted && finalSummary != "" {
|
||||
|
|
@ -1498,16 +1678,18 @@ func (al *AgentLoop) summarizeSession(agent *AgentInstance, sessionKey string) {
|
|||
}
|
||||
|
||||
if finalSummary != "" {
|
||||
agent.Sessions.SetSummary(sessionKey, finalSummary)
|
||||
agent.Sessions.TruncateHistory(sessionKey, 4)
|
||||
agent.Sessions.Save(sessionKey)
|
||||
sessions.SetSummary(sessionKey, finalSummary)
|
||||
sessions.TruncateHistory(sessionKey, 4)
|
||||
sessions.Save(sessionKey)
|
||||
}
|
||||
}
|
||||
|
||||
// summarizeBatch summarizes a batch of messages.
|
||||
// summarizeBatch summarizes a batch of messages using the given provider/model.
|
||||
func (al *AgentLoop) summarizeBatch(
|
||||
ctx context.Context,
|
||||
agent *AgentInstance,
|
||||
provider providers.LLMProvider,
|
||||
model string,
|
||||
agentID string,
|
||||
batch []providers.Message,
|
||||
existingSummary string,
|
||||
) (string, error) {
|
||||
|
|
@ -1526,15 +1708,15 @@ func (al *AgentLoop) summarizeBatch(
|
|||
}
|
||||
prompt := sb.String()
|
||||
|
||||
response, err := agent.Provider.Chat(
|
||||
response, err := provider.Chat(
|
||||
ctx,
|
||||
[]providers.Message{{Role: "user", Content: prompt}},
|
||||
nil,
|
||||
agent.Model,
|
||||
model,
|
||||
map[string]any{
|
||||
"max_tokens": 1024,
|
||||
"temperature": 0.3,
|
||||
"prompt_cache_key": agent.ID,
|
||||
"prompt_cache_key": agentID,
|
||||
},
|
||||
)
|
||||
if err != nil {
|
||||
|
|
|
|||
|
|
@ -18,8 +18,12 @@ import (
|
|||
"strconv"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/sipeed/picoclaw/pkg/logger"
|
||||
)
|
||||
|
||||
const oauthMaxResponseSize int64 = 1 << 20 // 1 MB — more than sufficient for any OAuth response
|
||||
|
||||
type OAuthProviderConfig struct {
|
||||
Issuer string
|
||||
ClientID string
|
||||
|
|
@ -33,7 +37,7 @@ type OAuthProviderConfig struct {
|
|||
func OpenAIOAuthConfig() OAuthProviderConfig {
|
||||
return OAuthProviderConfig{
|
||||
Issuer: "https://auth.openai.com",
|
||||
ClientID: "app_EMoamEEZ73f0CkXaXp7hrann",
|
||||
ClientID: getEnvOrDefault("PICOCLAW_OPENAI_CLIENT_ID", "app_EMoamEEZ73f0CkXaXp7hrann"),
|
||||
Scopes: "openid profile email offline_access",
|
||||
Originator: "codex_cli_rs",
|
||||
Port: 1455,
|
||||
|
|
@ -43,11 +47,16 @@ func OpenAIOAuthConfig() OAuthProviderConfig {
|
|||
// GoogleAntigravityOAuthConfig returns the OAuth configuration for Google Cloud Code Assist (Antigravity).
|
||||
// Client credentials are the same ones used by OpenCode/pi-ai for Cloud Code Assist access.
|
||||
func GoogleAntigravityOAuthConfig() OAuthProviderConfig {
|
||||
// These are the same client credentials used by the OpenCode antigravity plugin.
|
||||
clientID := decodeBase64(
|
||||
"MTA3MTAwNjA2MDU5MS10bWhzc2luMmgyMWxjcmUyMzV2dG9sb2poNGc0MDNlcC5hcHBzLmdvb2dsZXVzZXJjb250ZW50LmNvbQ==",
|
||||
clientID := getEnvOrDefault(
|
||||
"PICOCLAW_GOOGLE_CLIENT_ID",
|
||||
decodeBase64(
|
||||
"MTA3MTAwNjA2MDU5MS10bWhzc2luMmgyMWxjcmUyMzV2dG9sb2poNGc0MDNlcC5hcHBzLmdvb2dsZXVzZXJjb250ZW50LmNvbQ==",
|
||||
),
|
||||
)
|
||||
clientSecret := getEnvOrDefault(
|
||||
"PICOCLAW_GOOGLE_CLIENT_SECRET",
|
||||
decodeBase64("R09DU1BYLUs1OEZXUjQ4NkxkTEoxbUxCOHNYQzR6NnFEQWY="),
|
||||
)
|
||||
clientSecret := decodeBase64("R09DU1BYLUs1OEZXUjQ4NkxkTEoxbUxCOHNYQzR6NnFEQWY=")
|
||||
return OAuthProviderConfig{
|
||||
Issuer: "https://accounts.google.com/o/oauth2/v2",
|
||||
TokenURL: "https://oauth2.googleapis.com/token",
|
||||
|
|
@ -58,6 +67,13 @@ func GoogleAntigravityOAuthConfig() OAuthProviderConfig {
|
|||
}
|
||||
}
|
||||
|
||||
func getEnvOrDefault(key, fallback string) string {
|
||||
if v := os.Getenv(key); v != "" {
|
||||
return v
|
||||
}
|
||||
return fallback
|
||||
}
|
||||
|
||||
func decodeBase64(s string) string {
|
||||
data, err := base64.StdEncoding.DecodeString(s)
|
||||
if err != nil {
|
||||
|
|
@ -212,12 +228,14 @@ func RequestDeviceCode(cfg OAuthProviderConfig) (*DeviceCodeInfo, error) {
|
|||
}
|
||||
defer resp.Body.Close()
|
||||
|
||||
body, err := io.ReadAll(resp.Body)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("reading device code response: %w", err)
|
||||
}
|
||||
body, _ := io.ReadAll(io.LimitReader(resp.Body, oauthMaxResponseSize))
|
||||
if resp.StatusCode != http.StatusOK {
|
||||
return nil, fmt.Errorf("device code request failed: %s", string(body))
|
||||
logger.DebugCF(
|
||||
"auth",
|
||||
"device code request failed",
|
||||
map[string]any{"status": resp.StatusCode, "body": string(body)},
|
||||
)
|
||||
return nil, fmt.Errorf("device code request failed (HTTP %d)", resp.StatusCode)
|
||||
}
|
||||
|
||||
deviceResp, err := parseDeviceCodeResponse(body)
|
||||
|
|
@ -303,12 +321,14 @@ func LoginDeviceCode(cfg OAuthProviderConfig) (*AuthCredential, error) {
|
|||
}
|
||||
defer resp.Body.Close()
|
||||
|
||||
body, err := io.ReadAll(resp.Body)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("reading device code response: %w", err)
|
||||
}
|
||||
body, _ := io.ReadAll(io.LimitReader(resp.Body, oauthMaxResponseSize))
|
||||
if resp.StatusCode != http.StatusOK {
|
||||
return nil, fmt.Errorf("device code request failed: %s", string(body))
|
||||
logger.DebugCF(
|
||||
"auth",
|
||||
"device code request failed",
|
||||
map[string]any{"status": resp.StatusCode, "body": string(body)},
|
||||
)
|
||||
return nil, fmt.Errorf("device code request failed (HTTP %d)", resp.StatusCode)
|
||||
}
|
||||
|
||||
deviceResp, err := parseDeviceCodeResponse(body)
|
||||
|
|
@ -366,10 +386,7 @@ func pollDeviceCode(cfg OAuthProviderConfig, deviceAuthID, userCode string) (*Au
|
|||
return nil, fmt.Errorf("pending")
|
||||
}
|
||||
|
||||
body, err := io.ReadAll(resp.Body)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("reading device token response: %w", err)
|
||||
}
|
||||
body, _ := io.ReadAll(io.LimitReader(resp.Body, oauthMaxResponseSize))
|
||||
|
||||
var tokenResp struct {
|
||||
AuthorizationCode string `json:"authorization_code"`
|
||||
|
|
@ -410,12 +427,10 @@ func RefreshAccessToken(cred *AuthCredential, cfg OAuthProviderConfig) (*AuthCre
|
|||
}
|
||||
defer resp.Body.Close()
|
||||
|
||||
body, err := io.ReadAll(resp.Body)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("reading token refresh response: %w", err)
|
||||
}
|
||||
body, _ := io.ReadAll(io.LimitReader(resp.Body, oauthMaxResponseSize))
|
||||
if resp.StatusCode != http.StatusOK {
|
||||
return nil, fmt.Errorf("token refresh failed: %s", string(body))
|
||||
logger.DebugCF("auth", "token refresh failed", map[string]any{"status": resp.StatusCode, "body": string(body)})
|
||||
return nil, fmt.Errorf("token refresh failed (HTTP %d)", resp.StatusCode)
|
||||
}
|
||||
|
||||
refreshed, err := parseTokenResponse(body, cred.Provider)
|
||||
|
|
@ -506,12 +521,10 @@ func ExchangeCodeForTokens(cfg OAuthProviderConfig, code, codeVerifier, redirect
|
|||
}
|
||||
defer resp.Body.Close()
|
||||
|
||||
body, err := io.ReadAll(resp.Body)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("reading token exchange response: %w", err)
|
||||
}
|
||||
body, _ := io.ReadAll(io.LimitReader(resp.Body, oauthMaxResponseSize))
|
||||
if resp.StatusCode != http.StatusOK {
|
||||
return nil, fmt.Errorf("token exchange failed: %s", string(body))
|
||||
logger.DebugCF("auth", "token exchange failed", map[string]any{"status": resp.StatusCode, "body": string(body)})
|
||||
return nil, fmt.Errorf("token exchange failed (HTTP %d)", resp.StatusCode)
|
||||
}
|
||||
|
||||
return parseTokenResponse(body, provider)
|
||||
|
|
|
|||
|
|
@ -301,6 +301,10 @@ func (c *BaseChannel) HandleMessage(
|
|||
}
|
||||
}
|
||||
|
||||
// Bus returns the underlying MessageBus. This is used by channels that need to
|
||||
// publish directly (e.g. to set SessionKey on InboundMessage).
|
||||
func (c *BaseChannel) Bus() *bus.MessageBus { return c.bus }
|
||||
|
||||
func (c *BaseChannel) SetRunning(running bool) {
|
||||
c.running.Store(running)
|
||||
}
|
||||
|
|
|
|||
13
pkg/channels/magicform/init.go
Normal file
13
pkg/channels/magicform/init.go
Normal file
|
|
@ -0,0 +1,13 @@
|
|||
package magicform
|
||||
|
||||
import (
|
||||
"github.com/sipeed/picoclaw/pkg/bus"
|
||||
"github.com/sipeed/picoclaw/pkg/channels"
|
||||
"github.com/sipeed/picoclaw/pkg/config"
|
||||
)
|
||||
|
||||
func init() {
|
||||
channels.RegisterFactory("magicform", func(cfg *config.Config, b *bus.MessageBus) (channels.Channel, error) {
|
||||
return NewMagicFormChannel(cfg.Channels.MagicForm, b)
|
||||
})
|
||||
}
|
||||
420
pkg/channels/magicform/magicform.go
Normal file
420
pkg/channels/magicform/magicform.go
Normal file
|
|
@ -0,0 +1,420 @@
|
|||
package magicform
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"crypto/subtle"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"io"
|
||||
"net/http"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"sync"
|
||||
"time"
|
||||
|
||||
"github.com/sipeed/picoclaw/pkg/bus"
|
||||
"github.com/sipeed/picoclaw/pkg/channels"
|
||||
"github.com/sipeed/picoclaw/pkg/config"
|
||||
"github.com/sipeed/picoclaw/pkg/logger"
|
||||
)
|
||||
|
||||
// WebhookPayload is the inbound payload from MagicForm.
|
||||
type WebhookPayload struct {
|
||||
StackID string `json:"stackId"`
|
||||
ConversationID string `json:"conversationId"`
|
||||
UserID string `json:"userId"`
|
||||
Message string `json:"message"`
|
||||
Workspace string `json:"workspace"` // e.g. "s1/c1" — agent working directory (relative to workspace_root)
|
||||
ConfigDir string `json:"configDir,omitempty"` // e.g. "s1/config" — pre-provisioned config directory (relative to workspace_root)
|
||||
CallbackURL string `json:"callbackUrl"`
|
||||
|
||||
// Tool/skill filtering
|
||||
AllowedTools []string `json:"allowedTools,omitempty"` // Tool allowlist (empty = all)
|
||||
AllowedSkills []string `json:"allowedSkills,omitempty"` // Skill filter (empty = all)
|
||||
}
|
||||
|
||||
// CallbackPayload is the outbound payload sent back to MagicForm.
|
||||
type CallbackPayload struct {
|
||||
StackID string `json:"stackId"`
|
||||
ConversationID string `json:"conversationId"`
|
||||
Response string `json:"response"`
|
||||
Type string `json:"type"` // "final"
|
||||
}
|
||||
|
||||
// requestContext stores per-request state so Send() can resolve callback info.
|
||||
type requestContext struct {
|
||||
stackID string
|
||||
conversationID string
|
||||
userID string
|
||||
callbackURL string
|
||||
createdAt time.Time
|
||||
}
|
||||
|
||||
// MagicFormChannel implements the MagicForm channel plugin.
|
||||
type MagicFormChannel struct {
|
||||
*channels.BaseChannel
|
||||
config config.MagicFormConfig
|
||||
httpClient *http.Client
|
||||
requests sync.Map // chatID → *requestContext
|
||||
ctx context.Context
|
||||
cancel context.CancelFunc
|
||||
}
|
||||
|
||||
// NewMagicFormChannel creates a new MagicForm channel.
|
||||
func NewMagicFormChannel(cfg config.MagicFormConfig, msgBus *bus.MessageBus) (*MagicFormChannel, error) {
|
||||
base := channels.NewBaseChannel(
|
||||
"magicform",
|
||||
cfg,
|
||||
msgBus,
|
||||
cfg.AllowFrom,
|
||||
)
|
||||
|
||||
ctx, cancel := context.WithCancel(context.Background())
|
||||
|
||||
ch := &MagicFormChannel{
|
||||
BaseChannel: base,
|
||||
config: cfg,
|
||||
httpClient: &http.Client{
|
||||
Timeout: 30 * time.Second,
|
||||
},
|
||||
ctx: ctx,
|
||||
cancel: cancel,
|
||||
}
|
||||
|
||||
base.SetOwner(ch)
|
||||
return ch, nil
|
||||
}
|
||||
|
||||
// Start begins the channel and starts the TTL cleanup goroutine.
|
||||
func (c *MagicFormChannel) Start(_ context.Context) error {
|
||||
c.SetRunning(true)
|
||||
|
||||
// Background goroutine to clean up stale request contexts
|
||||
go c.cleanupLoop()
|
||||
|
||||
logger.InfoCF("magicform", "MagicForm channel started", nil)
|
||||
return nil
|
||||
}
|
||||
|
||||
// Stop shuts down the channel.
|
||||
func (c *MagicFormChannel) Stop(_ context.Context) error {
|
||||
c.cancel()
|
||||
c.SetRunning(false)
|
||||
logger.InfoCF("magicform", "MagicForm channel stopped", nil)
|
||||
return nil
|
||||
}
|
||||
|
||||
// WebhookPath returns the HTTP path for the inbound webhook.
|
||||
func (c *MagicFormChannel) WebhookPath() string {
|
||||
if c.config.WebhookPath != "" {
|
||||
return c.config.WebhookPath
|
||||
}
|
||||
return "/hooks/magicform"
|
||||
}
|
||||
|
||||
// HealthPath returns the HTTP path for the health check endpoint.
|
||||
func (c *MagicFormChannel) HealthPath() string {
|
||||
return "/health/magicform"
|
||||
}
|
||||
|
||||
// HealthHandler handles the health check HTTP request.
|
||||
func (c *MagicFormChannel) HealthHandler(w http.ResponseWriter, _ *http.Request) {
|
||||
w.Header().Set("Content-Type", "application/json")
|
||||
w.WriteHeader(http.StatusOK)
|
||||
json.NewEncoder(w).Encode(map[string]string{
|
||||
"status": "ok",
|
||||
"channel": "magicform",
|
||||
})
|
||||
}
|
||||
|
||||
// maxWebhookBodySize is the maximum allowed size for inbound webhook payloads (1 MB).
|
||||
const maxWebhookBodySize = 1 << 20
|
||||
|
||||
// ServeHTTP handles inbound webhook requests from MagicForm.
|
||||
func (c *MagicFormChannel) ServeHTTP(w http.ResponseWriter, r *http.Request) {
|
||||
if r.Method != http.MethodPost {
|
||||
http.Error(w, "Method not allowed", http.StatusMethodNotAllowed)
|
||||
return
|
||||
}
|
||||
|
||||
// Limit request body size to prevent memory exhaustion
|
||||
r.Body = http.MaxBytesReader(w, r.Body, maxWebhookBodySize)
|
||||
body, err := io.ReadAll(r.Body)
|
||||
if err != nil {
|
||||
http.Error(w, "Request body too large", http.StatusRequestEntityTooLarge)
|
||||
return
|
||||
}
|
||||
defer r.Body.Close()
|
||||
|
||||
// Verify Bearer token
|
||||
if !c.verifyToken(r) {
|
||||
http.Error(w, "Unauthorized", http.StatusUnauthorized)
|
||||
return
|
||||
}
|
||||
|
||||
var payload WebhookPayload
|
||||
if err := json.Unmarshal(body, &payload); err != nil {
|
||||
http.Error(w, "Invalid JSON", http.StatusBadRequest)
|
||||
return
|
||||
}
|
||||
|
||||
// Validate required fields
|
||||
if payload.StackID == "" || payload.ConversationID == "" || payload.Message == "" {
|
||||
http.Error(w, "Missing required fields: stackId, conversationId, message", http.StatusBadRequest)
|
||||
return
|
||||
}
|
||||
|
||||
// Validate workspace path: must be relative and resolve under workspace_root
|
||||
if payload.Workspace != "" {
|
||||
resolved, err := c.resolveWorkspace(payload.Workspace)
|
||||
if err != nil {
|
||||
http.Error(w, fmt.Sprintf("Invalid workspace: %v", err), http.StatusBadRequest)
|
||||
return
|
||||
}
|
||||
payload.Workspace = resolved
|
||||
}
|
||||
|
||||
// Validate configDir path against workspace_root
|
||||
if payload.ConfigDir != "" {
|
||||
resolved, err := c.resolveWorkspace(payload.ConfigDir)
|
||||
if err != nil {
|
||||
http.Error(w, fmt.Sprintf("Invalid configDir: %v", err), http.StatusBadRequest)
|
||||
return
|
||||
}
|
||||
payload.ConfigDir = resolved
|
||||
}
|
||||
|
||||
// Return 200 immediately, process asynchronously
|
||||
w.WriteHeader(http.StatusOK)
|
||||
|
||||
go c.processWebhook(c.ctx, payload)
|
||||
}
|
||||
|
||||
// resolveWorkspace validates and resolves the workspace path.
|
||||
// If workspace_root is configured, the workspace must be a relative path that
|
||||
// resolves under the root. If workspace_root is not configured, workspace is
|
||||
// rejected (no arbitrary path writes allowed).
|
||||
func (c *MagicFormChannel) resolveWorkspace(workspace string) (string, error) {
|
||||
root := c.config.WorkspaceRoot
|
||||
if root == "" {
|
||||
return "", fmt.Errorf("workspace_root not configured; workspace overrides are not allowed")
|
||||
}
|
||||
|
||||
absRoot, err := filepath.Abs(root)
|
||||
if err != nil {
|
||||
return "", fmt.Errorf("invalid workspace_root: %w", err)
|
||||
}
|
||||
|
||||
// Join the root with the provided workspace (which may be relative)
|
||||
resolved := filepath.Join(absRoot, workspace)
|
||||
resolved = filepath.Clean(resolved)
|
||||
|
||||
// Ensure the resolved path is under the root (prevents ../../../etc traversal)
|
||||
if !strings.HasPrefix(resolved, absRoot+string(filepath.Separator)) && resolved != absRoot {
|
||||
return "", fmt.Errorf("workspace path escapes workspace_root")
|
||||
}
|
||||
|
||||
return resolved, nil
|
||||
}
|
||||
|
||||
// verifyToken checks the Authorization Bearer token using constant-time comparison.
|
||||
func (c *MagicFormChannel) verifyToken(r *http.Request) bool {
|
||||
if c.config.Token == "" {
|
||||
return true // No token configured = allow all (dev mode)
|
||||
}
|
||||
|
||||
auth := r.Header.Get("Authorization")
|
||||
if !strings.HasPrefix(auth, "Bearer ") {
|
||||
return false
|
||||
}
|
||||
|
||||
token := strings.TrimPrefix(auth, "Bearer ")
|
||||
return subtle.ConstantTimeCompare([]byte(token), []byte(c.config.Token)) == 1
|
||||
}
|
||||
|
||||
// processWebhook handles an inbound webhook payload asynchronously.
|
||||
func (c *MagicFormChannel) processWebhook(ctx context.Context, p WebhookPayload) {
|
||||
// Check for shutdown before doing any work
|
||||
if ctx.Err() != nil {
|
||||
logger.WarnCF("magicform", "Skipping webhook processing: channel shutting down",
|
||||
map[string]any{"stack_id": p.StackID, "conversation_id": p.ConversationID})
|
||||
return
|
||||
}
|
||||
|
||||
chatID := "magicform:" + p.ConversationID
|
||||
senderID := p.UserID
|
||||
if senderID == "" {
|
||||
senderID = "anonymous"
|
||||
}
|
||||
|
||||
// Store request context for Send() to look up later
|
||||
c.requests.Store(chatID, &requestContext{
|
||||
stackID: p.StackID,
|
||||
conversationID: p.ConversationID,
|
||||
userID: p.UserID,
|
||||
callbackURL: p.CallbackURL,
|
||||
createdAt: time.Now(),
|
||||
})
|
||||
|
||||
peer := bus.Peer{Kind: "direct", ID: p.ConversationID}
|
||||
sender := bus.SenderInfo{
|
||||
Platform: "magicform",
|
||||
PlatformID: senderID,
|
||||
CanonicalID: "magicform:" + senderID,
|
||||
}
|
||||
|
||||
// Session key: per-stack per-conversation isolation
|
||||
sessionKey := fmt.Sprintf("agent:main:magicform:%s:%s", p.StackID, p.ConversationID)
|
||||
|
||||
metadata := map[string]string{
|
||||
"platform": "magicform",
|
||||
"stack_id": p.StackID,
|
||||
"conversation_id": p.ConversationID,
|
||||
}
|
||||
|
||||
if p.CallbackURL != "" {
|
||||
metadata["callback_url"] = p.CallbackURL
|
||||
}
|
||||
|
||||
// Workspace override — agent loop will pick this up
|
||||
if p.Workspace != "" {
|
||||
metadata["workspace_override"] = p.Workspace
|
||||
}
|
||||
|
||||
// Config directory — agent loop reads config.json and copies bootstrap files
|
||||
if p.ConfigDir != "" {
|
||||
metadata["config_dir"] = p.ConfigDir
|
||||
}
|
||||
|
||||
// Tool/skill filtering — passed via metadata, picked up by agent loop
|
||||
if len(p.AllowedTools) > 0 {
|
||||
metadata["allowed_tools"] = strings.Join(trimSlice(p.AllowedTools), ",")
|
||||
}
|
||||
if len(p.AllowedSkills) > 0 {
|
||||
metadata["allowed_skills"] = strings.Join(trimSlice(p.AllowedSkills), ",")
|
||||
}
|
||||
|
||||
messageID := fmt.Sprintf("mf-%s-%d", p.ConversationID, time.Now().UnixMilli())
|
||||
|
||||
// Build InboundMessage directly (not via HandleMessage) to set SessionKey.
|
||||
// MagicForm is API-to-API, so typing/reaction/placeholder don't apply.
|
||||
msg := bus.InboundMessage{
|
||||
Channel: "magicform",
|
||||
SenderID: sender.CanonicalID,
|
||||
Sender: sender,
|
||||
ChatID: chatID,
|
||||
Content: p.Message,
|
||||
Peer: peer,
|
||||
MessageID: messageID,
|
||||
SessionKey: sessionKey,
|
||||
Metadata: metadata,
|
||||
}
|
||||
|
||||
if err := c.Bus().PublishInbound(ctx, msg); err != nil {
|
||||
logger.ErrorCF("magicform", "Failed to publish inbound message",
|
||||
map[string]any{
|
||||
"chat_id": chatID,
|
||||
"stack_id": p.StackID,
|
||||
"conversation_id": p.ConversationID,
|
||||
"error": err.Error(),
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// trimSlice trims whitespace from each element in the slice.
|
||||
func trimSlice(s []string) []string {
|
||||
out := make([]string, len(s))
|
||||
for i, v := range s {
|
||||
out[i] = strings.TrimSpace(v)
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// Send delivers the agent response back to MagicForm via HTTP callback.
|
||||
func (c *MagicFormChannel) Send(ctx context.Context, msg bus.OutboundMessage) error {
|
||||
if !c.IsRunning() {
|
||||
return channels.ErrNotRunning
|
||||
}
|
||||
|
||||
// Look up request context
|
||||
val, ok := c.requests.LoadAndDelete(msg.ChatID)
|
||||
if !ok {
|
||||
return fmt.Errorf("%w: no request context for chatID %s", channels.ErrSendFailed, msg.ChatID)
|
||||
}
|
||||
reqCtx := val.(*requestContext)
|
||||
|
||||
// Resolve callback URL
|
||||
callbackURL := reqCtx.callbackURL
|
||||
if callbackURL == "" {
|
||||
callbackURL = c.config.BackendURL + "/claw-agent/callback"
|
||||
}
|
||||
|
||||
if callbackURL == "" {
|
||||
return fmt.Errorf("%w: no callback URL available", channels.ErrSendFailed)
|
||||
}
|
||||
|
||||
// Build callback payload
|
||||
payload := CallbackPayload{
|
||||
StackID: reqCtx.stackID,
|
||||
ConversationID: reqCtx.conversationID,
|
||||
Response: msg.Content,
|
||||
Type: "final",
|
||||
}
|
||||
|
||||
body, err := json.Marshal(payload)
|
||||
if err != nil {
|
||||
return fmt.Errorf("%w: marshal callback payload: %v", channels.ErrSendFailed, err)
|
||||
}
|
||||
|
||||
req, err := http.NewRequestWithContext(ctx, http.MethodPost, callbackURL, bytes.NewReader(body))
|
||||
if err != nil {
|
||||
return fmt.Errorf("%w: create callback request: %v", channels.ErrSendFailed, err)
|
||||
}
|
||||
req.Header.Set("Content-Type", "application/json")
|
||||
if c.config.Token != "" {
|
||||
req.Header.Set("Authorization", "Bearer "+c.config.Token)
|
||||
}
|
||||
|
||||
resp, err := c.httpClient.Do(req)
|
||||
if err != nil {
|
||||
return channels.ClassifyNetError(err)
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
|
||||
if resp.StatusCode >= 400 {
|
||||
respBody, _ := io.ReadAll(resp.Body)
|
||||
return channels.ClassifySendError(resp.StatusCode, fmt.Errorf("callback error: %s", respBody))
|
||||
}
|
||||
|
||||
logger.InfoCF("magicform", "Callback sent",
|
||||
map[string]any{
|
||||
"conversation_id": reqCtx.conversationID,
|
||||
"status": resp.StatusCode,
|
||||
})
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
// cleanupLoop periodically removes stale request contexts.
|
||||
func (c *MagicFormChannel) cleanupLoop() {
|
||||
ticker := time.NewTicker(60 * time.Second)
|
||||
defer ticker.Stop()
|
||||
|
||||
for {
|
||||
select {
|
||||
case <-c.ctx.Done():
|
||||
return
|
||||
case <-ticker.C:
|
||||
c.requests.Range(func(key, value any) bool {
|
||||
rc := value.(*requestContext)
|
||||
if time.Since(rc.createdAt) > 10*time.Minute {
|
||||
c.requests.Delete(key)
|
||||
logger.DebugCF("magicform", "Cleaned up stale request context",
|
||||
map[string]any{"chat_id": key})
|
||||
}
|
||||
return true
|
||||
})
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
@ -267,6 +267,10 @@ func (m *Manager) initChannels() error {
|
|||
m.initChannel("pico", "Pico")
|
||||
}
|
||||
|
||||
if m.config.Channels.MagicForm.Enabled && m.config.Channels.MagicForm.Token != "" {
|
||||
m.initChannel("magicform", "MagicForm")
|
||||
}
|
||||
|
||||
logger.InfoCF("channels", "Channel initialization completed", map[string]any{
|
||||
"enabled_channels": len(m.channels),
|
||||
})
|
||||
|
|
|
|||
|
|
@ -4,6 +4,7 @@ import (
|
|||
"encoding/json"
|
||||
"fmt"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"sync/atomic"
|
||||
|
||||
"github.com/caarlos0/env/v11"
|
||||
|
|
@ -231,6 +232,7 @@ type ChannelsConfig struct {
|
|||
WeComApp WeComAppConfig `json:"wecom_app"`
|
||||
WeComAIBot WeComAIBotConfig `json:"wecom_aibot"`
|
||||
Pico PicoConfig `json:"pico"`
|
||||
MagicForm MagicFormConfig `json:"magicform"`
|
||||
}
|
||||
|
||||
// GroupTriggerConfig controls when the bot responds in group chats.
|
||||
|
|
@ -414,6 +416,15 @@ type PicoConfig struct {
|
|||
Placeholder PlaceholderConfig `json:"placeholder,omitempty"`
|
||||
}
|
||||
|
||||
type MagicFormConfig struct {
|
||||
Enabled bool `json:"enabled" env:"PICOCLAW_CHANNELS_MAGICFORM_ENABLED"`
|
||||
Token string `json:"token" env:"PICOCLAW_CHANNELS_MAGICFORM_TOKEN"`
|
||||
BackendURL string `json:"backend_url" env:"PICOCLAW_CHANNELS_MAGICFORM_BACKEND_URL"`
|
||||
WebhookPath string `json:"webhook_path" env:"PICOCLAW_CHANNELS_MAGICFORM_WEBHOOK_PATH"`
|
||||
WorkspaceRoot string `json:"workspace_root" env:"PICOCLAW_CHANNELS_MAGICFORM_WORKSPACE_ROOT"`
|
||||
AllowFrom FlexibleStringSlice `json:"allow_from" env:"PICOCLAW_CHANNELS_MAGICFORM_ALLOW_FROM"`
|
||||
}
|
||||
|
||||
type HeartbeatConfig struct {
|
||||
Enabled bool `json:"enabled" env:"PICOCLAW_HEARTBEAT_ENABLED"`
|
||||
Interval int `json:"interval" env:"PICOCLAW_HEARTBEAT_INTERVAL"` // minutes, min 5
|
||||
|
|
@ -608,6 +619,7 @@ type ExecConfig struct {
|
|||
CustomDenyPatterns []string ` env:"PICOCLAW_TOOLS_EXEC_CUSTOM_DENY_PATTERNS" json:"custom_deny_patterns"`
|
||||
CustomAllowPatterns []string ` env:"PICOCLAW_TOOLS_EXEC_CUSTOM_ALLOW_PATTERNS" json:"custom_allow_patterns"`
|
||||
TimeoutSeconds int ` env:"PICOCLAW_TOOLS_EXEC_TIMEOUT_SECONDS" json:"timeout_seconds"` // 0 means use default (60s)
|
||||
FilterEnv bool ` env:"PICOCLAW_TOOLS_EXEC_FILTER_ENV" json:"filter_env"`
|
||||
}
|
||||
|
||||
type SkillsToolsConfig struct {
|
||||
|
|
@ -924,3 +936,160 @@ func (t *ToolsConfig) IsToolEnabled(name string) bool {
|
|||
return true
|
||||
}
|
||||
}
|
||||
|
||||
// WorkspaceConfig wraps a parsed Config along with the raw JSON bytes so that
|
||||
// merging can distinguish "field not present" from "field is zero-valued".
|
||||
type WorkspaceConfig struct {
|
||||
Config *Config
|
||||
rawJSON json.RawMessage
|
||||
}
|
||||
|
||||
// LoadWorkspaceConfig loads a workspace-local config.json from the given directory.
|
||||
// Returns nil, nil if the file does not exist.
|
||||
func LoadWorkspaceConfig(dir string) (*WorkspaceConfig, error) {
|
||||
path := filepath.Join(dir, "config.json")
|
||||
data, err := os.ReadFile(path)
|
||||
if err != nil {
|
||||
if os.IsNotExist(err) {
|
||||
return nil, nil
|
||||
}
|
||||
return nil, fmt.Errorf("reading workspace config %s: %w", path, err)
|
||||
}
|
||||
|
||||
var wc Config
|
||||
if err := json.Unmarshal(data, &wc); err != nil {
|
||||
return nil, fmt.Errorf("parsing workspace config %s: %w", path, err)
|
||||
}
|
||||
|
||||
for i := range wc.ModelList {
|
||||
if err := wc.ModelList[i].Validate(); err != nil {
|
||||
return nil, fmt.Errorf("workspace config model_list[%d]: %w", i, err)
|
||||
}
|
||||
}
|
||||
|
||||
return &WorkspaceConfig{Config: &wc, rawJSON: data}, nil
|
||||
}
|
||||
|
||||
// Clone returns a deep copy of the config via JSON round-trip.
|
||||
func (c *Config) Clone() *Config {
|
||||
data, err := json.Marshal(c)
|
||||
if err != nil {
|
||||
// Should never happen with a valid Config
|
||||
return DefaultConfig()
|
||||
}
|
||||
var clone Config
|
||||
if err := json.Unmarshal(data, &clone); err != nil {
|
||||
return DefaultConfig()
|
||||
}
|
||||
return &clone
|
||||
}
|
||||
|
||||
// MergeWorkspaceConfig overlays allowed fields from a workspace config onto this config.
|
||||
// Fields NOT honored (infrastructure-level): Gateway, Heartbeat, Devices, Providers.
|
||||
func (c *Config) MergeWorkspaceConfig(wc *WorkspaceConfig) {
|
||||
if wc == nil || wc.Config == nil {
|
||||
return
|
||||
}
|
||||
src := wc.Config
|
||||
|
||||
// model_list: replace if workspace has entries
|
||||
if len(src.ModelList) > 0 {
|
||||
c.ModelList = src.ModelList
|
||||
}
|
||||
|
||||
// agents.defaults: merge non-zero fields
|
||||
mergeAgentDefaults(&c.Agents.Defaults, &src.Agents.Defaults)
|
||||
|
||||
// agents.list: replace if workspace has entries
|
||||
if len(src.Agents.List) > 0 {
|
||||
c.Agents.List = src.Agents.List
|
||||
}
|
||||
|
||||
// tools & channels: use raw JSON overlay so that only keys actually present
|
||||
// in the workspace file are applied (avoids clobbering bool fields with false).
|
||||
mergeRawJSONField(wc.rawJSON, "tools", &c.Tools)
|
||||
mergeRawJSONField(wc.rawJSON, "channels", &c.Channels)
|
||||
|
||||
// bindings: replace if workspace has entries
|
||||
if len(src.Bindings) > 0 {
|
||||
c.Bindings = src.Bindings
|
||||
}
|
||||
|
||||
// session: merge non-zero fields (prevents cross-tenant identity leakage)
|
||||
mergeSessionConfig(&c.Session, &src.Session)
|
||||
}
|
||||
|
||||
// mergeAgentDefaults copies non-zero fields from src into dst.
|
||||
func mergeAgentDefaults(dst, src *AgentDefaults) {
|
||||
if src.Workspace != "" {
|
||||
dst.Workspace = src.Workspace
|
||||
}
|
||||
if src.RestrictToWorkspace {
|
||||
dst.RestrictToWorkspace = true
|
||||
}
|
||||
if src.AllowReadOutsideWorkspace {
|
||||
dst.AllowReadOutsideWorkspace = true
|
||||
}
|
||||
if src.Provider != "" {
|
||||
dst.Provider = src.Provider
|
||||
}
|
||||
if src.ModelName != "" {
|
||||
dst.ModelName = src.ModelName
|
||||
}
|
||||
if src.Model != "" {
|
||||
dst.Model = src.Model
|
||||
}
|
||||
if len(src.ModelFallbacks) > 0 {
|
||||
dst.ModelFallbacks = src.ModelFallbacks
|
||||
}
|
||||
if src.ImageModel != "" {
|
||||
dst.ImageModel = src.ImageModel
|
||||
}
|
||||
if len(src.ImageModelFallbacks) > 0 {
|
||||
dst.ImageModelFallbacks = src.ImageModelFallbacks
|
||||
}
|
||||
if src.MaxTokens > 0 {
|
||||
dst.MaxTokens = src.MaxTokens
|
||||
}
|
||||
if src.Temperature != nil {
|
||||
dst.Temperature = src.Temperature
|
||||
}
|
||||
if src.MaxToolIterations > 0 {
|
||||
dst.MaxToolIterations = src.MaxToolIterations
|
||||
}
|
||||
if src.SummarizeMessageThreshold > 0 {
|
||||
dst.SummarizeMessageThreshold = src.SummarizeMessageThreshold
|
||||
}
|
||||
if src.SummarizeTokenPercent > 0 {
|
||||
dst.SummarizeTokenPercent = src.SummarizeTokenPercent
|
||||
}
|
||||
if src.MaxMediaSize > 0 {
|
||||
dst.MaxMediaSize = src.MaxMediaSize
|
||||
}
|
||||
}
|
||||
|
||||
// mergeSessionConfig copies non-zero fields from src into dst.
|
||||
func mergeSessionConfig(dst, src *SessionConfig) {
|
||||
if src.DMScope != "" {
|
||||
dst.DMScope = src.DMScope
|
||||
}
|
||||
if len(src.IdentityLinks) > 0 {
|
||||
dst.IdentityLinks = src.IdentityLinks
|
||||
}
|
||||
}
|
||||
|
||||
// mergeRawJSONField extracts a top-level key from raw JSON and unmarshals it
|
||||
// onto dst. Because we use the original JSON bytes, only keys actually present
|
||||
// in the workspace file are applied — zero-valued fields (e.g. bool false) that
|
||||
// were never in the file are not included.
|
||||
func mergeRawJSONField[T any](rawJSON json.RawMessage, key string, dst *T) {
|
||||
var top map[string]json.RawMessage
|
||||
if err := json.Unmarshal(rawJSON, &top); err != nil {
|
||||
return
|
||||
}
|
||||
fieldData, ok := top[key]
|
||||
if !ok || string(fieldData) == "null" {
|
||||
return
|
||||
}
|
||||
json.Unmarshal(fieldData, dst) //nolint:errcheck
|
||||
}
|
||||
|
|
|
|||
|
|
@ -5,9 +5,11 @@ import (
|
|||
"encoding/json"
|
||||
"fmt"
|
||||
"io"
|
||||
"log/slog"
|
||||
"net/http"
|
||||
"net/url"
|
||||
"os"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/sipeed/picoclaw/pkg/utils"
|
||||
|
|
@ -31,12 +33,24 @@ type ClawHubRegistry struct {
|
|||
client *http.Client
|
||||
}
|
||||
|
||||
const defaultRegistryURL = "https://clawhub.ai"
|
||||
|
||||
// NewClawHubRegistry creates a new ClawHub registry client from config.
|
||||
func NewClawHubRegistry(cfg ClawHubConfig) *ClawHubRegistry {
|
||||
baseURL := cfg.BaseURL
|
||||
if baseURL == "" {
|
||||
baseURL = "https://clawhub.ai"
|
||||
baseURL = defaultRegistryURL
|
||||
}
|
||||
|
||||
// Validate base URL: require https unless targeting localhost.
|
||||
if parsedURL, err := url.Parse(baseURL); err != nil || parsedURL.Host == "" {
|
||||
slog.Warn("invalid clawhub base_url, falling back to default", "url", baseURL)
|
||||
baseURL = defaultRegistryURL
|
||||
} else if parsedURL.Scheme != "https" && !strings.HasPrefix(parsedURL.Host, "localhost") && !strings.HasPrefix(parsedURL.Host, "127.0.0.1") {
|
||||
slog.Warn("clawhub base_url must use https (unless localhost), falling back to default", "url", baseURL)
|
||||
baseURL = defaultRegistryURL
|
||||
}
|
||||
|
||||
searchPath := cfg.SearchPath
|
||||
if searchPath == "" {
|
||||
searchPath = "/api/v1/search"
|
||||
|
|
@ -234,6 +248,7 @@ func (c *ClawHubRegistry) DownloadAndInstall(
|
|||
result.IsMalwareBlocked = meta.IsMalwareBlocked
|
||||
result.IsSuspicious = meta.IsSuspicious
|
||||
result.Summary = meta.Summary
|
||||
result.MetadataAvailable = true
|
||||
}
|
||||
|
||||
// Step 2: Resolve version.
|
||||
|
|
|
|||
|
|
@ -7,12 +7,17 @@ import (
|
|||
"net/http"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"regexp"
|
||||
"time"
|
||||
|
||||
"github.com/sipeed/picoclaw/pkg/fileutil"
|
||||
"github.com/sipeed/picoclaw/pkg/utils"
|
||||
)
|
||||
|
||||
const maxSkillFileSize int64 = 5 << 20 // 5 MB — SKILL.md files should never approach this
|
||||
|
||||
var repoPattern = regexp.MustCompile(`^[a-zA-Z0-9_.-]+/[a-zA-Z0-9_.-]+$`)
|
||||
|
||||
type SkillInstaller struct {
|
||||
workspace string
|
||||
}
|
||||
|
|
@ -24,6 +29,10 @@ func NewSkillInstaller(workspace string) *SkillInstaller {
|
|||
}
|
||||
|
||||
func (si *SkillInstaller) InstallFromGitHub(ctx context.Context, repo string) error {
|
||||
if !repoPattern.MatchString(repo) {
|
||||
return fmt.Errorf("invalid repository format %q: must be 'owner/repo'", repo)
|
||||
}
|
||||
|
||||
skillDir := filepath.Join(si.workspace, "skills", filepath.Base(repo))
|
||||
|
||||
if _, err := os.Stat(skillDir); err == nil {
|
||||
|
|
@ -48,7 +57,7 @@ func (si *SkillInstaller) InstallFromGitHub(ctx context.Context, repo string) er
|
|||
return fmt.Errorf("failed to fetch skill: HTTP %d", resp.StatusCode)
|
||||
}
|
||||
|
||||
body, err := io.ReadAll(resp.Body)
|
||||
body, err := io.ReadAll(io.LimitReader(resp.Body, maxSkillFileSize))
|
||||
if err != nil {
|
||||
return fmt.Errorf("failed to read response: %w", err)
|
||||
}
|
||||
|
|
|
|||
|
|
@ -96,9 +96,37 @@ func NewSkillsLoader(workspace string, globalSkills string, builtinSkills string
|
|||
}
|
||||
}
|
||||
|
||||
// builtinSkillNames returns the set of validated skill names from the builtin directory.
|
||||
func (sl *SkillsLoader) builtinSkillNames() map[string]bool {
|
||||
names := make(map[string]bool)
|
||||
if sl.builtinSkills == "" {
|
||||
return names
|
||||
}
|
||||
dirs, err := os.ReadDir(sl.builtinSkills)
|
||||
if err != nil {
|
||||
return names
|
||||
}
|
||||
for _, d := range dirs {
|
||||
if !d.IsDir() {
|
||||
continue
|
||||
}
|
||||
skillFile := filepath.Join(sl.builtinSkills, d.Name(), "SKILL.md")
|
||||
if _, err := os.Stat(skillFile); err != nil {
|
||||
continue
|
||||
}
|
||||
name := d.Name()
|
||||
if metadata := sl.getSkillMetadata(skillFile); metadata != nil && metadata.Name != "" {
|
||||
name = metadata.Name
|
||||
}
|
||||
names[name] = true
|
||||
}
|
||||
return names
|
||||
}
|
||||
|
||||
func (sl *SkillsLoader) ListSkills() []SkillInfo {
|
||||
skills := make([]SkillInfo, 0)
|
||||
seen := make(map[string]bool)
|
||||
builtinNames := sl.builtinSkillNames()
|
||||
|
||||
addSkills := func(dir, source string) {
|
||||
if dir == "" {
|
||||
|
|
@ -133,6 +161,12 @@ func (sl *SkillsLoader) ListSkills() []SkillInfo {
|
|||
if seen[info.Name] {
|
||||
continue
|
||||
}
|
||||
// Block workspace/global skills from shadowing builtins.
|
||||
if source != "builtin" && builtinNames[info.Name] {
|
||||
slog.Warn("skill shadows a builtin and will be skipped",
|
||||
"name", info.Name, "source", source)
|
||||
continue
|
||||
}
|
||||
seen[info.Name] = true
|
||||
skills = append(skills, info)
|
||||
}
|
||||
|
|
@ -147,7 +181,15 @@ func (sl *SkillsLoader) ListSkills() []SkillInfo {
|
|||
}
|
||||
|
||||
func (sl *SkillsLoader) LoadSkill(name string) (string, bool) {
|
||||
// 1. load from workspace skills first (project-level)
|
||||
// If this is a builtin skill, always load from builtin to prevent shadowing.
|
||||
if sl.builtinSkills != "" {
|
||||
skillFile := filepath.Join(sl.builtinSkills, name, "SKILL.md")
|
||||
if content, err := os.ReadFile(skillFile); err == nil {
|
||||
return sl.stripFrontmatter(string(content)), true
|
||||
}
|
||||
}
|
||||
|
||||
// 1. load from workspace skills (project-level)
|
||||
if sl.workspaceSkills != "" {
|
||||
skillFile := filepath.Join(sl.workspaceSkills, name, "SKILL.md")
|
||||
if content, err := os.ReadFile(skillFile); err == nil {
|
||||
|
|
@ -163,14 +205,6 @@ func (sl *SkillsLoader) LoadSkill(name string) (string, bool) {
|
|||
}
|
||||
}
|
||||
|
||||
// 3. finally load from builtin skills
|
||||
if sl.builtinSkills != "" {
|
||||
skillFile := filepath.Join(sl.builtinSkills, name, "SKILL.md")
|
||||
if content, err := os.ReadFile(skillFile); err == nil {
|
||||
return sl.stripFrontmatter(string(content)), true
|
||||
}
|
||||
}
|
||||
|
||||
return "", false
|
||||
}
|
||||
|
||||
|
|
@ -191,14 +225,18 @@ func (sl *SkillsLoader) LoadSkillsForContext(skillNames []string) string {
|
|||
}
|
||||
|
||||
func (sl *SkillsLoader) BuildSkillsSummary() string {
|
||||
allSkills := sl.ListSkills()
|
||||
if len(allSkills) == 0 {
|
||||
return FormatSkillsSummary(sl.ListSkills())
|
||||
}
|
||||
|
||||
// FormatSkillsSummary renders a list of SkillInfo entries as an XML summary.
|
||||
func FormatSkillsSummary(skillList []SkillInfo) string {
|
||||
if len(skillList) == 0 {
|
||||
return ""
|
||||
}
|
||||
|
||||
var lines []string
|
||||
lines = append(lines, "<skills>")
|
||||
for _, s := range allSkills {
|
||||
for _, s := range skillList {
|
||||
escapedName := escapeXML(s.Name)
|
||||
escapedDesc := escapeXML(s.Description)
|
||||
escapedPath := escapeXML(s.Path)
|
||||
|
|
|
|||
|
|
@ -163,7 +163,7 @@ func TestListSkillsWorkspaceOverridesGlobal(t *testing.T) {
|
|||
assert.Equal(t, "workspace version", skills[0].Description)
|
||||
}
|
||||
|
||||
func TestListSkillsGlobalOverridesBuiltin(t *testing.T) {
|
||||
func TestListSkillsBuiltinCannotBeShadowed(t *testing.T) {
|
||||
tmp := t.TempDir()
|
||||
ws := filepath.Join(tmp, "workspace")
|
||||
global := filepath.Join(tmp, "global")
|
||||
|
|
@ -176,8 +176,8 @@ func TestListSkillsGlobalOverridesBuiltin(t *testing.T) {
|
|||
skills := sl.ListSkills()
|
||||
|
||||
assert.Len(t, skills, 1)
|
||||
assert.Equal(t, "global", skills[0].Source)
|
||||
assert.Equal(t, "global version", skills[0].Description)
|
||||
assert.Equal(t, "builtin", skills[0].Source)
|
||||
assert.Equal(t, "builtin version", skills[0].Description)
|
||||
}
|
||||
|
||||
func TestListSkillsMetadataNameDedup(t *testing.T) {
|
||||
|
|
|
|||
|
|
@ -36,10 +36,11 @@ type SkillMeta struct {
|
|||
// InstallResult is returned by DownloadAndInstall to carry metadata
|
||||
// back to the caller for moderation and user messaging.
|
||||
type InstallResult struct {
|
||||
Version string
|
||||
IsMalwareBlocked bool
|
||||
IsSuspicious bool
|
||||
Summary string
|
||||
Version string
|
||||
IsMalwareBlocked bool
|
||||
IsSuspicious bool
|
||||
Summary string
|
||||
MetadataAvailable bool
|
||||
}
|
||||
|
||||
// SkillRegistry is the interface that all skill registries must implement.
|
||||
|
|
|
|||
|
|
@ -315,7 +315,7 @@ func (t *CronTool) ExecuteJob(ctx context.Context, job *cron.CronJob) string {
|
|||
}
|
||||
|
||||
// For deliver=false, process through agent (for complex tasks)
|
||||
sessionKey := fmt.Sprintf("cron-%s", job.ID)
|
||||
sessionKey := fmt.Sprintf("agent:main:cron:%s", job.ID)
|
||||
|
||||
// Call agent with job's message
|
||||
response, err := t.executor.ProcessDirectWithChannel(
|
||||
|
|
|
|||
|
|
@ -2,17 +2,20 @@ package tools
|
|||
|
||||
import (
|
||||
"context"
|
||||
"crypto/rand"
|
||||
"encoding/hex"
|
||||
"fmt"
|
||||
"io/fs"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"regexp"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/sipeed/picoclaw/pkg/fileutil"
|
||||
)
|
||||
|
||||
const maxWriteSize = 20 * 1024 * 1024 // 20 MB — limit for file writes via the write tool
|
||||
|
||||
// validatePath ensures the given path is within the workspace if restrict is true.
|
||||
func validatePath(path, workspace string, restrict bool) (string, error) {
|
||||
if workspace == "" {
|
||||
|
|
@ -178,6 +181,10 @@ func (t *WriteFileTool) Execute(ctx context.Context, args map[string]any) *ToolR
|
|||
return ErrorResult("content is required")
|
||||
}
|
||||
|
||||
if len(content) > maxWriteSize {
|
||||
return ErrorResult(fmt.Sprintf("content too large: %d bytes exceeds %d byte limit", len(content), maxWriteSize))
|
||||
}
|
||||
|
||||
if err := t.fs.WriteFile(path, []byte(content)); err != nil {
|
||||
return ErrorResult(err.Error())
|
||||
}
|
||||
|
|
@ -334,7 +341,11 @@ func (r *sandboxFs) WriteFile(path string, data []byte) error {
|
|||
|
||||
// Use atomic write pattern with explicit sync for flash storage reliability.
|
||||
// Using 0o600 (owner read/write only) for secure default permissions.
|
||||
tmpRelPath := fmt.Sprintf(".tmp-%d-%d", os.Getpid(), time.Now().UnixNano())
|
||||
randBytes := make([]byte, 8)
|
||||
if _, err := rand.Read(randBytes); err != nil {
|
||||
return fmt.Errorf("failed to generate random bytes for temp file: %w", err)
|
||||
}
|
||||
tmpRelPath := fmt.Sprintf(".tmp-%s", hex.EncodeToString(randBytes))
|
||||
|
||||
tmpFile, err := root.OpenFile(tmpRelPath, os.O_WRONLY|os.O_CREATE|os.O_EXCL, 0o600)
|
||||
if err != nil {
|
||||
|
|
|
|||
|
|
@ -23,6 +23,7 @@ type ExecTool struct {
|
|||
allowPatterns []*regexp.Regexp
|
||||
customAllowPatterns []*regexp.Regexp
|
||||
restrictToWorkspace bool
|
||||
filterEnv bool
|
||||
}
|
||||
|
||||
var (
|
||||
|
|
@ -136,6 +137,11 @@ func NewExecToolWithConfig(workingDir string, restrict bool, config *config.Conf
|
|||
timeout = time.Duration(config.Tools.Exec.TimeoutSeconds) * time.Second
|
||||
}
|
||||
|
||||
filterEnv := false
|
||||
if config != nil {
|
||||
filterEnv = config.Tools.Exec.FilterEnv
|
||||
}
|
||||
|
||||
return &ExecTool{
|
||||
workingDir: workingDir,
|
||||
timeout: timeout,
|
||||
|
|
@ -143,6 +149,7 @@ func NewExecToolWithConfig(workingDir string, restrict bool, config *config.Conf
|
|||
allowPatterns: nil,
|
||||
customAllowPatterns: customAllowPatterns,
|
||||
restrictToWorkspace: restrict,
|
||||
filterEnv: filterEnv,
|
||||
}, nil
|
||||
}
|
||||
|
||||
|
|
@ -221,6 +228,10 @@ func (t *ExecTool) Execute(ctx context.Context, args map[string]any) *ToolResult
|
|||
cmd.Dir = cwd
|
||||
}
|
||||
|
||||
if t.filterEnv {
|
||||
cmd.Env = filterEnvironment(os.Environ())
|
||||
}
|
||||
|
||||
prepareCommandForTermination(cmd)
|
||||
|
||||
var stdout, stderr bytes.Buffer
|
||||
|
|
@ -381,3 +392,38 @@ func (t *ExecTool) SetAllowPatterns(patterns []string) error {
|
|||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// safeEnvVars is the allowlist of environment variable names preserved when
|
||||
// filterEnv is enabled. Variables with a PICOCLAW_ prefix are always kept.
|
||||
var safeEnvVars = map[string]bool{
|
||||
"PATH": true,
|
||||
"HOME": true,
|
||||
"USER": true,
|
||||
"LANG": true,
|
||||
"TERM": true,
|
||||
"SHELL": true,
|
||||
"TMPDIR": true,
|
||||
"TMP": true,
|
||||
"TEMP": true,
|
||||
"SystemRoot": true,
|
||||
"COMSPEC": true,
|
||||
"USERPROFILE": true,
|
||||
"APPDATA": true,
|
||||
"LOCALAPPDATA": true,
|
||||
}
|
||||
|
||||
// filterEnvironment returns a filtered copy of environ keeping only safe vars.
|
||||
func filterEnvironment(environ []string) []string {
|
||||
filtered := make([]string, 0, len(safeEnvVars)+4)
|
||||
for _, entry := range environ {
|
||||
eqIdx := strings.IndexByte(entry, '=')
|
||||
if eqIdx < 0 {
|
||||
continue
|
||||
}
|
||||
name := entry[:eqIdx]
|
||||
if safeEnvVars[name] || strings.HasPrefix(name, "PICOCLAW_") {
|
||||
filtered = append(filtered, entry)
|
||||
}
|
||||
}
|
||||
return filtered
|
||||
}
|
||||
|
|
|
|||
|
|
@ -159,10 +159,13 @@ func (t *InstallSkillTool) Execute(ctx context.Context, args map[string]any) *To
|
|||
_ = err
|
||||
}
|
||||
|
||||
// Build result with moderation warning if suspicious.
|
||||
// Build result with moderation warnings.
|
||||
var output string
|
||||
if !result.MetadataAvailable {
|
||||
output += fmt.Sprintf("Warning: safety metadata was not available for skill %q. Exercise caution.\n\n", slug)
|
||||
}
|
||||
if result.IsSuspicious {
|
||||
output = fmt.Sprintf("⚠️ Warning: skill %q is flagged as suspicious (may contain risky patterns).\n\n", slug)
|
||||
output += fmt.Sprintf("⚠️ Warning: skill %q is flagged as suspicious (may contain risky patterns).\n\n", slug)
|
||||
}
|
||||
output += fmt.Sprintf("Successfully installed skill %q v%s from %s registry.\nLocation: %s\n",
|
||||
slug, result.Version, registry.Name(), targetDir)
|
||||
|
|
|
|||
|
|
@ -22,8 +22,9 @@ const (
|
|||
perplexityTimeout = 30 * time.Second // Perplexity (LLM-based, slower)
|
||||
fetchTimeout = 60 * time.Second // WebFetchTool
|
||||
|
||||
defaultMaxChars = 50000
|
||||
maxRedirects = 5
|
||||
defaultMaxChars = 50000
|
||||
maxRedirects = 5
|
||||
searchMaxResponseSize int64 = 2 << 20 // 2 MB — limit for search provider responses
|
||||
)
|
||||
|
||||
// Pre-compiled regexes for HTML text extraction
|
||||
|
|
@ -104,7 +105,7 @@ func (p *BraveSearchProvider) Search(ctx context.Context, query string, count in
|
|||
}
|
||||
defer resp.Body.Close()
|
||||
|
||||
body, err := io.ReadAll(resp.Body)
|
||||
body, err := io.ReadAll(io.LimitReader(resp.Body, searchMaxResponseSize))
|
||||
if err != nil {
|
||||
return "", fmt.Errorf("failed to read response: %w", err)
|
||||
}
|
||||
|
|
@ -191,7 +192,7 @@ func (p *TavilySearchProvider) Search(ctx context.Context, query string, count i
|
|||
}
|
||||
defer resp.Body.Close()
|
||||
|
||||
body, err := io.ReadAll(resp.Body)
|
||||
body, err := io.ReadAll(io.LimitReader(resp.Body, searchMaxResponseSize))
|
||||
if err != nil {
|
||||
return "", fmt.Errorf("failed to read response: %w", err)
|
||||
}
|
||||
|
|
@ -253,7 +254,7 @@ func (p *DuckDuckGoSearchProvider) Search(ctx context.Context, query string, cou
|
|||
}
|
||||
defer resp.Body.Close()
|
||||
|
||||
body, err := io.ReadAll(resp.Body)
|
||||
body, err := io.ReadAll(io.LimitReader(resp.Body, searchMaxResponseSize))
|
||||
if err != nil {
|
||||
return "", fmt.Errorf("failed to read response: %w", err)
|
||||
}
|
||||
|
|
@ -367,7 +368,7 @@ func (p *PerplexitySearchProvider) Search(ctx context.Context, query string, cou
|
|||
}
|
||||
defer resp.Body.Close()
|
||||
|
||||
body, err := io.ReadAll(resp.Body)
|
||||
body, err := io.ReadAll(io.LimitReader(resp.Body, searchMaxResponseSize))
|
||||
if err != nil {
|
||||
return "", fmt.Errorf("failed to read response: %w", err)
|
||||
}
|
||||
|
|
|
|||
Loading…
Add table
Reference in a new issue