picoclaw/pkg/agent
Guoguo cb1e1a3595
fix(feishu): fix image download with API fallback and post image support (#2708)
* fix(feishu): fix image download with API fallback and post image support

- Add Image.Get API fallback when MessageResource.Get fails (different
  permission scope: im:resource vs im:message:readonly)
- Extract and download images from post (rich text) messages
- Extract images from interactive card messages
- Deduplicate post image keys across locales
- Add comprehensive tests for new helpers

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat(media): add image path tags alongside base64 for LLM file access

Images are still base64-encoded into msg.Media for multimodal LLMs,
but now also get [image:path] tags injected into message content so
the LLM knows the local file path for save/forward operations.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* refactor(media): only auto-inject images for tool results, not user messages

Channel-received images (role=user) now get path tags only, letting
the LLM decide whether to view via load_image or just operate on
the file. Tool result images (role=tool, e.g. load_image) are
base64-encoded into a synthetic user message appended after the tool
message, since many LLM APIs don't support image_url in tool messages.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(media): preserve tool-message ordering for multi-tool-call scenarios

Move synthetic user message (carrying base64 tool images) to after the
entire contiguous tool-message block instead of immediately after each
tool message. This preserves the assistant→tool→tool ordering required
by OpenAI-compatible APIs.

Also fix load_image to use generic [image: photo] placeholder so
injectPathTags can properly replace it with the actual path.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(test): update load_image test for [image: photo] placeholder

The test was checking ForLLM for the media:// ref, but load_image now
emits the generic [image: photo] placeholder instead.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(media): match all channel image placeholders in injectPathTags

Different channels emit different placeholder formats — Telegram/Feishu
use [image: photo], WeCom/WeChat/Line use bare [image], QQ/Discord use
[image: <filename>]. The previous string-match code only handled
[image: photo], so for the other channels the path tag was appended as
a duplicate, producing content like "[image] [image:/path]".

Switch to per-type regex that matches all generic placeholder shapes
while leaving path tags ([image:/path]) untouched. Also fixes the same
issue for [audio], [video], [file] tags. Added test coverage for the
various placeholder shapes.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(media): skip path tag append for JSON content (Feishu cards/posts)

When content is structured JSON (interactive cards, post messages),
injectPathTags now skips the fallback append — only placeholder
replacement is attempted. This prevents corrupting JSON payloads
like {"schema":"2.0",...} with appended [image:/path] tags.

Adds looksLikeJSON() helper and three test cases covering JSON
objects, arrays, and an end-to-end resolveMediaRefs scenario with
Feishu card content.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(media): prepend path tags for JSON content, narrow looksLikeJSON

Two fixes from code review:

1. looksLikeJSON now only checks for '{' prefix (not '['), avoiding
   false positives on regular text like "[update] see attached".

2. For JSON content (Feishu cards/posts), path tags are prepended
   before the JSON instead of being silently dropped. This ensures
   the LLM can discover attached images via the path tag while the
   JSON payload stays valid for downstream parsing.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-30 11:08:00 +08:00
..
adapters refactor(agent): Agent Looper refactor phase2, restructure pipeline and rename loop files to agent (#2585) 2026-04-21 10:55:50 +08:00
interfaces refactor(agent): Agent Looper refactor phase2, restructure pipeline and rename loop files to agent (#2585) 2026-04-21 10:55:50 +08:00
agent.go feat(pico): add support for tool_calls in chat messages 2026-04-25 23:43:10 +08:00
agent_command.go feat(commands): add MCP slash commands and tool details 2026-04-21 11:01:04 +02:00
agent_event.go refactor(agent): Agent Looper refactor phase2, restructure pipeline and rename loop files to agent (#2585) 2026-04-21 10:55:50 +08:00
agent_init.go feat(config): wire serial tool into runtime and dashboard 2026-04-28 12:56:27 +08:00
agent_inject.go refactor(agent): Agent Looper refactor phase2, restructure pipeline and rename loop files to agent (#2585) 2026-04-21 10:55:50 +08:00
agent_mcp.go feat(agent): migrate tool prompts to capability slots 2026-04-24 19:36:46 +08:00
agent_mcp_test.go fix(mcp): surface MCP init failures to command handlers 2026-04-21 11:01:04 +02:00
agent_media.go fix(feishu): fix image download with API fallback and post image support (#2708) 2026-04-30 11:08:00 +08:00
agent_message.go refactor(agent): Agent Looper refactor phase2, restructure pipeline and rename loop files to agent (#2585) 2026-04-21 10:55:50 +08:00
agent_outbound.go fix(tool-feedback): dedupe duplicate content and keep full explanations 2026-04-26 00:40:55 +08:00
agent_steering.go refactor(agent): Agent Looper refactor phase2, restructure pipeline and rename loop files to agent (#2585) 2026-04-21 10:55:50 +08:00
agent_test.go fix(feishu): fix image download with API fallback and post image support (#2708) 2026-04-30 11:08:00 +08:00
agent_transcribe.go refactor(agent): Agent Looper refactor phase2, restructure pipeline and rename loop files to agent (#2585) 2026-04-21 10:55:50 +08:00
agent_utils.go fix(tool-feedback): dedupe duplicate content and keep full explanations 2026-04-26 00:40:55 +08:00
context.go feat(agent): migrate tool prompts to capability slots 2026-04-24 19:36:46 +08:00
context_budget.go feat(seahorse): implement short-term memory engine (LCM) (#2285) 2026-04-05 09:05:16 +08:00
context_budget_test.go feat(seahorse): implement short-term memory engine (LCM) (#2285) 2026-04-05 09:05:16 +08:00
context_cache_test.go feat(web): support image messages in pico chat (#2299) 2026-04-03 14:15:20 +08:00
context_legacy.go merge: integrate main seahorse context changes 2026-04-13 23:02:38 +08:00
context_manager.go feat(agent): /clear now clears seahorse DB in addition to JSONL 2026-04-13 11:04:45 +08:00
context_manager_test.go feat(agent): /clear now clears seahorse DB in addition to JSONL 2026-04-13 11:04:45 +08:00
context_seahorse.go feat(agent): /clear now clears seahorse DB in addition to JSONL 2026-04-13 11:04:45 +08:00
context_seahorse_test.go feat(seahorse): implement short-term memory engine (LCM) (#2285) 2026-04-05 09:05:16 +08:00
context_seahorse_unsupported.go fix(agent): disable seahorse context manager on freebsd/arm (#2417) 2026-04-08 10:57:22 +08:00
context_test.go fix(agent): preserve reused tool call IDs across turns (#2528) 2026-04-15 20:18:09 +08:00
context_usage.go feat(agent): add context usage ring indicator and /context command (#2537) 2026-04-21 16:30:02 +08:00
definition.go merge: resolve conflicts between refactor/agent and main 2026-03-22 19:21:58 +08:00
definition_test.go merge: resolve conflicts between refactor/agent and main 2026-03-22 19:21:58 +08:00
dispatch_request.go fix(session): address review regressions 2026-04-13 22:51:44 +08:00
dispatch_request_test.go fix(session): address review regressions 2026-04-13 22:51:44 +08:00
eventbus.go merge: resolve conflicts between refactor/agent and main 2026-03-22 19:21:58 +08:00
eventbus_test.go Merge branch 'main' into refactor-inbound-context-routing-session 2026-04-07 21:41:02 +08:00
events.go Merge branch 'main' into refactor-inbound-context-routing-session 2026-04-07 21:41:02 +08:00
hook_mount.go merge: resolve conflicts between refactor/agent and main 2026-03-22 19:21:58 +08:00
hook_mount_test.go Merge branch 'main' into version 2026-03-23 10:54:08 +08:00
hook_process.go Feat/support isolation (#2423) 2026-04-08 18:15:42 +08:00
hook_process_test.go Feat/support isolation (#2423) 2026-04-08 18:15:42 +08:00
hooks.go Merge pull request #2666 from afjcjsbx/fix/mcp-nil-arguments 2026-04-25 11:38:48 +08:00
hooks_test.go Merge pull request #2666 from afjcjsbx/fix/mcp-nil-arguments 2026-04-25 11:38:48 +08:00
instance.go refactor: support explicit provider field in model list entries (#2609) 2026-04-22 11:28:47 +08:00
instance_test.go refactor: support explicit provider field in model list entries (#2609) 2026-04-22 11:28:47 +08:00
llm_media.go refactor: make agent loop support parallel and update docs 2026-04-16 14:43:15 +08:00
memory.go refactor(modernize): apply safe modernize fixes 2026-02-27 16:35:07 +08:00
mock_provider_test.go feat(fmt): Fix formatting 2026-02-19 22:05:15 +02:00
model_resolution.go refactor: support explicit provider field in model list entries (#2609) 2026-04-22 11:28:47 +08:00
pipeline.go refactor(agent): Agent Looper refactor phase2, restructure pipeline and rename loop files to agent (#2585) 2026-04-21 10:55:50 +08:00
pipeline_execute.go fix(tool-feedback): dedupe duplicate content and keep full explanations 2026-04-26 00:40:55 +08:00
pipeline_finalize.go fix(reasoning): persist canonical history for DeepSeek and web chat 2026-04-24 21:45:41 +08:00
pipeline_llm.go fix(tool-feedback): dedupe duplicate content and keep full explanations 2026-04-26 00:40:55 +08:00
pipeline_setup.go feat(agent): add structured prompt layering 2026-04-24 18:14:28 +08:00
prompt.go feat(agent): migrate tool prompts to capability slots 2026-04-24 19:36:46 +08:00
prompt_contributors.go feat(agent): migrate tool prompts to capability slots 2026-04-24 19:36:46 +08:00
prompt_test.go fix(agent): preserve prompt hook and cache semantics 2026-04-25 01:25:17 +08:00
prompt_turn.go fix(agent): preserve prompt hook and cache semantics 2026-04-25 01:25:17 +08:00
registry.go refactor(runtime): drop non-session legacy context compatibility 2026-04-01 20:56:48 +08:00
registry_test.go refactor Config to add Version and migratable 2026-03-12 13:52:55 +08:00
steering.go feat(agent): add structured prompt layering 2026-04-24 18:14:28 +08:00
steering_test.go fix(feishu): fix image download with API fallback and post image support (#2708) 2026-04-30 11:08:00 +08:00
subturn.go fix(reasoning): persist canonical history for DeepSeek and web chat 2026-04-24 21:45:41 +08:00
subturn_test.go Feat(channels): unify animated tool feedback across chat channels and Pico (#2622) 2026-04-23 10:35:50 +08:00
thinking.go feat: add extended thinking support for Anthropic models (#1076) 2026-03-05 09:51:18 +08:00
thinking_test.go feat: add extended thinking support for Anthropic models (#1076) 2026-03-05 09:51:18 +08:00
turn_context.go refactor(session): replace dm scope with dimensions policy 2026-04-01 17:19:50 +08:00
turn_coord.go feat(agent): add structured prompt layering 2026-04-24 18:14:28 +08:00
turn_coord_test.go fix(tools): improve web search provider fallback (#2629) 2026-04-23 15:39:16 +08:00
turn_state.go feat(agent): add structured prompt layering 2026-04-24 18:14:28 +08:00