Max 20fc9c24df feat(image): introduce image reading and analysis tools

- Added `image_read` tool for reading and analyzing images using vision-capable models, with a detailed input schema for image path, prompt, max size, and provider.
- Implemented `ReadImage` function to handle image processing and integration with vision models, returning descriptive text.
- Created comprehensive tests for image resizing, data URI handling, and various input scenarios to ensure robust functionality.
- Updated documentation to include usage examples and parameter descriptions for the new image tools.

2026-05-06 10:58:52 +08:00

1.7 KiB

Raw Blame History

name	description
yao-vision	Image understanding expert. ALWAYS invoke this skill when you need to read, analyze, or describe an image that you cannot process directly. Use for screenshots, photos, charts, diagrams, or any visual content.

Vision Tools

Use when you encounter images you cannot read natively (e.g., as a text-only model).

image_read

Send an image to a vision-capable model and get a text description.

Local file (most common):

tai tool image_read '{"image_path": "/path/to/image.png", "prompt": "Describe this image"}'

URL:

tai tool image_read '{"image_path": "https://example.com/photo.jpg", "prompt": "What is shown?"}'

Cross-workspace file:

tai tool image_read '{"image_path": "workspace://ws-id/path/to/image.png", "prompt": "Analyze"}'

Attachment file:

tai tool image_read '{"image_path": "attach://__yao.attachment/file-id-123", "prompt": "Describe"}'

Yao data file:

tai tool image_read '{"image_path": "yao://uploads/photo.jpg", "prompt": "Describe"}'

Parameter	Type	Required	Description
image_path	string	yes	File path, URL, workspace://, attach://, or yao:// URI
prompt	string	no	Analysis instruction (default: describe in detail)
max_size	integer	no	Max dimension in pixels for longest edge (default: 1080)

Images are automatically resized (preserving aspect ratio) before sending to the vision model. Supported formats: PNG, JPEG, GIF, WebP.

1.7 KiB Raw Blame History