- Added support for processing image content in the agent, providing placeholder text when image parsing fails.
- Implemented a fallback mechanism to utilize vision tools for image analysis, improving the handling of image content.
- Updated tests to validate the new image handling behavior, ensuring correct responses when vision support is unavailable.
- Enhanced system prompts to guide users on using the new `image_read` tool for image analysis, improving user experience.
- Added support for tool loop processing when tool call responses are present and sandbox mode is disabled, improving the assistant's ability to handle complex tool interactions.
- Implemented fallback delegation to a loop fallback mechanism in case of tool loop execution failure, enhancing robustness in error scenarios.
- Updated the assistant message structure to include reasoning content, providing better context for generated responses.
- Enhanced error logging in tool call execution to include detailed content, improving diagnostics for tool call failures.
- Updated system configuration to include a new loop fallback agent, expanding the assistant's capabilities.
- Integrated logging for connector resolution failures, providing clearer diagnostics during fallback scenarios.
- Simplified connector ID extraction in various components, ensuring accurate handling of model identifiers.
- Updated model configuration to support new parameters and capabilities, improving overall provider management.
- Enhanced OpenAPI settings to reflect changes in model options and connector behavior, ensuring better alignment with upstream API requirements.
- Updated the system configuration to include new role-level defaults for Light, Vision, and Audio connectors.
- Refactored the resolveSystemConnector function to prioritize per-agent overrides, improving connector resolution logic.
- Enhanced LLMConnector integration across various components to streamline settings retrieval and capabilities management.
- Improved error handling and logging for connector-related operations, ensuring better diagnostics and user feedback.
- Refactor GPT-5 test cases to improve clarity and maintainability.
- Comment out tests for temperature handling in GPT-5, indicating they are temporarily disabled.
- Remove the obsolete hostexec test file to clean up the codebase.
- Enhance the sandbox manager to support host execution capabilities and improve lifecycle management.
Made-with: Cursor
- Remove the `LoadModelCapabilities` test and associated model capabilities initialization from the agent, streamlining the loading process.
- Update the LLM provider implementations to utilize a unified `Capabilities` structure, replacing references to `openai.Capabilities` with `llm.Capabilities`.
- Enhance capability retrieval methods to simplify the extraction of connector capabilities, ensuring compatibility across different LLM providers.
- Clean up unused functions and variables related to model capabilities, improving code maintainability.
- Update `buildAPIURL` function in OpenAI provider to delegate URL construction to `connector.BuildAPIURL`, ensuring consistent URL formatting across the agent LLM and sandbox proxy paths.
- Modify backend URL construction in Claude's `BuildProxyConfig` to utilize the shared `connector.BuildAPIURL` helper, applying the necessary `/v1` prefix for compatibility.
This change enhances code maintainability and consistency in API URL handling across different components.
- Eliminated debug print statements for the "thinking" parameter and the full request body in the OpenAI provider, streamlining the request construction process.
- Maintained the functionality of including the "thinking" parameter for models that support reasoning/thinking mode, ensuring improved performance without unnecessary logging.
- Added support for a new "thinking" parameter in the request body for models that support reasoning/thinking mode, improving functionality.
- Updated the request body construction to include debugging output for the full request body, aiding in troubleshooting and development.
- Ensured that the model and other settings are retrieved from the connector, maintaining consistency in configuration handling.
- Added a User-Agent header to HTTP requests in the OpenAI provider to improve request identification.
- Updated TODO.md to reflect the completion status of various agent configurations and test scenarios, marking several tasks as done.
- Ensured all relevant agents and expert configurations are now marked as complete, enhancing clarity on project progress.
- Marked the Cache Implementation as complete in TODO.md, confirming all tasks are finished with comprehensive integration tests.
- Updated cache.go to provide a thread-safe in-memory cache for Robot instances, improving performance and reliability.
- Enhanced the Load method to include pagination and configurable model name, ensuring efficient data handling.
- Added detailed comments and structured code for better readability and maintainability across the cache implementation.
- Improved validation and error handling in various utility functions to ensure robustness in data processing.
- Added support for a new `Metadata` field in the `Options` struct to allow passing custom data to hooks, enhancing flexibility in context management.
- Updated the `ToMap` and `OptionsFromMap` methods to include serialization and deserialization of the `Metadata` field.
- Enhanced the test case structure to include an `Options` field, allowing for per-test-case configuration, including metadata and skip options.
- Updated documentation to reflect the new `options` and `metadata` fields, providing clear examples for users on how to utilize these features in test cases.
- Updated the TestGPT5Vision function to support various content types in responses, including strings and slices of ContentPart.
- Implemented logic to concatenate text from multimodal responses, improving the robustness of image description handling.
- Added logging for cases where content is nil or of unexpected types, enhancing test feedback and debugging capabilities.
- Added a new method to set store settings during assistant initialization, allowing for configuration of storage parameters such as MaxSize and TTL.
- Updated context creation methods to streamline the setup process, ensuring that essential fields are populated consistently across various test contexts.
- Revised tests to validate the new initialization behavior and context management, ensuring proper handling of assistant settings and context properties.
- Updated the Stream, BuildContent, and LLM execution methods to accept an Options parameter, allowing for more flexible context handling.
- Removed debug print statements to clean up the code and improve readability.
- Enhanced locale handling in the loadMap function to automatically inject assistant name and description into all locales, ensuring better localization support.
- Introduced output skipping functionality in context options to manage internal A2A calls more effectively.
- Improved output writer resolution logic to prioritize context settings, enhancing output management during agent calls.
- Updated the Assistant methods to accept an Options parameter, enhancing flexibility in context management.
- Removed the Connector field from the context and related structures, transitioning to a more streamlined options-based approach.
- Adjusted various tests to accommodate the new options handling, ensuring comprehensive coverage of the updated functionality.
- Enhanced the Create and Next hooks to return options alongside responses, improving the overall usability of the API.
- Cleaned up deprecated fields and improved context initialization for better maintainability.
- Updated the model capabilities throughout the agent to utilize the new gouOpenAI.Capabilities struct instead of the previous ModelCapabilities.
- Adjusted related methods and types to ensure compatibility with the new capabilities structure, enhancing clarity and maintainability.
- Improved context handling and message processing by directly integrating OpenAI capabilities, streamlining the overall architecture.
- Updated handleToolCall method to parse tool call delta data from OpenAI, improving the handling of single and multiple tool calls.
- Implemented dynamic delta actions ("append" for arguments, "merge" for static fields) to optimize message sending.
- Enhanced message structure to include appropriate DeltaAction and DeltaPath for better frontend integration.
- Improved error handling during JSON parsing to ensure robustness in tool call processing.
- Added detailed debug logging throughout the Stream method to trace execution flow and tool call results, improving visibility during runtime.
- Implemented error tracing with the new traceAgentFail method to capture failures in agent nodes, enhancing error handling.
- Updated MCP tools debugging to provide insights into tool application and validation processes, aiding in troubleshooting.
- Refactored output handling in traceAgentOutput for better clarity and consistency in response management.
- Replaced group-related event types with message-specific types, enhancing clarity in event management.
- Updated stream handling functions to utilize new message start and end events, improving the granularity of message tracking.
- Refactored context management to integrate a context-scoped ID generator for unique message identifiers.
- Removed deprecated group handling methods and streamlined message processing logic.
- Updated tests and documentation to reflect changes in message handling and ensure proper functionality.
- Updated message structures to replace 'StreamStartData' and 'StreamEndData' with 'EventStreamStartData' and 'EventStreamEndData' for better clarity and consistency.
- Introduced 'EventMessageStartData' and 'EventMessageEndData' to represent individual message lifecycle events, enhancing the granularity of message tracking.
- Refactored the 'streamState' and 'groupTracker' to utilize the new message structures, improving the organization and handling of streaming events.
- Enhanced the context management by integrating an ID generator for unique message identifiers, facilitating better tracking of message sequences.
- Updated documentation and tests to reflect the new message structures and ensure proper functionality across the system.
- Updated the interrupt controller to accept context IDs during initialization, improving traceability of interrupt signals.
- Enhanced logging in the Assistant's Stream method to provide detailed trace information for stream lifecycle events.
- Refactored context release logic to improve cleanup processes and added logging for context management.
- Adjusted test cases to align with the new interrupt handling and logging structure, ensuring robust functionality and traceability.
- Integrated i18n translations for trace logging messages in the Assistant's Stream method, improving localization for user-facing messages.
- Updated logging in the OpenAI provider and CUI writer to utilize i18n for error messages, enhancing clarity and consistency across different locales.
- Added new embedded template handling in the i18n package, allowing for more flexible message formatting with variable substitutions.
- Improved test coverage for i18n functionalities, ensuring robust handling of various translation scenarios and edge cases.
- Adjusted indentation and formatting in the OpenAI provider's logging statements to enhance readability and maintainability.
- Cleaned up test files by removing unnecessary blank lines, improving overall code clarity and consistency.
- Ensured that logging for request and response details is properly structured for better debugging capabilities.
- Introduced new methods in the trace manager for retrieving events, trace info, nodes, logs, and spaces from storage, improving data access and management.
- Updated the driver implementations to support loading and unarchiving traces, ensuring robust handling of trace data.
- Enhanced error handling and logging in the OpenAI provider to facilitate better debugging and traceability of requests and responses.
- Added a new TraceSpaceData struct to encapsulate space metadata along with key-value data for improved API responses.
- Integrated trace management into the Assistant's Stream method, allowing for detailed logging of processing steps and errors.
- Improved error handling in the OpenAI stream and post methods, adding trace logs for retries and failures.
- Enhanced CUI and OpenAI writers to log message adaptation and sending errors, improving debugging capabilities.
- Updated trace manager to use milliseconds for timestamps, ensuring consistency across the trace system.
- Refactored log methods in the trace node to streamline logging and broadcasting of events.
- Added an interrupt controller to manage user interrupts during streaming operations.
- Introduced methods for registering and unregistering contexts in a global registry to facilitate interrupt handling.
- Enhanced the Assistant's Stream method to check for force interrupts and handle them appropriately.
- Updated context creation to include unique IDs for better identification during interrupts.
- Implemented logic for graceful and force interrupts, allowing for flexible response to user signals.
- Added a new field to track the current message type (text, thinking, tool_call) in the streamState struct.
- Updated message handling methods to set the current message type appropriately.
- Modified the group end handling to use the tracked message type when sending messages, ensuring accurate type representation.
- Enhanced unit tests to verify the correct handling of group end events and their associated types.
- Integrated Claude API keys into GitHub workflows for both unit and PR tests.
- Introduced a new VisionFormat type and constants to manage image input formats.
- Updated ModelCapabilities to support vision input, allowing for flexible handling of image formats.
- Enhanced VisionAdapter to preprocess messages and convert image URLs to base64 format for Claude compatibility.
- Improved OpenAI provider to utilize vision support in message preprocessing, ensuring better integration with vision capabilities.
- Renamed and refactored functions and variables to transition from connector settings to model capabilities, enhancing clarity and consistency.
- Updated the loading mechanism to read model capabilities from `models.yml` instead of `connectors.yml`.
- Adjusted the assistant's global settings to utilize model capabilities, ensuring proper integration with the new configuration structure.
- Enhanced the reasoning adapter to support temperature adjustment based on model capabilities, improving flexibility in handling reasoning parameters.
- Added DeepSeek API keys and model configurations to the GitHub workflows for both unit and PR tests.
- Introduced a new `ReasoningEffort` parameter in the `CompletionOptions` struct to manage reasoning levels for models like o1 and GPT-5.
- Updated the `ReasoningAdapter` to handle the new `ReasoningEffort` parameter, ensuring it is stripped if not supported by the model.
- Enhanced the OpenAI provider to preprocess options through adapters, improving the handling of reasoning content and ensuring compatibility with DeepSeek R1 reasoning format.
- Removed legacy and audio providers, consolidating functionality into a new architecture that separates API format handling from capability management.
- Updated the OpenAI provider to support capability adapters for tool calls, vision, audio, and reasoning, enhancing modularity and extensibility.
- Introduced a new method for detecting API formats and streamlined the provider selection process.
- Enhanced documentation to reflect the new architecture and clarify provider capabilities and usage.
- Added support for lifecycle event messages, including a new event type for tracking stream states (e.g., stream_start, stream_end).
- Introduced a NewEventMessage function to create event messages with structured properties.
- Updated the output package to include event messages in the built-in types, ensuring compatibility with CUI clients while remaining silent for OpenAI clients.
- Enhanced the DefaultStreamHandler to utilize event messages for better lifecycle tracking during streaming operations.
- Improved error handling and logging in the OpenAI provider to capture and report streaming errors effectively.
- Added context cancellation handling in the OpenAI provider's Stream and Post methods, allowing for graceful termination of operations when the context is cancelled.
- Implemented a new test, TestOpenAIStreamContextCancellation, to validate that streaming respects context cancellation, ensuring proper error handling and event emission during cancellation scenarios.
- Refactored context usage in existing methods to improve consistency and reliability in handling context across streaming operations.
- Removed unnecessary blank lines in the TestOpenAIStreamLifecycleEvents function to improve code readability.
- Adjusted log formatting for group_end event to ensure consistent output during test execution.
- Added support for stream lifecycle events including stream_start, stream_end, group_start, and group_end.
- Introduced new data structures for handling lifecycle event data, enhancing communication of stream boundaries and metadata.
- Updated the OpenAI provider to track and emit these events during streaming operations, improving UI/UX capabilities.
- Added tests to validate the correct emission and order of lifecycle events during streaming.
- Introduced TestOpenAIJSONMode and TestOpenAIJSONModePost to validate JSON response formatting for streaming and non-streaming requests.
- Added TestOpenAIJSONSchema and TestOpenAIJSONSchemaPost to ensure compliance with strict JSON schema validation.
- Enhanced request body construction in the OpenAI provider to support JSON schema details in response formats.
- Added message preprocessing in the base provider to filter unsupported content types (vision and audio) based on model capabilities.
- Introduced new methods in the OpenAI provider for handling streaming and non-streaming requests with retry logic and tool call validation.
- Enhanced request body building to support various options and improved error handling for API interactions.
- Implemented validation for tool call arguments against JSON schemas, ensuring compliance with expected formats.
- Added global uses configuration to the assistant, allowing for centralized management of vision, audio, search, and fetch settings.
- Updated the Assistant struct and related methods to support the new Uses configuration, improving flexibility in assistant operations.
- Refactored the Stream method to utilize the new CompletionResponse type, enhancing response handling.
- Introduced new methods for building requests and managing capabilities, streamlining the assistant's interaction with various connectors.
