Server lifecycle:
- Introduce service.Service to manage HTTP + gRPC startup/shutdown
- Fix gRPC mutex deadlock in StartServer when port is occupied
- Add GracefulStop with 5s timeout before forced Stop in grpc.go
- Pre-check HTTP and gRPC port availability in cmd/start.go
- Print gRPC server address in startup access-points block
gRPC client refactor:
- Move token manager and client from tai/grpc/ to grpc/client/
- Add backward-compatible aliases in tai/yao.go and tai/token.go
- Update cmd/run.go to import grpc/client directly (no tai dependency)
Sandbox v2 docker migration:
- Delete sandbox/v2/docker/ (moved to tai repo)
- Update sandbox/docker/build.sh hint to point to tai repo
- Clean up .gitignore entries for removed docker directory
- Temporarily disable SandboxV2Test and BenchmarkSandboxV2 in CI
(docker images need rebuild after tai repo migration)
Tai integration:
- Add direct-mode registration API handlers in tai/api/
- Add heartbeat handler and token management wrappers
- Update tai/registry and tai/tunnel for latest protocol
- Replace yao-grpc references with tai call in docs
Made-with: Cursor
The bubbletea-based TUI for agent request visualization was not
practical and added significant binary size. This removes the TUI
entirely: the --tui flag, the agent context TUI model/messages, and
all SendTUI/GetTUIProgram call sites. Development-mode logging now
always prints ANSI-colored output to stdout.
Drops direct dependencies on charmbracelet/bubbletea and lipgloss.
Made-with: Cursor
- Introduce a new "tunnel" scheme for the Tai client, allowing connections through Yao's reverse proxy.
- Enhance the Tai registry to manage tunnel-connected nodes and their ports.
- Add WebSocket and reverse proxy routes for tunnel connections in the OpenAPI server.
- Implement tunnel-specific proxy and VNC handling to facilitate communication with containerized environments.
- Update gRPC environment variable handling to support tunnel connections.
These changes improve the Tai service's flexibility and connectivity options, enabling better integration with remote and containerized environments.
- Add steps to generate kubeconfig specifically for benchmarks in both `pr-test.yml` and `unit-test.yml`, ensuring accurate configuration for Kubernetes interactions.
- Modify the Docker run command to mount the newly generated kubeconfig for benchmarks, enhancing the integration of Tai with K8s.
- Remove redundant kubeconfig generation steps to streamline the workflow.
These changes improve the CI workflows for the Tai service by ensuring proper kubeconfig setup for benchmark testing.
- Add Docker socket volume mount to the Docker run commands in both `pr-test.yml` and `unit-test.yml`, allowing the Tai K8s container to interact with the host Docker daemon.
- Update kubeconfig generation steps to dynamically retrieve the K3D IP address, ensuring accurate configuration for Kubernetes interactions.
These changes improve the integration of the Tai service with Kubernetes by enabling better communication with the Docker environment during CI workflows.
- Implement steps to generate kubeconfig for both the Tai K8s container and the test runner, ensuring proper configuration for Kubernetes interactions.
- Update the Docker run command to mount the generated kubeconfig, enhancing the integration of Tai with K8s.
- Remove the previous kubeconfig generation step for the Tai K8s proxy, streamlining the workflow.
These changes improve the CI workflows for the Tai service by ensuring accurate kubeconfig generation and integration with Kubernetes environments.
- Update the `testPools` function to include a new "containerized" pool configuration, allowing for testing with Tai running in a containerized environment.
- Revise comments to clarify the conditions under which each pool configuration is available, improving documentation for future reference.
These changes improve the flexibility of the testing framework by accommodating additional deployment scenarios for the Tai service.
- Add installation and setup steps for k3d to create a local Kubernetes cluster for testing.
- Implement health checks for both Docker and K8s instances of Tai, improving readiness verification.
- Generate kubeconfig for K8s benchmarks, ensuring proper configuration for testing environments.
- Update environment variables to accommodate new K8s service configurations, enhancing overall CI reliability.
These changes improve the testing framework for the Tai service by integrating Kubernetes support and refining service readiness checks.
- Rename and enhance the Docker instance startup steps for Tai in CI workflows, improving clarity and readiness checks for both HTTP and gRPC services.
- Update health check logic to ensure accurate reporting of service readiness, including specific error messages for failures.
- Modify environment variable configurations to streamline the setup for K8s and Docker instances, ensuring consistent port usage across tests.
These changes improve the reliability and clarity of the CI processes for the Tai service, enhancing overall testing and deployment workflows.
- Add workspace integration as a new feature in Sandbox V2, allowing for persistent user storage.
- Update the Manager API to include new methods for workspace management and image handling.
- Refactor the Pool struct to include additional options and improve lifecycle management.
- Revise implementation status in IMPL.md to reflect completed phases and optimizations, ensuring clarity on the current state of the project.
These changes enhance the functionality and documentation of Sandbox V2, improving user experience and system architecture.
- Introduce new unit test targets for Workspace and Sandbox V2 integration in the Makefile, ensuring comprehensive testing of the new features.
- Update CI workflows to support MongoDB service for Sandbox V2 tests and improve Docker image handling.
- Implement Workspace as a first-class entity, allowing for persistent storage decoupled from container lifecycles.
- Enhance the Box struct to manage workspace IDs and update related methods for improved functionality.
- Refactor tests to accommodate new Workspace features, ensuring robust testing of the integration.
These changes significantly improve the testing framework and functionality of the Sandbox V2, enhancing the overall architecture and user experience.
- Add new test cases for VNC URL generation and connection in `box_attach_test.go`, ensuring proper functionality of VNC features.
- Update the Docker test image to include VNC desktop components, enhancing the testing environment for graphical applications.
- Modify the entrypoint script to start a virtual framebuffer and VNC server, allowing for remote desktop access during tests.
- Increase the test timeout in the Makefile to accommodate longer-running VNC tests.
These changes improve the testing framework for Sandbox V2 by integrating VNC capabilities, facilitating better testing of graphical applications.
- Add new gRPC endpoint for Heartbeat in the Yao service, enabling communication with the sandbox.
- Update Makefile to include a dedicated unit test target for Sandbox V2, ensuring proper testing of new features.
- Enhance CI workflows to incorporate Sandbox V2 tests, allowing for dual-mode testing (local and remote) with Docker.
- Modify .gitignore to exclude specific Docker files while allowing shell scripts for Sandbox V2.
- Update documentation in DESIGN.md to reflect the new architecture and capabilities of the Sandbox V2.
These changes enhance the Yao SDK's functionality, providing improved support for sandbox operations and testing.
- Add TAI_TEST_HOST_IP environment variable to CI workflows for unit tests, allowing better connectivity to the gRPC server from Docker containers.
- Update the `run.go` file to parse command-line arguments correctly.
- Modify the test utility to return the gRPC address reachable from Docker, improving integration test reliability.
- Refactor integration tests to utilize the new relay address function, ensuring proper communication with the Yao gRPC server.
These changes improve the testing framework and enhance the configuration for better service interaction during CI runs.
- Improve health check logic in CI workflows for both HTTP and gRPC readiness of the Tai service, ensuring clearer error reporting if the service fails to start.
- Update the OAuth Device Flow implementation to support additional claims during device authorization, enhancing the flexibility of the authorization process.
- Refactor the `AuthorizeDevice` method to accept extra claims, allowing for more detailed user context during authorization.
- Introduce a new utility function to extract bearer tokens from requests, streamlining token handling across the OpenAPI service.
These changes enhance the robustness of service readiness checks and improve the OAuth device authorization flow, contributing to a more reliable and flexible authentication mechanism.
- Update health check messages in CI workflows to specify HTTP and gRPC readiness for the Tai service.
- Refactor integration tests to use `require` assertions instead of `assert`, improving error handling and test reliability.
- Ensure that responses from gRPC calls are not nil, enhancing test robustness.
These changes improve the clarity of service readiness checks and strengthen the integration test suite.
- Add support for the OAuth Device Authorization Flow (RFC 8628) in the OpenAPI service, allowing devices with limited input capabilities to obtain authorization.
- Implement `DeviceAuthorization()` and `AuthorizeDevice()` methods to handle device and user code generation, storage, and authorization.
- Update the OAuth endpoints to include `/device/authorize` for user code authorization and fix the discovery endpoint path for device authorization.
- Introduce MongoDB service in CI workflows for testing and enhance the unit test workflow with Redis setup.
- Update Go module dependencies to include necessary packages for the new features.
This commit significantly advances the OAuth capabilities of the application, enabling a more flexible authorization process for devices.
- Mark Phase 4 and Phase 5 as complete, indicating successful removal of fixed upstream connections and the introduction of dynamic routing based on request metadata.
- Implement `TokenManager` for handling authentication tokens and update gRPC client methods to support new features.
- Enhance tests for dynamic routing and token management, achieving significant coverage improvements.
- Begin preparations for Phase 6, outlining the structure for OAuth Device Flow and related tasks.
This commit finalizes key gRPC features and sets the stage for upcoming authentication enhancements.
- Add gRPC server configuration to the application, allowing for gRPC communication.
- Introduce new Makefile targets for gRPC unit testing and proto code generation.
- Update CI workflows to include gRPC tests with SQLite as the transport layer.
- Refactor the sandbox design to support multi-node capabilities and improve isolation.
- Enhance the service layer to facilitate internal request forwarding for gRPC APIs.
This commit lays the groundwork for integrating gRPC into the Yao SDK, improving performance and scalability.
- Add K8s sandbox implementation (sandbox/k8s.go) using client-go
- Extend tai.New() to support K8s runtime via Tai proxy
- Remove containerd support, fix default port assignment bug
- Update CI workflows to spin up k3d cluster and route K8s tests
through Tai proxy for full end-to-end coverage
- Make test ports configurable via environment variables
Made-with: Cursor
- Introduce new Tai SDK tests in the GitHub workflows, requiring a Tai container with Docker socket mount for execution.
- Update the Makefile to include a dedicated target for running Tai SDK tests, enhancing test coverage for the Tai integration.
- Modify the Go module dependencies to include the pierrec/lz4 package, ensuring compatibility with the new tests.
- Adjust test folder selection logic in the Makefile to exclude additional directories, streamlining the testing process.
- Introduce a fallback mechanism in the GetConnector method to use a default connector if the specified connector is not found.
- Add logging to notify when falling back to the default connector, improving debugging and traceability.
- Update comments for clarity on connector selection priority, ensuring better understanding of the logic flow.
- Add a `--force` flag to the `push` command in both agent and MCP modules, allowing users to overwrite existing versions by deleting them before pushing.
- Update the `PushOptions` struct to include the `Force` field, ensuring the functionality is integrated into the push logic.
- Enhance E2E tests to validate the behavior of the force push feature, ensuring that existing versions are deleted as expected when the flag is used.
- Introduce tests for packing directories to respect `.yaoignore` rules, improving file exclusion handling during the packaging process.
- Change the MCP server configuration from a map to a slice in both the E2E test helper functions and the agent management code, improving consistency in data representation.
- Update the test cases to reflect the new server structure, ensuring compatibility with the modified MCP configuration.
- Increase the timeout for the unit-test-registry target in the Makefile to 5 minutes for improved test execution.
- Refactor the prompt structure in E2E tests to align with the new format, enhancing clarity and consistency in the test setup.
- Change the test scope identifier from "@test" to "@yaoagents" in client tests for consistency.
- Remove the outdated E2E test file `e2e_test.go` from the manager directory, as it is no longer needed.
- Enhance the agent manager to utilize the MCP manager for dependency installations, improving the handling of MCP-type dependencies.
- Introduce new MCP and Robot command groups in the CLI, allowing for better organization of package management commands.
- Add corresponding command descriptions for MCP and Robot functionalities to improve user guidance.
- Update the agent command group to include additional commands for enhanced agent management.
- Modify .gitignore to exclude specific design markdown files from version control.
- Add environment variable for test application path in GitHub workflows to streamline testing setup.
- Add environment variables for registry initialization in both `pr-test.yml` and `unit-test.yml`.
- Implement a waiting mechanism for the registry service to ensure readiness before executing tests.
- Remove redundant steps for user creation and service restart, streamlining the test setup process.
- Enhance the client code to support auto-discovery of the API prefix, improving flexibility in server URL configuration.
- Introduce comprehensive error handling in client tests to cover various edge cases and improve test coverage.
- Update .gitignore to exclude registry data files.
- Modify Makefile to include new unit-test-registry target for running tests against the Yao Registry service.
- Add comprehensive Registry Client SDK tests in GitHub workflows, ensuring proper setup and execution of tests with Docker service dependencies.
- Implement steps for user creation, service restart, and test execution, along with reporting results back to the pull request.
- Introduce a new Tags field in the Server struct to allow for additional metadata.
- Update the listServers function to populate the Tags field from the server metadata, enhancing the server representation with more contextual information.
- Consolidate JSON serialization for assistant fields into a single function, `marshalJSONFields`, to streamline the code and improve maintainability.
- Introduce a new utility function, `isNil`, to accurately check for nil values, including typed nils in interface{} types.
- Remove redundant marshaling logic for options, tags, and modes, enhancing clarity and reducing code duplication.
- Update the `UpdateAssistant` method to utilize the new nil-checking logic for JSON fields, ensuring correct handling of nil values.
- Introduce a new field for dependencies in the Assistant model to manage external MCP client dependencies with version constraints.
- Implement deep copy functionality for dependencies in the Clone method to ensure integrity during assistant cloning.
- Enhance the Update method to handle dependencies input from data maps, supporting both string and interface types.
- Update the Map method to include dependencies in the serialized output.
- Add comprehensive tests to validate loading, cloning, and mapping of dependencies, ensuring correct behavior across various scenarios.
- Introduce a refresh gate mechanism to prevent duplicate refresh attempts for the same token, allowing concurrent requests to safely handle expired tokens.
- Update the guard and refresh token logic to utilize the new error handling for in-progress refresh operations, enhancing the reliability of the authentication flow.
- Remove the previous caching mechanism for refresh status, streamlining the code and improving clarity in the refresh process.
- Introduce a new method to control the maximum number of history messages loaded, prioritizing user-defined options over store settings.
- Update the history loading logic to filter out non-semantic message types and convert tool call and action messages into historical summaries for better context.
- Refactor the message conversion functions to handle different formats and ensure clarity in the historical context provided to the LLM.
- Add comprehensive tests to validate the new behavior and ensure accurate message handling in various scenarios.
- Enhance the OAuth authentication flow to allow concurrent requests to safely handle expired tokens without triggering multiple refresh attempts.
- Introduce a mechanism to mark refresh tokens as being rotated, ensuring that only one request processes the refresh while others can proceed with valid claims.
- Update the guard logic to reflect these changes, improving the overall efficiency and reliability of token management.
- Update the delivery handling logic to only push messages to integration channels when the task originates from one, ensuring more accurate message routing.
- Enhance the error logging to include channel information for better debugging during integration reply failures.
- Streamline the conditions for message delivery to improve code clarity and maintainability.
- Add locale support for DingTalk and Discord integrations to ensure messages are sent in the correct language.
- Update message formatting logic in DingTalk and Discord to utilize dedicated formatting functions for Markdown.
- Implement image and file handling improvements in Feishu integration, allowing for better attachment processing and error handling.
- Refactor message sending methods to streamline the process and enhance code clarity across integrations.
- Introduce a new sanitizeDSL function to clean up unsupported wildcards in the generated DSL.
- Adjust the sequence of operations in SearchWithContext to reflect the new sanitization step.
- Update comments to accurately reflect the changes in the execution flow and improve code clarity.
- Introduce a new POST endpoint for verifying integration credentials at /robots/integrations/verify.
- Ensure the endpoint is positioned correctly in the routing to avoid conflicts with existing routes.
- Enhance the API's capabilities for integration management by allowing credential verification.
- Introduce Discord adapter in the robot lifecycle to enable integration with Discord events.
- Update the integration dispatcher to recognize and handle events from Discord.
- Modify the configuration structure to include settings for Discord integration.
- Enhance the integration parsing logic to support Discord configurations.
<message>
- Introduce Feishu and DingTalk adapters in the robot lifecycle for enhanced integration capabilities.
- Update the integration dispatcher to include new adapters for handling events from Feishu and DingTalk.
- Modify the configuration structure to support settings for both Feishu and DingTalk integrations.
- Enhance the integration parsing logic to recognize and process configurations for Feishu and DingTalk.
- Update GetMessages function to ensure that when a limit is specified, the most recent messages are retrieved in descending order and then reversed for chronological output.
- Adjust pagination handling to apply a large limit when only an offset is provided, maintaining clarity in message retrieval.
- Enhance query ordering to prioritize message creation time and sequence for consistent results.
- Add Telegram integration support by introducing a dispatcher for handling Telegram events and messages.
- Implement event notifications for robot configuration changes (creation, update, deletion) to facilitate integration with external services.
- Refactor the robot initialization process to load robots into cache and start the dispatcher, improving the overall system setup.
- Update the delivery event structure to include additional metadata for better context during message handling.
- Enhance logging capabilities for better observability during robot execution and event processing.
- Introduce ServerURL field in YaoMetadata to provide the public server URL for the instance, with a fallback mechanism.
- Implement resolveServerURL function to determine the ServerURL based on the YAO_SERVER_URL environment variable or the issuer_url.
- Update yaoMetadata method to populate the ServerURL field, enhancing API endpoint construction for clients and integrations.
- Implement TestEnsureChatMetadata to verify that metadata, including robot_id, is correctly persisted in chat records.
- Update EnsureChat method to store metadata from the context when creating chat records.
- Introduce TestExecutorGoalsInjection to validate that pre-confirmed goals are injected into executions from TriggerInput.Data.
- Enhance executor logic to handle goal injection and persistence, ensuring accurate execution titles.
- Modify chat filtering to support chat_id_prefix for improved chat retrieval based on robot identifiers.
- Update the filename logic in convertAttachments to prefer the semantic title from the delivery agent over the raw storage filename.
- Ensure the original file extension is preserved when using the human-readable title, improving clarity for email clients.
- Adjust logging to reflect the updated filename handling for better observability during attachment processing.
