Commit graph

173 commits

Author SHA1 Message Date
Max
1060c0f0e0 Enhance attachment manager with dual text content handling
- Updated `GetText` and `SaveText` methods to support both full content and a preview (first 2000 characters) for improved performance and usability.
- Added comprehensive tests in `manager_test.go` to validate the new functionality, including UTF-8 character handling and content retrieval scenarios.
- Modified the attachment model to include a `content_preview` field alongside the existing `content` field, ensuring efficient data management.
- Enhanced the README.md to document the new dual storage feature, providing clear examples for users on how to utilize the updated methods.
2025-12-03 18:29:24 +08:00
Max
4170fbd13b Add text content storage functionality to attachment manager
- Implemented `GetText` and `SaveText` methods in the attachment manager to handle storing and retrieving parsed text content from files.
- Enhanced the `manager_test.go` with comprehensive tests for various scenarios, including saving, updating, and retrieving text, as well as handling non-existent file IDs.
- Updated the README.md to document the new text content storage features, providing examples for saving and retrieving parsed text.
- Modified the attachment model to include a new `content` field for storing text content, improving the overall functionality of the attachment management system.
2025-12-03 18:18:46 +08:00
Max
ed62033761 Add modes and default mode fields to Assistant model
- Introduced `modes` field to support multiple operational modes for the assistant, allowing for flexible functionality (e.g., chat, task).
- Added `default_mode` field to specify the primary mode of operation for the assistant.
- Updated the `Map`, `Clone`, and `Update` methods to handle the new fields appropriately.
- Enhanced the `ToAssistantModel` conversion function to include the new fields.
- Added tests to validate the integration and functionality of modes and default mode within the assistant model.
2025-12-02 19:02:31 +08:00
Max
da48896bdf Add database support to Assistant model and related functionalities
- Introduced a new `Database` type to encapsulate database models and options.
- Updated the `Assistant` model to include a `DB` field for database configuration.
- Enhanced the `Map`, `Clone`, and `Update` methods to handle the new `DB` field appropriately.
- Implemented the `ToDatabase` conversion function to support various input types for database initialization.
- Added comprehensive tests to validate the integration of the database functionality within the assistant model.
- Updated relevant fields and methods to ensure consistent handling of the new database configuration.
2025-12-02 18:45:03 +08:00
Max
45029269af Enhance assistant model with global prompts functionality
- Introduced the `disable_global_prompts` field in the Assistant model to control the usage of global prompts.
- Updated the Load and Get methods to initialize and retrieve global prompts from the configuration.
- Refactored tests to validate the new global prompts functionality and ensure proper loading and context handling.
- Improved the overall structure and clarity of the assistant's capabilities and configurations.
2025-12-02 15:37:41 +08:00
Max
339a486eb4 Refactor assistant capabilities and update data handling
- Enhanced the getConnectorCapabilities method to prioritize model capabilities and connector settings, improving capability retrieval logic.
- Deprecated the tools field in the Assistant model, transitioning to MCP for tool management, and updated related methods accordingly.
- Introduced new fields for connector options and prompt presets in the Assistant model, allowing for more flexible configurations.
- Updated the GetAssistant method to support field selection, improving data retrieval efficiency and flexibility.
- Refactored tests and documentation to reflect changes in the assistant structure and capabilities, ensuring clarity and maintainability.
2025-12-02 12:36:47 +08:00
Max
124bd38f7a Implement global uses configuration and enhance assistant capabilities
- Added global uses configuration to the assistant, allowing for centralized management of vision, audio, search, and fetch settings.
- Updated the Assistant struct and related methods to support the new Uses configuration, improving flexibility in assistant operations.
- Refactored the Stream method to utilize the new CompletionResponse type, enhancing response handling.
- Introduced new methods for building requests and managing capabilities, streamlining the assistant's interaction with various connectors.
2025-11-14 19:17:21 +08:00
Max
1d5f27cd98 Refactor assistant handling and storage integration
- Updated assistant data handling to utilize structured models, improving code clarity and maintainability.
- Enhanced the assistant save and load functions to work with the new AssistantModel structure.
- Refactored mention handling to streamline the conversion process from raw data to structured mentions.
- Improved error handling in the assistant save process, ensuring better feedback for invalid data.
- Updated API responses to use the new gin.H format for consistency across the application.
2025-11-07 11:20:59 +08:00
Max
95b2547c54 Refactor to decouple attachments, RAG, and AI assistant components.
- Deleted session-related functions (UserID, GuestID, UserRoles, UserOrGuestID) from the agent package to streamline the codebase.
- Removed RAG-related code and references, simplifying the agent's architecture.
- Updated API and load functions to reflect these changes, ensuring consistency across the agent module.
- Enhanced test coverage by cleaning up deprecated test cases related to removed functionalities.
2025-11-06 19:09:44 +08:00
Max
5d18e76acd Enhance attachment management with public and share fields
- Introduced new fields in the attachment model for public access control and sharing options, allowing attachments to be marked as public or shared with specific teams.
- Updated the upload process to handle new permission fields, ensuring proper handling of public and share options during file uploads.
- Implemented permission checks in the file retrieval and deletion processes to enforce access control based on user roles and attachment settings.
- Added comprehensive tests for the new permission fields to validate functionality and ensure correct behavior in various scenarios.
2025-11-05 18:11:36 +08:00
Max
0eff0091a0 Update asset modification times and refactor import statements
- Updated the modification times for various assets in `bindata.go` to reflect recent changes.
- Refactored import statements in `team_invitation.go` to maintain consistency and improve code organization by moving the `utils` import to the correct position.
2025-10-31 11:13:57 +08:00
Max
d17832c016 Update asset metadata and enhance collection features in the API
- Updated the modification times for various assets in `bindata.go` to reflect recent changes.
- Enhanced the collection management in the API by adding new fields for `preset`, `public`, and `share` in the collection and document models, allowing for better control over collection visibility and sharing options.
- Updated the `CreateCollection` function to incorporate the new `share` field, improving the handling of collection data based on user permissions.
- Refactored utility functions to streamline type conversions, ensuring consistent data handling across the user and team management functionalities.
2025-10-31 11:05:46 +08:00
Max
0ee38a1cb3 Implement attachment wrapper parsing and Base64 conversion in attachment manager
- Added `Parse` function to handle attachment wrapper strings, extracting uploader name and file ID.
- Introduced `Base64` function to convert attachment wrappers to Base64 format, with optional data URI support.
- Enhanced `readFilePathAsBase64` to read files from the filesystem and return Base64 encoded content.
- Updated `teamInvitationGetPublic` to process team logos and inviter pictures as Base64 for direct display in the response.
2025-10-30 09:59:09 +08:00
Max
7de562f80d Update robot member functionality with new fields and validation
- Introduced the `robot_email`, `authorized_senders`, and `email_filter_rules` fields to the robot member model, enhancing the configuration options for robot members.
- Updated the member creation and existence check logic to ensure the uniqueness of `robot_email` globally, improving data integrity.
- Refactored API endpoints and request/response structures to accommodate the new fields, ensuring consistency across the member management system.
- Enhanced test cases to validate the new robot member features, including scenarios for email uniqueness and configuration updates, improving overall test coverage and reliability.
2025-10-27 11:34:07 +08:00
Max
420a17f844 Enhance member model and team management with is_owner field
- Added the is_owner field to the member model to indicate if a member is the team owner, improving clarity and performance for queries related to team ownership.
- Updated the team creation process to automatically set the is_owner field for the creator of the team, ensuring accurate member status upon creation.
- Refactored related tests to verify the correct assignment of the is_owner field, enhancing test coverage and reliability in team membership scenarios.
- Removed redundant permission and restriction fields from the member model to streamline the data structure and improve maintainability.
2025-10-26 12:25:15 +08:00
Max
5f6f8d7911 Enhance member management with member_id and email checks
- Introduced member_id as a unique identifier for members, improving data integrity and consistency across member operations.
- Updated member-related functions to utilize member_id instead of internal IDs, enhancing API usability and clarity.
- Implemented email existence checks for team members, allowing for better validation during member creation and updates.
- Refactored existing tests to accommodate changes in member identification and email validation, ensuring comprehensive coverage and reliability.
- Added new API endpoint for checking if an email exists within a team, improving user experience during member management.
2025-10-26 09:49:45 +08:00
Max
f89dbc2109 Update member model and robot member handling
- Enhanced the member model by adding 'avatar' and updating 'display_name' and 'bio' fields for improved identity representation.
- Refactored robot member creation and validation to utilize the new fields, ensuring consistency across user and robot profiles.
- Updated test cases to reflect changes in member data structure, improving coverage and accuracy in robot member operations.
- Adjusted database indices and constraints to accommodate new fields and ensure efficient querying.
2025-10-25 17:41:30 +08:00
Max
f31b3f5882 Update member model to include profile fields and enhance invitation handling
- Added display_name, bio, and email fields to the member model for improved team-specific identity.
- Updated team invitation creation and acceptance logic to handle new profile fields, ensuring proper data management and user experience.
- Implemented logic to copy user profile fields when creating or updating members, enhancing data consistency.
2025-10-23 19:40:29 +08:00
Max
b1db5505ea Update team model and creation logic to support role management
- Added role_id field to the team model for specifying team owner roles.
- Enhanced team creation logic to set default role_id based on team configuration if not provided.
- Updated tests to reflect changes in team type and role handling, ensuring accurate assertions.
- Improved documentation for team configuration to clarify default role settings for team creators.
2025-10-20 17:00:33 +08:00
Max
9a2a636314 Add invitation model and related functionality
- Introduced the invitation model in bindata.go, including the new invitation.mod.yao file.
- Updated system models to include the invitation model for better integration.
- Added invitation code related error handling in default.go, enhancing user experience during invitation processes.
- Implemented invitation code management methods in the user provider interface, allowing for creation, usage, and deletion of invitation codes.
- Enhanced entry configuration to support invitation code verification, improving the user registration flow.
- Added a new endpoint for verifying invitation codes in user routing, streamlining the invitation redemption process.
2025-10-16 19:08:54 +08:00
Max
48f531cf82 Update asset modification times and enhance user authentication flow
- Updated modification times for various asset files in bindata.go to reflect recent changes.
- Enhanced user authentication by adding support for custom claims in the context during authorization, improving flexibility in handling user data.
- Refactored entry verification logic to generate and send OTPs for user registration, streamlining the registration process and improving user experience.
- Introduced new status handling for user accounts, ensuring proper feedback for pending, unverified, and disabled accounts during login attempts.
2025-10-16 10:28:00 +08:00
Max
87465379be Enhance user type management with locale support
- Added 'locale' field to user type structure for language localization.
- Updated API methods to include locale in pricing and published types retrieval.
- Enhanced tests to validate locale handling in user type operations, ensuring robust support for multiple languages.
2025-10-13 16:16:40 +08:00
Max
15f0750253 Enhance user type management with pricing and status fields
- Updated user type structure to include new fields for pricing: price_daily, price_monthly, price_yearly, and credits_monthly.
- Added status field to manage publishing states (draft, published, archived) for user types.
- Enhanced the user type API to support pricing operations, including retrieval and updates for pricing information.
- Refactored tests to cover new pricing functionalities and ensure robust handling of user type statuses.
2025-10-13 15:55:41 +08:00
Max
1af0a20015 Enhance user login functionality with detailed context tracking
- Updated the UpdateUserLastLogin method to accept a LoginContext, allowing for tracking of last login IP, user agent, device, and platform.
- Modified the LoginThirdParty and LoginByUserID functions to utilize the new LoginContext, improving the accuracy of login tracking.
- Enhanced the user model to include fields for last login details, ensuring comprehensive user activity logging.
- Refactored tests to validate the new login context handling, ensuring robust coverage of the updated functionality.
2025-10-13 09:58:16 +08:00
Max
b9a912b7d9 Enhance member invitation management and related functionality
- Introduced invitation_id handling for member management, allowing for unique identification of pending invitations.
- Added methods to create, update, and remove members by invitation_id, improving the invitation workflow.
- Updated member data structures and API responses to include invitation-related fields, enhancing clarity and usability.
- Implemented tests for invitation ID operations, ensuring robust validation and error handling.
- Refactored team access checks to streamline member management processes.
2025-09-23 17:47:43 +08:00
Max
a2df75b6e2 Enhance member model to support robot members and their configurations
- Updated the member model to include a new member_type field, allowing differentiation between human and robot members.
- Added fields specific to robot members, including robot_name, robot_description, and robot_avatar, to capture essential information.
- Introduced robot configuration fields such as robot_config, agents, tools, and data_access_permissions for enhanced functionality.
- Implemented scheduling and automation features for robots, including is_active_robot, schedule_config, and activity_frequency.
- Enhanced indexing to support efficient queries for robot members and their activities, improving overall system performance.
2025-09-16 10:00:04 +08:00
Max
580f3683e2 Refactor user and model structures to enhance organization and role management
- Reorganized models by replacing the assistant and chat models with a new agent structure, including assistant, chat, and history models.
- Updated user model relationships to reflect changes in role and type references, ensuring consistency with the new model structure.
- Removed obsolete organization and user role models, streamlining the overall architecture.
- Adjusted system model mappings to incorporate new member and role models, enhancing clarity in user management.
2025-09-16 09:35:18 +08:00
Max
5a23afb22c Add organization_user model and enhance relationships in existing models
- Introduced the organization_user model to manage relationships between organizations and users.
- Updated organization and user models to include new relationships for managing memberships and ownerships.
- Refactored system and test model mappings to incorporate the new organization_user model, ensuring consistency across the application.
2025-09-04 17:32:39 +08:00
Max
d9c593728e Add organization model and enhance user model indexing
- Introduced the organization model with associated data structure and methods for asset management.
- Updated user model to include new indexing on locale and zoneinfo for improved internationalization queries.
- Refactored system and test model mappings to include the new organization model, ensuring consistency across the application.
2025-09-04 17:17:16 +08:00
Max
71f7dcbbad Enhance job management and category handling
- Introduced field lists for job, category, execution, and log queries to streamline data retrieval.
- Updated ListJobs, GetJob, GetCategories, and ListLogs functions to utilize new field lists for improved query efficiency.
- Enhanced job creation logic to set default values for enabled status, ensuring new jobs are active by default.
- Added category name resolution in job retrieval, enriching job data with associated category names.
- Improved error handling and logging throughout job and category management processes for better traceability.
2025-09-02 18:36:14 +08:00
Max
58f830b7b6 Enhance job and category management with unique ID generation
- Introduced unique ID generation for jobs, categories, and executions using gonanoid, improving ID handling and reducing collisions.
- Updated SaveJob and SaveCategory functions to utilize new ID generation methods, ensuring consistent and unique identifiers.
- Enhanced job creation logic to allow for category name handling, enabling automatic category ID assignment based on provided names.
- Refactored ensureCategoryExists to check for existing categories by name, streamlining category management during job operations.
- Improved error handling and logging for ID generation and category retrieval processes, ensuring better traceability and reliability.
2025-09-02 16:57:23 +08:00
Max
a1869726d5 Enhance job execution and management features
- Implemented execution options to allow priority settings and shared data for job executions.
- Introduced new execution types for processes and commands, improving flexibility in job handling.
- Updated job and execution models to include new fields for execution options and priority.
- Refactored job submission logic to support context-aware execution, enhancing error handling and process management.
- Improved test coverage for job execution scenarios, ensuring robust validation of new features.
2025-09-01 12:26:29 +08:00
Max
38f86aa77b Implement job management functionality and update job model
- Added new job management functions: Once, Cron, and Daemon for creating jobs with different scheduling types.
- Introduced methods for job manipulation, including Start, Cancel, and various setters for job properties.
- Updated job model to reflect changes in the job execution process type, renaming "process_type" to "mode" and adjusting enum values to uppercase.
- Removed obsolete interfaces and types related to job management, streamlining the codebase.
2025-08-31 11:59:16 +08:00
Max
1664355401 Add job-related models to bindata and update system models
- Added new models for job management: category, execution, job, and log.
- Updated bindata.go to include paths for the new job models.
- Modified model.go and utils.go to register the new job models in system and test mappings, ensuring they are accessible in the application.
2025-08-30 17:43:08 +08:00
Max
28bbed7908 Enhance collection and document management in API
- Introduced readonly field for collections and documents to indicate if they are read-only.
- Updated ListCollections endpoint to support pagination and filtering by various parameters, including keywords, status, and system flags.
- Enhanced query capabilities with dynamic field selection and sorting options for collections.
- Refactored GetCollections to ListCollections for improved clarity in functionality.
- Updated API response structures to reflect new filtering and sorting features, ensuring better data access and management.
2025-08-29 18:02:37 +08:00
Max
91d67d3b54 Enhance collection management and segment handling in the API
- Added GetCollection endpoint to retrieve collections by ID, including error handling for missing IDs and uninitialized knowledge base.
- Updated CreateCollectionConfig to use more descriptive field names for embedding provider and option IDs.
- Refactored UpdateSegments function to improve validation and error handling, ensuring document IDs and segment texts are properly checked.
- Introduced new request structures for updating votes, scores, and weights for segments, enhancing the API's capabilities.
- Removed the outdated vote handling file to streamline the codebase.
2025-08-20 18:25:28 +08:00
Max
57082d1d42 Enhance document management by adding file ID support and updating related fields
- Introduced `file_id` field in the document model to link documents with their corresponding files in the attachment system.
- Updated `AddFileProcess` to include `file_id` in the response, improving document tracking.
- Modified `GetDocument` function to support the new `file_id` field in document retrieval.
- Removed the unused `ScrollDocuments` endpoint to streamline the API.
- Improved error handling and response structure for document-related operations.
2025-08-15 10:47:06 +08:00
Max
b2f0ef2a67 Update asset modification timestamps and enhance chunking options configuration
- Updated modification timestamps for various asset files to reflect recent changes.
- Modified chunking options to include new parameters: Separator and EnableDebug, with default values set.
- Enhanced the Options method in Structured and Semantic providers to handle additional properties from the option map, improving flexibility in configuration.
- Added new fields in the document model for converter and fetcher option IDs, allowing for better integration with provider configurations.
2025-08-14 10:06:05 +08:00
Max
6acea5b004 Refactor extractor to extraction provider and update related configurations
- Renamed extractor provider to extraction provider across the codebase for consistency and clarity.
- Updated references in configuration files, provider factories, and asset management to reflect the new terminology.
- Removed the extractor provider implementation and associated test files, streamlining the provider structure.
- Adjusted test cases and documentation to align with the new extraction provider framework.
2025-08-13 16:32:30 +08:00
Max
c9fdb2c01c Update asset modification timestamps and add support for application/octet-stream in attachment.local.yao
- Updated modification timestamps for various asset files to reflect recent changes.
- Added support for "application/octet-stream" MIME type in the attachment.local.yao file to enhance file upload capabilities.
2025-08-13 11:12:52 +08:00
Max
bab6eec3c3 Refactor knowledge base model structure to support new collection and document formats
- Replaced the existing kb.mod.yao file with two new files: collection.mod.yao and document.mod.yao, enhancing the organization of knowledge base models.
- Updated references in system and test models to reflect the new structure, ensuring compatibility with the updated model paths.
- Adjusted asset loading functions to accommodate the new model files, improving the overall asset management system.
2025-08-12 17:20:45 +08:00
Max
eb1bf43d61 Refactor provider schemas to utilize bindata for enhanced localization support
- Updated Schema methods for various providers (OpenAI, Fastembed, MCP, OCR, Office, UTF8, Video, Vision, Whisper) to retrieve schemas from bindata, improving localization capabilities.
- Removed outdated JSON files for embedding, extractor, and fetcher providers, streamlining the data structure.
- Enhanced the overall provider management by ensuring schemas are dynamically loaded based on locale, facilitating better multi-language support.
2025-08-10 15:51:34 +08:00
Max
0251bdc8d7 Add provider management functionality and enhance schema retrieval
- Implemented GetProviders and GetProvider methods to retrieve providers based on type and ID, improving the knowledge base's provider management capabilities.
- Enhanced the Schema methods for chunking providers to retrieve schemas from bindata, ensuring better integration with the overall system.
- Updated OpenAPI routes to include endpoints for provider management, allowing for easier access to provider information via HTTP requests.
2025-08-09 11:23:51 +08:00
Max
392b6bee1b Enhance user management functionality with IP tracking and scope retrieval
- Added `last_login_ip` field to the user model to track the IP address of the last login.
- Updated `UpdateUserLastLogin` method to accept an IP address parameter for logging.
- Introduced `GetUserWithScopes` method to retrieve user information along with associated scopes and permissions.
- Modified `LoginThirdParty` and `LoginByUserID` functions to include IP address handling during login processes.
- Improved user provider interface to reflect the new methods and parameters, enhancing overall user management capabilities.
2025-08-03 18:30:34 +08:00
Max
8992cb9130 Update model options to include permission field for user-related models
- Added "permission": true to the option field in user, user_role, user_type, user_oauth_account, and dsl models, enhancing permission management capabilities.
- This change ensures consistency across user-related models and improves the overall functionality of the user management system.
2025-08-03 15:59:14 +08:00
Max
e3e4b1bb77 Implement MFA functionality in user management
- Added Multi-Factor Authentication (MFA) capabilities to the user provider, including methods for generating MFA secrets, enabling/disabling MFA, and verifying MFA codes.
- Introduced MFA configuration options, allowing customization of issuer, algorithm, digits, and recovery codes.
- Enhanced error handling for MFA operations, providing clear feedback for failures.
- Updated user model to include MFA-related fields and improved database interactions for MFA management.
- Refactored existing user provider methods to integrate MFA functionality seamlessly.
2025-08-02 21:08:37 +08:00
Max
46c8fb7786 Update asset metadata timestamps and remove obsolete index from user_type model
- Updated timestamps for various asset metadata to reflect recent changes.
- Removed the obsolete index for role relationships from the user_type model, streamlining the database schema.
2025-08-02 11:02:59 +08:00
Max
6b53086e53 Add user role and type models, update user model fields
- Introduced user_role and user_type models to enhance user management capabilities.
- Updated user model fields to reference role and type identifiers, improving clarity and compliance with relational standards.
- Adjusted asset metadata timestamps and system/test model mappings to include new models, ensuring consistency across the application.
2025-08-02 10:53:59 +08:00
Max
ea4a492974 Add user_oauth_account model and update asset metadata
- Introduced the user_oauth_account model to enhance user authentication capabilities.
- Updated asset metadata timestamps for various files to reflect recent changes.
- Ensured consistency in the system and test models by including the new user_oauth_account model in both configurations.
2025-08-02 10:39:25 +08:00
Max
ae2b267aa1 Update user model and remove obsolete test file
- Refactored the user model to align with OIDC standards, including renaming fields for clarity and compliance.
- Updated field descriptions and added new fields for enhanced user profile management.
- Removed the obsolete test file for user-related functionality to streamline the codebase and eliminate redundancy.
2025-08-02 10:13:34 +08:00