- Introduced new fields in the Job struct for Yao integration, including YaoCreatedBy, YaoUpdatedBy, YaoTeamID, and YaoTenantID.
- Updated the makeJob function to utilize YaoCreatedBy for setting the CreatedBy field.
- Enhanced permission checks in various functions to validate user access to collections, ensuring proper authorization handling.
- Implemented permission validation in document handling functions to restrict access based on user roles and collection settings.
- Implemented authorization checks in the RemoveCollection function to ensure users have the necessary permissions before removing a collection.
- Introduced error handling for unauthorized access attempts, returning appropriate responses for permission denials.
- Enhanced security by validating user access based on their authorization information.
- Implemented authorization checks in the UpdateCollectionMetadata function to ensure users have the necessary permissions before making updates.
- Introduced a new checkCollectionPermission function to validate user access based on team and owner constraints.
- Enhanced error handling to return appropriate responses for unauthorized access attempts.
- Integrated authorization information retrieval in `ListCollections` and `ListDocuments` functions to enhance security.
- Implemented permission-based filtering to ensure users can only access collections and documents they are authorized to view.
- Refactored query parameter construction for improved readability and maintainability.
- Updated the modification times for various assets in `bindata.go` to reflect recent changes.
- Enhanced the collection management in the API by adding new fields for `preset`, `public`, and `share` in the collection and document models, allowing for better control over collection visibility and sharing options.
- Updated the `CreateCollection` function to incorporate the new `share` field, improving the handling of collection data based on user permissions.
- Refactored utility functions to streamline type conversions, ensuring consistent data handling across the user and team management functionalities.
- Updated AddFileProcess, AddTextProcess, and AddURLProcess functions to use new UpdateDocumentCountWithSync method for document count updates, ensuring synchronization with GraphRag.
- Enhanced CreateCollection function to utilize UpdateCollectionWithSync for collection status updates, improving consistency in metadata management.
- Modified RemoveDocs function to sync document count updates to GraphRag, ensuring accurate tracking of affected collections.
- Introduced new utility functions for updating collections and document counts with GraphRag synchronization, enhancing overall API functionality.
- Introduced readonly field for collections and documents to indicate if they are read-only.
- Updated ListCollections endpoint to support pagination and filtering by various parameters, including keywords, status, and system flags.
- Enhanced query capabilities with dynamic field selection and sorting options for collections.
- Refactored GetCollections to ListCollections for improved clarity in functionality.
- Updated API response structures to reflect new filtering and sorting features, ensuring better data access and management.
- Added DocumentCount and UpdateDocumentCount methods to manage document counts in collections, enhancing metadata accuracy.
- Introduced RemoveDocumentsByCollectionID method for bulk document removal, improving collection management.
- Updated AddFileProcess, AddTextProcess, and AddURLProcess functions to include document and segment count updates after file operations.
- Enhanced RemoveCollection function to report the number of documents removed during collection deletion.
- Implemented segment count updates in RemoveSegments and RemoveSegmentsByDocID functions, ensuring accurate tracking of document segments.
- Added GetCollection endpoint to retrieve collections by ID, including error handling for missing IDs and uninitialized knowledge base.
- Updated CreateCollectionConfig to use more descriptive field names for embedding provider and option IDs.
- Refactored UpdateSegments function to improve validation and error handling, ensuring document IDs and segment texts are properly checked.
- Introduced new request structures for updating votes, scores, and weights for segments, enhancing the API's capabilities.
- Removed the outdated vote handling file to streamline the codebase.
- Added GetConfig function to retrieve the knowledge base configuration, ensuring proper initialization checks.
- Refactored CreateCollection, AddFile, AddText, and AddURL functions to utilize the new GetConfig method for improved error handling and database record management.
- Introduced preparation functions (PrepareCreateCollection, PrepareAddFile, PrepareAddText, PrepareAddURL) to streamline request handling and database data preparation.
- Enhanced error responses and rollback mechanisms for document and collection operations, improving robustness and clarity in error handling.
- Introduced a Validator interface and a validateRequest function to streamline request validation across handlers.
- Added checkKBInstance function to ensure the knowledge base instance is initialized before processing requests.
- Implemented handleAsync function to manage asynchronous processing for file and text addition.
- Updated AddFile and AddText functions to utilize the new validation and async handling logic.
- Added async variants for AddFile and AddText to support non-blocking operations.
- Enhanced AddURL and its async variant with similar validation and processing improvements.
- Introduced ProviderSettings type to encapsulate provider configuration details.
- Implemented getProviderSettings function to resolve provider settings based on provider ID and option value.
- Updated CreateCollection function to utilize provider settings, including dimension and metadata enhancements.
- Refactored CreateCollectionRequest to include a new CreateCollectionConfig struct for better organization of embedding provider details.
- Implemented UpdateCollectionMetadata function to handle metadata updates for existing collections.
- Added request structure and validation for updating collection metadata.
- Integrated new endpoint in the router for updating collection metadata via PUT request.
- Modified the validation logic in the CreateCollectionRequest to allow the "collection name cannot be empty" error to be ignored during CreateCollectionOptions validation, improving flexibility in request handling.
- Added a new test for resolving environment variables during configuration parsing, ensuring that environment variables are correctly substituted in the configuration.
- Refactored the configuration handling to resolve environment variables immediately after parsing, improving the clarity and usability of the configuration structure.
- Enhanced the collection management API by standardizing error responses and success responses using a custom response structure, improving consistency across endpoints.
- Implemented cleanup logic for test collections to ensure proper resource management during testing.
- Deleted obsolete test files for various OpenAPI components, including config_test.go, dsl_test.go, hello_test.go, oauth_test.go, oauth_token_test.go, and openapi_test.go, to streamline the codebase.
- Refactored OAuth response handling by integrating response methods from the response package, ensuring consistent error and success responses across OAuth endpoints.
- Enhanced error handling and response structure for improved clarity and maintainability, aligning with best practices for API responses.
- Introduced the Knowledge Base (KB) API by creating a new kb package with various endpoints for collection and document management, segment management, and search functionalities.
- Updated the OpenAPI router to attach the KB API, ensuring OAuth protection for all endpoints.
- Implemented logging for configuration validation to enhance error handling and user feedback.
