- Updated the LoginResponse structure to include MFAEnabled status, reflecting the user's MFA configuration.
- Modified the authback function in oauth.go to return the MFAEnabled status in the response.
- Introduced a utility function to convert various types to boolean for determining MFA status from user data.
- Removed the refreshCaptcha function from the login.go file to streamline the codebase.
- Updated the authback function in oauth.go to return a structured LoginSuccessResponse, including session ID, access token, refresh token, and their expiration times for improved response clarity.
- Introduced a new LoginSuccessResponse type in types.go to standardize the login success response format.
- Deleted the entire signin module, including related files and configurations, to streamline the authentication process.
- Updated OpenAPI to remove references to signin, including the loading of signin configurations and associated handlers.
- Refactored user authentication routes to integrate captcha functionality directly within the user module, enhancing user experience and security.
- Adjusted routing structure to reflect the removal of signin, ensuring clarity and consistency in user management operations.
- Consolidated cookie management into a new SendLoginCookies function, which sends access token, refresh token, and session ID cookies with appropriate security settings.
- Updated authback functions in the signin and oauth files to utilize the new SendLoginCookies function, enhancing code clarity and reducing duplication.
- Integrated user configuration loading in the OpenAPI server initialization to enhance user management capabilities.
- Migrated authentication routes from /signin to /login, updating handler functions for clarity and consistency.
- Adjusted OAuth provider routes to reflect new paths, improving the overall routing structure for user authentication.
- Renamed 2FA references to MFA in the user management API documentation and routing structure for consistency and clarity.
- Adjusted README to reflect the new endpoint paths for TOTP and SMS MFA functionalities.
- Updated README to reflect changes in user management APIs, renaming balance management to credits management and updating endpoints accordingly.
- Enhanced team management section with detailed CRUD operations and member management, including invitation handling.
- Refactored user.go to align with new API structure, replacing balance with credits and invite management with referral management, while adding invitation response handling.
- Integrated user and team handler functions into the OpenAPI server's routing structure.
- Established new routes for user and team management under the OpenAPI framework, enhancing the server's capabilities for handling user-related operations.
- Updated TestMemberInvitationExpiry to use a consistent expired time for invitations.
- Simplified invitation expiration check in AcceptInvitation by utilizing a new checkTimeExpired function.
- Enhanced member ID parsing in AcceptInvitation to handle potential errors more gracefully.
- Refactored total extraction logic in CountOAuthAccounts, CountRoles, CountTeams, CountTypes, and CountUsers to use a new utility function for improved clarity and consistency.
- Introduced parseTimeFromDB and parseIntFromDB functions to streamline data parsing from database fields.
- Introduced new constants for team and member error handling, enhancing error reporting.
- Added default field lists for teams and members, improving data structure organization.
- Expanded the UserProvider interface with methods for team and member management, including creation, updates, and retrieval.
- Enhanced the NewDefaultUser function to support team and member configurations, ensuring flexibility in user management.
- Reorganized models by replacing the assistant and chat models with a new agent structure, including assistant, chat, and history models.
- Updated user model relationships to reflect changes in role and type references, ensuring consistency with the new model structure.
- Removed obsolete organization and user role models, streamlining the overall architecture.
- Adjusted system model mappings to incorporate new member and role models, enhancing clarity in user management.
- Added Label field to the Provider struct in types.go to allow for better identification of third-party login providers.
- Updated createPublicConfig function in signin.go to populate the new Label field when creating public provider configurations.
- Implemented a keywords filter in the ListJobs function to allow searching by job name and description.
- Refactored enabled status handling to support multiple truthy and falsy values, improving flexibility in job retrieval.
- Default behavior remains to show all records regardless of enabled status, ensuring no data is inadvertently filtered out.
- Introduced field lists for job, category, execution, and log queries to streamline data retrieval.
- Updated ListJobs, GetJob, GetCategories, and ListLogs functions to utilize new field lists for improved query efficiency.
- Enhanced job creation logic to set default values for enabled status, ensuring new jobs are active by default.
- Added category name resolution in job retrieval, enriching job data with associated category names.
- Improved error handling and logging throughout job and category management processes for better traceability.
- Introduced unique ID generation for jobs, categories, and executions using gonanoid, improving ID handling and reducing collisions.
- Updated SaveJob and SaveCategory functions to utilize new ID generation methods, ensuring consistent and unique identifiers.
- Enhanced job creation logic to allow for category name handling, enabling automatic category ID assignment based on provided names.
- Refactored ensureCategoryExists to check for existing categories by name, streamlining category management during job operations.
- Improved error handling and logging for ID generation and category retrieval processes, ensuring better traceability and reliability.
- Updated job execution identifiers in AddFileAsync, AddTextAsync, and AddURLAsync to use more descriptive names.
- Removed obsolete async processing functions for file, text, and URL, streamlining the codebase.
- Enhanced the registration of document processing handlers in the knowledge base initialization, ensuring clarity and consistency.
- Renamed the Start method to Push for better semantic clarity in job execution.
- Introduced OnceAndSave, CronAndSave, and DaemonAndSave methods for immediate job creation and saving.
- Added new test cases for OnceAndSave, CronAndSave, and DaemonAndSave methods to ensure proper functionality.
- Updated AddFileProcess, AddTextProcess, and AddURLProcess to utilize new job handling methods, improving consistency across document operations.
- Enhanced error handling and logging in document processing functions for better traceability.
- Updated AddFileProcess, AddTextProcess, and AddURLProcess functions to use new UpdateDocumentCountWithSync method for document count updates, ensuring synchronization with GraphRag.
- Enhanced CreateCollection function to utilize UpdateCollectionWithSync for collection status updates, improving consistency in metadata management.
- Modified RemoveDocs function to sync document count updates to GraphRag, ensuring accurate tracking of affected collections.
- Introduced new utility functions for updating collections and document counts with GraphRag synchronization, enhancing overall API functionality.
- Introduced readonly field for collections and documents to indicate if they are read-only.
- Updated ListCollections endpoint to support pagination and filtering by various parameters, including keywords, status, and system flags.
- Enhanced query capabilities with dynamic field selection and sorting options for collections.
- Refactored GetCollections to ListCollections for improved clarity in functionality.
- Updated API response structures to reflect new filtering and sorting features, ensuring better data access and management.
- Added DocumentCount and UpdateDocumentCount methods to manage document counts in collections, enhancing metadata accuracy.
- Introduced RemoveDocumentsByCollectionID method for bulk document removal, improving collection management.
- Updated AddFileProcess, AddTextProcess, and AddURLProcess functions to include document and segment count updates after file operations.
- Enhanced RemoveCollection function to report the number of documents removed during collection deletion.
- Implemented segment count updates in RemoveSegments and RemoveSegmentsByDocID functions, ensuring accurate tracking of document segments.
- Enhanced GetSegmentGraph function to retrieve segment entities and relationships based on query parameters, improving data access.
- Introduced new endpoint GetSegmentRelationshipsByEntities for fetching relationships connected to entities, with appropriate error handling.
- Updated response structures to reflect counts and types of queries used, ensuring clarity in API responses.
- Modified go.mod and go.sum to update the dependency for github.com/ulikunitz/xz to version 0.5.14.
- Implemented new endpoints for retrieving segment entities and relationships, improving data access for clients.
- Added error handling for missing document and segment IDs, ensuring robust validation in GetSegmentEntities and GetSegmentRelationships functions.
- Enhanced GetSegmentGraph function to support filtering options for entities and relationships, allowing for more flexible responses.
- Updated response structures to include counts for entities and relationships, improving clarity in API responses.
- Added error handling for segment not found and access denied scenarios in GetSegmentParents function.
- Implemented response formatting based on query parameters, including metadata options.
- Introduced formatSegmentTreeResponse function to structure segment data for API responses.
- Improved validation to ensure segments belong to the specified document, enhancing security and clarity in responses.
- Implemented error handling for uninitialized knowledge base in GetHit and GetVote functions.
- Added detailed error responses for cases where hits or votes are not found.
- Updated response structures to return the retrieved hit and vote data along with associated identifiers.
- Added error handling for uninitialized knowledge base in hit and vote endpoints.
- Implemented ScrollHits and ScrollVotes methods to retrieve paginated results with filtering options.
- Updated AddHits and AddVotes functions to handle request validation and response formatting.
- Enhanced response structures to include updated counts for added and removed hits and votes.
- Introduced new request types for updating hits and votes, allowing for default reactions in segments.
- Updated API response structures across multiple endpoints to replace `document_id` with `doc_id` for consistency.
- Enhanced error handling in segment retrieval to ensure proper validation of document ownership.
- Improved response clarity by ensuring all relevant endpoints reflect the updated field naming convention.
- Renamed segment extraction functions to better reflect their purpose, changing `ExtractSegmentEntities` to `ExtractSegmentGraph` and `ExtractSegmentEntitiesAsync` to `ExtractSegmentGraphAsync`.
- Updated API routes to use the new function names for segment extraction, enhancing clarity and consistency.
- Introduced new request structures for batch updates of scores and weights, improving the API's capabilities for segment management.
- Removed the outdated `store.go` file to streamline the codebase.
- Rearranged and updated API endpoints for collections, including adding GetCollections and adjusting the order of operations for better clarity.
- Enhanced segment management by introducing new endpoints for segment scores, weights, and votes, along with async operations for adding and updating segments.
- Improved error handling for missing document IDs in segment-related functions, ensuring robust validation.
- Streamlined response structures to include document and segment IDs in relevant endpoints, enhancing the API's usability.
- Added GetCollection endpoint to retrieve collections by ID, including error handling for missing IDs and uninitialized knowledge base.
- Updated CreateCollectionConfig to use more descriptive field names for embedding provider and option IDs.
- Refactored UpdateSegments function to improve validation and error handling, ensuring document IDs and segment texts are properly checked.
- Introduced new request structures for updating votes, scores, and weights for segments, enhancing the API's capabilities.
- Removed the outdated vote handling file to streamline the codebase.
- Replaced ListSegments function with ScrollSegments to implement iterator-style pagination.
- Updated related comments and error messages to reflect the new scrolling functionality.
- Streamlined API routes by removing the redundant ListSegments endpoint, enhancing overall clarity.
- Enhanced ListSegments function to support pagination with customizable limit and offset parameters.
- Added error handling for missing docID and uninitialized knowledge base instance.
- Introduced filtering options for score, weight, and vote, allowing for refined segment retrieval.
- Improved response structure to return success or error messages based on the operation outcome.
- Introduced `file_id` field in the document model to link documents with their corresponding files in the attachment system.
- Updated `AddFileProcess` to include `file_id` in the response, improving document tracking.
- Modified `GetDocument` function to support the new `file_id` field in document retrieval.
- Removed the unused `ScrollDocuments` endpoint to streamline the API.
- Improved error handling and response structure for document-related operations.
- Enhanced ListDocuments function to support pagination with customizable page size and sorting options.
- Added filtering capabilities for keywords, tags, collection IDs, and status, allowing for more refined document retrieval.
- Introduced validation for requested fields and sorting parameters to ensure only valid options are processed.
- Improved error handling for document search operations, returning appropriate error responses when necessary.
- Updated AddFileProcess, AddTextProcess, and AddURLProcess functions to accept an optional job ID for async operations.
- Enhanced error handling by requiring document IDs to be provided by the caller before processing.
- Streamlined document creation and upsert operations, ensuring better validation and rollback mechanisms.
- Implemented async processing capabilities in AddFileAsync, AddTextAsync, and AddURLAsync functions, returning job IDs and document IDs in responses.
- Improved code maintainability and readability by encapsulating business logic and reducing dependencies on Gin context.
- Updated modification timestamps for various asset files to reflect recent changes.
- Modified chunking options to include new parameters: Separator and EnableDebug, with default values set.
- Enhanced the Options method in Structured and Semantic providers to handle additional properties from the option map, improving flexibility in configuration.
- Added new fields in the document model for converter and fetcher option IDs, allowing for better integration with provider configurations.
- Introduced separate processing functions (AddFileProcess, ProcessAddTextRequest) to encapsulate business logic for file and text additions, making them Gin-agnostic.
- Updated AddFile, AddText, and AddURL functions to handle request parsing and validation more effectively.
- Enhanced error handling and rollback mechanisms during document creation and upsert operations.
- Implemented async processing capabilities for AddFileAsync, AddTextAsync, and AddURLAsync functions using pre-parsed request data.
- Streamlined utility functions for preparing requests and validating input, improving code maintainability and readability.
- Added ListDocuments and ScrollDocuments functions for paginated document retrieval.
- Implemented GetDocument function to fetch document details by ID, including error handling for missing IDs.
- Introduced RemoveDocs function to handle document deletion requests.
- Removed outdated AddFile, AddText, and AddURL functions to streamline document management logic.
- Renamed extractor provider to extraction provider across the codebase for consistency and clarity.
- Updated references in configuration files, provider factories, and asset management to reflect the new terminology.
- Removed the extractor provider implementation and associated test files, streamlining the provider structure.
- Adjusted test cases and documentation to align with the new extraction provider framework.
- Updated the resolveProviderOption function to include providerType as a required parameter, enhancing error handling for missing provider types.
- Simplified the provider retrieval logic to directly fetch providers based on the specified type, improving clarity and maintainability.
- Removed redundant comments and streamlined the code for better readability.
- Added GetConfig function to retrieve the knowledge base configuration, ensuring proper initialization checks.
- Refactored CreateCollection, AddFile, AddText, and AddURL functions to utilize the new GetConfig method for improved error handling and database record management.
- Introduced preparation functions (PrepareCreateCollection, PrepareAddFile, PrepareAddText, PrepareAddURL) to streamline request handling and database data preparation.
- Enhanced error responses and rollback mechanisms for document and collection operations, improving robustness and clarity in error handling.
- Introduced a Validator interface and a validateRequest function to streamline request validation across handlers.
- Added checkKBInstance function to ensure the knowledge base instance is initialized before processing requests.
- Implemented handleAsync function to manage asynchronous processing for file and text addition.
- Updated AddFile and AddText functions to utilize the new validation and async handling logic.
- Added async variants for AddFile and AddText to support non-blocking operations.
- Enhanced AddURL and its async variant with similar validation and processing improvements.
- Modified test cases for CreateCollection, RemoveCollection, and CollectionIntegration to use the new embedding provider and option values.
- Removed outdated fields related to the previous embedding model and added locale support for provider reading.
- Ensured consistency in the configuration structure across all relevant tests.
- Introduced ProviderSettings type to encapsulate provider configuration details.
- Implemented getProviderSettings function to resolve provider settings based on provider ID and option value.
- Updated CreateCollection function to utilize provider settings, including dimension and metadata enhancements.
- Refactored CreateCollectionRequest to include a new CreateCollectionConfig struct for better organization of embedding provider details.
- Implemented loading of providers from directories, allowing for multi-language configurations.
- Updated the KnowledgeBase struct to include a Providers field for managing provider configurations.
- Refactored GetProviders and GetProvider methods to utilize the new multi-language provider system, improving localization support.
- Added comprehensive test cases to validate provider loading and retrieval functionality across different languages.
- Implemented GetProviders and GetProvider methods to retrieve providers based on type and ID, improving the knowledge base's provider management capabilities.
- Enhanced the Schema methods for chunking providers to retrieve schemas from bindata, ensuring better integration with the overall system.
- Updated OpenAPI routes to include endpoints for provider management, allowing for easier access to provider information via HTTP requests.
- Implemented UpdateCollectionMetadata function to handle metadata updates for existing collections.
- Added request structure and validation for updating collection metadata.
- Integrated new endpoint in the router for updating collection metadata via PUT request.
- Modified the validation logic in the CreateCollectionRequest to allow the "collection name cannot be empty" error to be ignored during CreateCollectionOptions validation, improving flexibility in request handling.
