- Introduced unique ID generation for jobs, categories, and executions using gonanoid, improving ID handling and reducing collisions.
- Updated SaveJob and SaveCategory functions to utilize new ID generation methods, ensuring consistent and unique identifiers.
- Enhanced job creation logic to allow for category name handling, enabling automatic category ID assignment based on provided names.
- Refactored ensureCategoryExists to check for existing categories by name, streamlining category management during job operations.
- Improved error handling and logging for ID generation and category retrieval processes, ensuring better traceability and reliability.
- Updated job execution identifiers in AddFileAsync, AddTextAsync, and AddURLAsync to use more descriptive names.
- Removed obsolete async processing functions for file, text, and URL, streamlining the codebase.
- Enhanced the registration of document processing handlers in the knowledge base initialization, ensuring clarity and consistency.
- Renamed the Start method to Push for better semantic clarity in job execution.
- Introduced OnceAndSave, CronAndSave, and DaemonAndSave methods for immediate job creation and saving.
- Added new test cases for OnceAndSave, CronAndSave, and DaemonAndSave methods to ensure proper functionality.
- Updated AddFileProcess, AddTextProcess, and AddURLProcess to utilize new job handling methods, improving consistency across document operations.
- Enhanced error handling and logging in document processing functions for better traceability.
- Updated AddFileProcess, AddTextProcess, and AddURLProcess functions to use new UpdateDocumentCountWithSync method for document count updates, ensuring synchronization with GraphRag.
- Enhanced CreateCollection function to utilize UpdateCollectionWithSync for collection status updates, improving consistency in metadata management.
- Modified RemoveDocs function to sync document count updates to GraphRag, ensuring accurate tracking of affected collections.
- Introduced new utility functions for updating collections and document counts with GraphRag synchronization, enhancing overall API functionality.
- Added DocumentCount and UpdateDocumentCount methods to manage document counts in collections, enhancing metadata accuracy.
- Introduced RemoveDocumentsByCollectionID method for bulk document removal, improving collection management.
- Updated AddFileProcess, AddTextProcess, and AddURLProcess functions to include document and segment count updates after file operations.
- Enhanced RemoveCollection function to report the number of documents removed during collection deletion.
- Implemented segment count updates in RemoveSegments and RemoveSegmentsByDocID functions, ensuring accurate tracking of document segments.
- Updated AddFileProcess, AddTextProcess, and AddURLProcess functions to accept an optional job ID for async operations.
- Enhanced error handling by requiring document IDs to be provided by the caller before processing.
- Streamlined document creation and upsert operations, ensuring better validation and rollback mechanisms.
- Implemented async processing capabilities in AddFileAsync, AddTextAsync, and AddURLAsync functions, returning job IDs and document IDs in responses.
- Improved code maintainability and readability by encapsulating business logic and reducing dependencies on Gin context.
- Introduced separate processing functions (AddFileProcess, ProcessAddTextRequest) to encapsulate business logic for file and text additions, making them Gin-agnostic.
- Updated AddFile, AddText, and AddURL functions to handle request parsing and validation more effectively.
- Enhanced error handling and rollback mechanisms during document creation and upsert operations.
- Implemented async processing capabilities for AddFileAsync, AddTextAsync, and AddURLAsync functions using pre-parsed request data.
- Streamlined utility functions for preparing requests and validating input, improving code maintainability and readability.
- Added ListDocuments and ScrollDocuments functions for paginated document retrieval.
- Implemented GetDocument function to fetch document details by ID, including error handling for missing IDs.
- Introduced RemoveDocs function to handle document deletion requests.
- Removed outdated AddFile, AddText, and AddURL functions to streamline document management logic.
