Commit graph

2520 commits

Author SHA1 Message Date
Max
a39184c2fa
Merge pull request #1045 from trheyi/main
Enhance attachment management with file storage improvements
2025-07-26 19:26:13 +08:00
Max
070ff59225 Enhance attachment management with file storage improvements
- Refactored the attachment manager to support file uploads with a new storage path and improved metadata handling.
- Implemented chunked uploads and direct content retrieval, enhancing performance and flexibility.
- Updated the file management API to include comprehensive operations for file uploads, downloads, and metadata management.
- Added support for multiple storage backends, including local and S3, with improved error handling and validation.
- Enhanced test coverage for file operations, ensuring reliability and consistency across different storage implementations.
2025-07-26 19:25:41 +08:00
Max
4deb845701
Merge pull request #1044 from trheyi/main
Update S3 tests
2025-07-26 11:07:55 +08:00
Max
0fe42b517f Update S3 storage tests to use UUIDs for file identifiers
- Modified test cases in `storage_test.go` to generate unique file IDs using UUIDs, ensuring no conflicts during tests.
- Updated the download test for non-existent files to utilize a UUID-based identifier, enhancing test reliability.
2025-07-26 11:07:20 +08:00
Max
5d5d0d4ce9 Improve error messages in S3 storage operations
- Enhanced error handling in the S3 storage implementation by including the fileID in error messages for upload, upload chunk, merge chunks, and download operations. This change provides clearer context for debugging and improves the overall error reporting mechanism.
2025-07-25 19:15:18 +08:00
Max
18547852b7 Update S3 configuration in CI workflows
- Replaced the S3_CACHE_PATH variable with S3_PUBLIC_URL in both `pr-test.yml` and `unit-test.yml` workflows, enhancing the integration with S3 by providing a public URL for access.
- This change aligns with the ongoing improvements to S3-related environment variables for better deployment and testing processes.
2025-07-25 19:05:42 +08:00
Max
aab4fde9b4
Merge pull request #1043 from trheyi/main
Add S3 configuration to CI workflows
2025-07-25 18:29:07 +08:00
Max
d8cc1f2390 Add S3 configuration to CI workflows
- Added S3-related environment variables to both `pr-test.yml` and `unit-test.yml` workflows, enabling integration with S3 for testing and deployment.
- Included API, access key, secret key, bucket, and cache path settings to facilitate S3 interactions during CI processes.
2025-07-25 18:28:41 +08:00
Max
6a894df269
Merge pull request #1042 from trheyi/main
Enhance attachment model and add local uploader support
2025-07-25 18:16:24 +08:00
Max
2f13dbbffb Enhance attachment model and add local uploader support
- Introduced a new `Groups` field in the `Attachment` struct to support categorization of attachments.
- Integrated `types.MetaInfo` into `ManagerOption` for improved metadata handling.
- Added a local uploader for attachments, enabling file uploads from a specified local path.
- Updated asset management functions to reflect changes in file metadata and structure.
2025-07-25 18:15:51 +08:00
Max
2376d13979
Merge pull request #1041 from trheyi/main
Refactor attachment handling and enhance upload options
2025-07-25 17:25:54 +08:00
Max
435b482c38 Refactor attachment handling and enhance upload options
- Updated the API to include new upload options for handling different storage types: chat, knowledge, and assets.
- Improved group building logic for attachment uploads based on user, chat, and assistant IDs.
- Removed deprecated attachment files and refactored related code to streamline the attachment management process.
- Enhanced error handling for required fields in upload options, ensuring better validation and user feedback.
2025-07-25 17:25:12 +08:00
Max
51d15e3afd
Merge pull request #1040 from trheyi/main
Implement file and segment management API endpoints with error handling
2025-07-25 16:35:52 +08:00
Max
1c4fd3d20c Implement file and segment management API endpoints with error handling
- Added AddFile, AddText, AddURL, and AddSegments functions to handle file and segment uploads, including JSON request parsing and validation.
- Integrated error handling for invalid requests and uninitialized Knowledge Base instances, ensuring robust API responses.
- Enhanced response structures for success and error cases, improving consistency across the API.
- Updated AutoDetectConverter function to use a single content type parameter instead of multiple content types.
2025-07-25 16:35:19 +08:00
Max
106c107666
Merge pull request #1039 from trheyi/main
Remove obsolete configuration and test files for Knowledge Base
2025-07-25 11:36:54 +08:00
Max
7fa825a334 Remove obsolete configuration and test files for Knowledge Base
- Deleted config.go, config_test.go, types.go, and related test files to streamline the codebase and remove deprecated components.
- Refactored Knowledge Base loading logic to utilize updated configuration structures, enhancing clarity and maintainability.
- Updated references in kb.go to align with the new configuration types, ensuring compatibility with the latest changes.
2025-07-25 11:36:15 +08:00
Max
29a4ad0912
Merge pull request #1038 from trheyi/main
Implement environment variable resolution in configuration parsing an…
2025-07-24 17:09:28 +08:00
Max
bb64eef20d Implement environment variable resolution in configuration parsing and enhance collection management API
- Added a new test for resolving environment variables during configuration parsing, ensuring that environment variables are correctly substituted in the configuration.
- Refactored the configuration handling to resolve environment variables immediately after parsing, improving the clarity and usability of the configuration structure.
- Enhanced the collection management API by standardizing error responses and success responses using a custom response structure, improving consistency across endpoints.
- Implemented cleanup logic for test collections to ensure proper resource management during testing.
2025-07-24 17:08:58 +08:00
Max
6d6230fe5f
Merge pull request #1037 from trheyi/main
Remove deprecated test files and refactor OAuth response handling
2025-07-24 15:53:55 +08:00
Max
8d35e824ae Remove deprecated test files and refactor OAuth response handling
- Deleted obsolete test files for various OpenAPI components, including config_test.go, dsl_test.go, hello_test.go, oauth_test.go, oauth_token_test.go, and openapi_test.go, to streamline the codebase.
- Refactored OAuth response handling by integrating response methods from the response package, ensuring consistent error and success responses across OAuth endpoints.
- Enhanced error handling and response structure for improved clarity and maintainability, aligning with best practices for API responses.
2025-07-24 15:53:29 +08:00
Max
f2234d96d4
Merge pull request #1036 from trheyi/main
Refactor Knowledge Base loading in Load and Reload functions
2025-07-23 18:01:12 +08:00
Max
40519e8429 Refactor Knowledge Base loading in Load and Reload functions
- Moved Knowledge Base loading logic from the Load function to ensure it is consistently handled in both Load and Reload functions.
- Improved error handling by printing errors during Knowledge Base loading in the Reload function, enhancing user feedback.
2025-07-23 17:56:51 +08:00
Max
d8e0130787
Merge pull request #1035 from trheyi/main
Enhance CI workflows and Knowledge Base integration
2025-07-23 17:42:59 +08:00
Max
d25ce68ab9 Enhance CI workflows and Knowledge Base integration
- Updated GitHub Actions workflows to use the latest Ubuntu version and added services for Neo4j, Qdrant, and FastEmbed.
- Integrated FFmpeg and additional utilities into the CI environment, ensuring they are installed and tested during the workflow.
- Implemented environment variable resolution in the Knowledge Base configuration, allowing dynamic configuration based on environment settings.
- Added tests for environment variable resolution in the Knowledge Base configuration, ensuring correct parsing and handling of variables.
2025-07-23 17:42:10 +08:00
Max
694b3f589d
Merge pull request #1034 from trheyi/main
Add Knowledge Base configuration and integration with GraphRag
2025-07-23 17:08:47 +08:00
Max
b76afe23bf Add Knowledge Base configuration and integration with GraphRag
- Introduced a new configuration structure for the Knowledge Base, supporting vector and graph database configurations.
- Implemented JSON parsing methods for loading configurations from files and converting them to JSON format.
- Enhanced the Knowledge Base instance to integrate with GraphRag, allowing for dynamic configuration of vector and graph stores.
- Added feature detection capabilities to determine available functionalities based on the current configuration.
- Updated the Knowledge Base loading process to read from a specified configuration file, improving flexibility and usability.
2025-07-23 17:07:38 +08:00
Max
b0f15fbc92
Merge pull request #1033 from trheyi/main
Add Knowledge Base API endpoints and integrate with OpenAPI router
2025-07-23 15:20:33 +08:00
Max
aae1bccf88 Add Knowledge Base API endpoints and integrate with OpenAPI router
- Introduced the Knowledge Base (KB) API by creating a new kb package with various endpoints for collection and document management, segment management, and search functionalities.
- Updated the OpenAPI router to attach the KB API, ensuring OAuth protection for all endpoints.
- Implemented logging for configuration validation to enhance error handling and user feedback.
2025-07-23 15:19:40 +08:00
Max
1589380d38
Merge pull request #1032 from trheyi/main
Deprecate studio API functions and comment out implementation details
2025-07-22 17:41:30 +08:00
Max
a921a76778 Deprecate studio API functions and comment out implementation details
- Marked the studio API functions (Start, Load, loadDSL, loadScripts) as deprecated, returning an error message to indicate their status.
- Commented out the implementation details of these functions to prevent execution of deprecated functionality, ensuring clarity in the codebase.
2025-07-22 17:41:03 +08:00
Max
2e630c5b03
Merge pull request #1031 from trheyi/main
Deprecate studio API functions and comment out test cases
2025-07-22 17:30:13 +08:00
Max
7bdeec85f9 Deprecate studio API functions and comment out test cases
- Marked the studio API functions (Start, Load, loadDSL, loadScripts) as deprecated, returning an error message to indicate their status.
- Commented out all test cases in studio_test.go to prevent execution of deprecated functionality, ensuring clarity in the test suite.
2025-07-22 17:29:42 +08:00
Max
d3077c217c
Merge pull request #1030 from trheyi/main
Remove hello world endpoints and refactor routing to use new hello pa…
2025-07-22 16:55:49 +08:00
Max
92d21c5389 Remove hello world endpoints and refactor routing to use new hello package
- Deleted the existing hello world handlers from hello.go to streamline the codebase.
- Updated openapi.go to attach the new hello package for handling hello world routes, ensuring OAuth protection is applied correctly.
- Enhanced the README.md to reflect the new structure and provide comprehensive documentation for the hello world API endpoints.
2025-07-22 16:55:24 +08:00
Max
d9392f14b6
Merge pull request #1029 from trheyi/main
Refactor OAuth response handling and improve content type management
2025-07-22 15:37:11 +08:00
Max
d54572a7d1 Refactor OAuth response handling and improve content type management
- Updated response methods to standardize Content-Type header to "application/json" across OAuth endpoints, enhancing compliance with JSON standards.
- Refactored error and success response methods to streamline response generation without unnecessary wrappers, improving clarity and maintainability.
- Enhanced security by ensuring all responses include appropriate OAuth security headers, aligning with best practices for sensitive endpoints.
- Simplified test assertions for Content-Type in OAuth tests, ensuring consistency in response validation.
2025-07-22 15:36:41 +08:00
Max
285ca7b5ed
Merge pull request #1028 from trheyi/main
Enhance OAuth token management with PKCE support and refactor tests
2025-07-22 12:56:33 +08:00
Max
7e8d4a9ba9 Enhance OAuth token management with PKCE support and refactor tests
- Updated OAuth token handling to include PKCE (Proof Key for Code Exchange) parameters, improving security for authorization code grants.
- Refactored token management tests to incorporate PKCE code verifier and challenge, ensuring compliance with OAuth 2.1 standards.
- Enhanced refresh token handling to validate requested scopes against originally granted scopes, improving security and compliance.
- Updated various methods to support optional scope parameters, streamlining token management and validation processes.
- Improved test coverage for token introspection and exchange scenarios, ensuring robust validation of token handling logic.
2025-07-22 12:55:44 +08:00
Max
a09ddd70ac
Merge pull request #1027 from trheyi/main
Implement JWKS endpoint and enhance OAuth tests
2025-07-21 20:08:28 +08:00
Max
41c44cb726 Implement JWKS endpoint and enhance OAuth tests
- Added the JWKS endpoint to return JSON Web Key Set in compliance with RFC 7517, including necessary security headers.
- Refactored the JWKS generation logic to retrieve signing certificates and construct the JWK from the RSA public key.
- Introduced comprehensive tests for the JWKS endpoint, validating response format, compliance, and security headers.
- Updated go.mod to include the MongoDB driver as a required dependency.
2025-07-21 20:07:46 +08:00
Max
966e0cfd00 Enhance OAuth test setup with reusable certificate management
- Introduced global test certificate paths to avoid redundant certificate generation across tests, improving efficiency.
- Implemented a function to create temporary certificates once for all tests, ensuring consistent usage of signing certificates.
- Updated test configurations to utilize the new certificate management, enhancing clarity and maintainability.
- Added cleanup functionality for global test certificates to ensure proper resource management after tests.
2025-07-21 19:44:00 +08:00
Max
e1428551ba Refactor OAuth token management tests and remove deprecated methods
- Removed outdated token management tests from the user provider, including tests for storing, revoking, and retrieving tokens.
- Updated the user provider interface to reflect the removal of token management methods, ensuring cleaner code and improved maintainability.
- Streamlined the test suite by focusing on relevant user management functionalities, enhancing overall test clarity and effectiveness.
2025-07-21 18:59:26 +08:00
Max
8d1174d566 Refactor hello world endpoints and add OAuth protection
- Renamed existing hello world endpoints to public and added a new protected endpoint with OAuth guard.
- Updated test cases to reflect the new endpoint structure and added tests for protected endpoint access with and without valid tokens.
- Enhanced response handling for public and protected endpoints to ensure consistent output and proper status codes.
2025-07-21 18:49:20 +08:00
Max
55931bb59b Add OAuth token revocation and introspection tests
- Implemented comprehensive tests for the OAuth token revocation and introspection endpoints, ensuring correct handling of valid, invalid, and missing token scenarios.
- Enhanced the oauthRevoke and oauthIntrospect methods to comply with RFC specifications, returning appropriate status codes and responses.
- Introduced a utility function for obtaining access tokens directly in tests, streamlining the testing process for OAuth endpoints.
- Improved error handling and logging for better debugging and verification during tests.
2025-07-21 17:57:34 +08:00
Max
8bce24a00b
Merge pull request #1026 from trheyi/main
Add OAuth token handling and enhance grant type support
2025-07-21 17:41:10 +08:00
Max
2b171c3540 Refactor OAuth token handling and enhance grant type support
- Consolidated token grant handling into a unified method for authorization code, client credentials, and device code grants, improving code organization and reducing duplication.
- Introduced new methods for handling token exchange and refresh token grants, ensuring compliance with relevant RFCs.
- Enhanced error handling and validation for client credentials and grant types, improving robustness and security.
- Updated tests to utilize real authorization codes and ensure comprehensive coverage of the new functionality.
2025-07-21 17:40:08 +08:00
Max
e4a02d6b9c Add OAuth authorization tests and improve error handling 2025-07-21 16:21:40 +08:00
Max
b1996594a2
Merge pull request #1025 from trheyi/main
Enhance OAuth functionality with registration tests and response impr…
2025-07-21 16:08:18 +08:00
Max
e936407452 Enhance OAuth functionality with registration tests and response improvements
- Added a comprehensive test for the OAuth client registration endpoint, ensuring proper handling of valid requests and responses.
- Updated the response handling methods to comply with RFC standards, including a new method for direct OAuth responses.
- Refactored existing response methods to improve security header management and streamline response generation.
- Enhanced test logging for better debugging and verification of response data during OAuth registration tests.
2025-07-21 16:07:40 +08:00
Max
437d15af10
Merge pull request #1024 from trheyi/main
Implement OAuth endpoint logic and enhance test setup
2025-07-21 15:35:23 +08:00