- Introduced a Validator interface and a validateRequest function to streamline request validation across handlers.
- Added checkKBInstance function to ensure the knowledge base instance is initialized before processing requests.
- Implemented handleAsync function to manage asynchronous processing for file and text addition.
- Updated AddFile and AddText functions to utilize the new validation and async handling logic.
- Added async variants for AddFile and AddText to support non-blocking operations.
- Enhanced AddURL and its async variant with similar validation and processing improvements.
- Modified test cases for CreateCollection, RemoveCollection, and CollectionIntegration to use the new embedding provider and option values.
- Removed outdated fields related to the previous embedding model and added locale support for provider reading.
- Ensured consistency in the configuration structure across all relevant tests.
- Introduced ProviderSettings type to encapsulate provider configuration details.
- Implemented getProviderSettings function to resolve provider settings based on provider ID and option value.
- Updated CreateCollection function to utilize provider settings, including dimension and metadata enhancements.
- Refactored CreateCollectionRequest to include a new CreateCollectionConfig struct for better organization of embedding provider details.
- Implemented loading of providers from directories, allowing for multi-language configurations.
- Updated the KnowledgeBase struct to include a Providers field for managing provider configurations.
- Refactored GetProviders and GetProvider methods to utilize the new multi-language provider system, improving localization support.
- Added comprehensive test cases to validate provider loading and retrieval functionality across different languages.
- Implemented GetProviders and GetProvider methods to retrieve providers based on type and ID, improving the knowledge base's provider management capabilities.
- Enhanced the Schema methods for chunking providers to retrieve schemas from bindata, ensuring better integration with the overall system.
- Updated OpenAPI routes to include endpoints for provider management, allowing for easier access to provider information via HTTP requests.
- Implemented UpdateCollectionMetadata function to handle metadata updates for existing collections.
- Added request structure and validation for updating collection metadata.
- Integrated new endpoint in the router for updating collection metadata via PUT request.
- Modified the validation logic in the CreateCollectionRequest to allow the "collection name cannot be empty" error to be ignored during CreateCollectionOptions validation, improving flexibility in request handling.
- Added functionality to capture and return the raw query string and post payload in the JSON response for both public and protected hello world endpoints.
- Improved response structure to include additional context for debugging and client-side processing.
- Updated MakeRefreshToken and generateRefreshToken methods to accept an optional expiresIn parameter for customizable token expiration.
- Modified storeRefreshTokenWithScope to handle dynamic expiration times based on the provided parameter.
- Enhanced the authback function to send cookies with access and refresh tokens, including their respective expiration times.
- Updated LoginResponse structure to include RefreshTokenExpiresIn for better client-side management of token lifetimes.
- Adjusted client configuration to set default refresh token expiration duration.
- Updated test cases for dynamic client registration to include a scope field with values "openid profile email".
- Ensured that tests reflect the expected behavior when scope is provided, enhancing validation coverage for client registration requests.
- Added support for numeric ID generation in the OAuth service, replacing the previous NanoID approach for better compatibility.
- Refactored client ID and secret generation methods to be public and renamed them for consistency.
- Enhanced dynamic client registration to allow optional client ID usage.
- Updated client configuration loading to include validation and registration of clients if not found.
- Improved error handling and logging for client configuration processes.
- Adjusted tests to reflect changes in ID generation and client configuration handling.
- Added `SignIDToken` method to the OAuth service for signing ID tokens with user data and claims.
- Introduced `MakeAccessToken` and `MakeRefreshToken` methods for generating access and refresh tokens.
- Implemented user fingerprint management with `Subject` and `UserID` methods for mapping user IDs to unique NanoIDs.
- Enhanced `LoginByUserID` to generate and return OIDC tokens and access tokens during user login.
- Updated `OIDCUserInfo` mapping to include additional user attributes and improved handling of optional fields.
- Added `last_login_ip` field to the user model to track the IP address of the last login.
- Updated `UpdateUserLastLogin` method to accept an IP address parameter for logging.
- Introduced `GetUserWithScopes` method to retrieve user information along with associated scopes and permissions.
- Modified `LoginThirdParty` and `LoginByUserID` functions to include IP address handling during login processes.
- Improved user provider interface to reflect the new methods and parameters, enhancing overall user management capabilities.
- Updated test cases to utilize unique client IDs and user emails with suffixes for better isolation during parallel test execution.
- Enhanced the setupTestData function to generate unique identifiers for test clients and users, preventing conflicts in concurrent test runs.
- Implemented comprehensive cleanup patterns to ensure all test data is removed after execution, improving test reliability and maintainability.
- Modified the cleanupTestData function to query by user_id, ensuring consistency with the updated user provider interface.
- This change enhances the accuracy of test data cleanup and aligns with recent refactoring efforts in user management.
- Updated GetUserProvider to return an error alongside the user provider for better error handling.
- Modified CreateUser, CreateRole, and CreateType methods to return user and role IDs as strings instead of interfaces, enhancing type safety.
- Adjusted test data setup in oauth_test.go to reflect changes in user ID handling and ensure compatibility with the updated user provider interface.
- Removed unnecessary nil checks and improved assertions in tests for clarity and reliability.
- Updated GetProvider function to retrieve providers using only the provider ID, removing locale dependency.
- Introduced a global providers map to decouple provider configurations from locale-specific settings.
- Enhanced Load function to prioritize loading provider configurations before signin configurations.
- Added new helper functions for loading providers and signin configurations, improving code organization and maintainability.
- Updated tests to reflect changes in provider handling and ensure proper functionality.
- Implemented methods to check the existence of OAuth accounts, roles, types, and users by their respective identifiers, enhancing the user provider's functionality.
- Added error handling for these methods to ensure robust feedback in case of failures.
- Updated the user provider interface to include the new existence check methods, improving overall usability and maintainability.
- Added Multi-Factor Authentication (MFA) capabilities to the user provider, including methods for generating MFA secrets, enabling/disabling MFA, and verifying MFA codes.
- Introduced MFA configuration options, allowing customization of issuer, algorithm, digits, and recovery codes.
- Enhanced error handling for MFA operations, providing clear feedback for failures.
- Updated user model to include MFA-related fields and improved database interactions for MFA management.
- Refactored existing user provider methods to integrate MFA functionality seamlessly.
- Added methods for creating, retrieving, updating, and deleting user types, improving user type management capabilities.
- Introduced type field lists in DefaultUser and DefaultUserOptions for better type configuration.
- Implemented error handling for user type operations, ensuring robust feedback for failures.
- Enhanced user deletion process to clean up associated data before removing user accounts.
- Updated tests to cover new user type functionalities and ensure proper cleanup of test data.
- Deleted the DefaultUser implementation and its related test file, streamlining the codebase by removing unused functionality.
- This cleanup enhances maintainability and focuses on the current user management architecture.
- Added role field lists to DefaultUser and DefaultUserOptions for better role configuration.
- Implemented methods for creating, retrieving, updating, and deleting user roles, improving role management capabilities.
- Enhanced error handling for role operations, ensuring robust feedback for failures.
- Introduced ClearUserRole method to remove role assignments from users, streamlining user-role management.
- Updated tests to ensure proper cleanup of role data and maintain a clean testing environment.
- Updated the TestUser model to align with the latest user structure, replacing the 'Subject' field with 'UserID' for consistency with the new user provider interface.
- Modified all relevant test cases to use 'UserID' instead of 'Subject' when storing tokens and refresh tokens, ensuring compatibility with the updated user model.
- Enhanced test data setup to reflect changes in user creation and management, improving overall test reliability and clarity.
- Added methods for creating, retrieving, updating, and deleting OAuth accounts, improving user authentication capabilities.
- Introduced default OAuth account fields and options for better configuration management.
- Enhanced error handling for OAuth account operations, ensuring robust feedback for failures.
- Updated tests to ensure unique identifiers and improved cleanup processes for test data, maintaining a clean testing environment.
- Replaced the jaevor/go-nanoid library with the updated matoous/go-nanoid/v2 for generating unique IDs.
- Implemented user retrieval methods in the user provider, including GetUsers, PaginateUsers, and CountUsers, enhancing user management capabilities.
- Improved error handling and ensured default select fields are set for user queries, streamlining the user data access process.
- Added the go-nanoid library for generating unique IDs, enhancing user ID management.
- Refactored user provider methods to improve clarity and consistency, including updates to user retrieval and authentication processes.
- Adjusted user model fields to align with new ID generation strategy, ensuring compliance with best practices.
- Cleaned up code by removing obsolete test files and improving overall structure for better maintainability.
- Refactored the user model to align with OIDC standards, including renaming fields for clarity and compliance.
- Updated field descriptions and added new fields for enhanced user profile management.
- Removed the obsolete test file for user-related functionality to streamline the codebase and eliminate redundancy.
- Introduced a global OAuth service variable to streamline access across the application.
- Added new methods in the OAuth Service for retrieving cache and store, improving data management.
- Refactored session state handling in the Signin API to utilize the global OAuth service for better cache interactions.
- Updated key management functions for user info and redirect URIs to enhance clarity and maintainability.
- Introduced UUID-based state parameter generation for improved uniqueness and security in OAuth flows.
- Added validation for user-provided state parameters to ensure they conform to UUID format, with warnings included in the response.
- Refactored the generateRandomState function to utilize UUID generation instead of cryptographic random bytes.
- Updated OAuthAuthorizationURLResponse to include optional warnings about state format issues.
- Added OIDC standard types for ID Token and User Info, improving compliance with OpenID Connect specifications.
- Updated the Provider struct to support user info source types, allowing retrieval from endpoints, ID tokens, or access tokens.
- Enhanced the authback and user info retrieval functions to handle cached user data and improve error management.
- Introduced new utility functions for managing user info mapping and merging cached data, streamlining the user information processing flow.
- Removed unused OAuth response structures to streamline the codebase.
- Improved the authback function to handle redirect URIs and provider retrieval more effectively, enhancing error responses for invalid requests.
- Updated the getOAuthAuthorizationURL function to include better handling of provider configurations and session state management.
- Introduced a new function to normalize expiration formats for client secrets, ensuring consistent duration handling across providers.
- Removed unused maps import and added new structures for OAuth callback request and response, improving clarity and organization.
- Updated the authback function to utilize the new request structure, enhancing error handling and validation for incoming parameters.
- Improved response handling by returning the OAuth callback parameters directly, streamlining the success response process.
- Introduced SecureCookieOptions struct to define secure cookie configurations, enhancing cookie security.
- Added functions for sending secure cookies, including session and access token cookies, with customizable options.
- Updated signin and OAuth authorization URL handling to manage session IDs and state validation, improving security and user experience.
- Enhanced error handling for OAuth state and redirect URI management, ensuring robust session management during authentication flows.
- Introduced a new ResponseMode field in the Provider struct to accommodate additional OAuth provider requirements.
- Updated the getOAuthAuthorizationURL function to include response_mode in the authorization URL when specified, enhancing compatibility with providers like Apple that require this parameter.
- Added debug logging for redirect URI and client ID in the OAuth authorization URL function to aid in troubleshooting.
- Refactored the createPublicConfig function to perform deep copies of configuration structures, ensuring sensitive data is not exposed while maintaining necessary fields for frontend use.
- Improved handling of Captcha and ThirdParty provider configurations to exclude sensitive information, enhancing security in public-facing configurations.
- Integrated Captcha functionality by adding a new route for captcha handlers within the OpenAPI structure.
- Updated the Attach method to include the Captcha endpoint, ensuring OAuth protection for the new route.
- Enhanced overall routing organization by grouping captcha-related handlers under a dedicated path.
- Added Cloudflare Turnstile site key and secret to the environment variables in both `pr-test.yml` and `unit-test.yml` workflows.
- Introduced a new endpoint in the Signin API for generating OAuth authorization URLs, improving support for third-party authentication providers.
- Enhanced the handling of OAuth provider configurations, including validation and error responses for missing or incomplete settings.
- Updated tests to cover the new OAuth authorization URL functionality and ensure sensitive data is not exposed in public configurations.
- Introduced a comprehensive Signin API for user authentication, supporting multiple OAuth providers (Google, GitHub, Microsoft, Apple).
- Updated the file management API endpoints to use a singular `/file` path instead of `/files`, enhancing consistency across the API.
- Revised README documentation to include detailed descriptions of the new Signin API and updated file management endpoints.
- Enhanced test cases to reflect the changes in endpoint structure and ensure robust coverage for the new Signin functionality.
- Replaced the Agent API with the Chat API, focusing on AI chat completions with full OpenAI client compatibility and real-time streaming capabilities.
- Updated routing to attach chat handlers instead of agent handlers, ensuring OAuth protection for all endpoints.
- Revised README documentation to reflect the new Chat API structure, including detailed descriptions of endpoints, features, and usage examples.
- Removed the deprecated agent files and their associated documentation to streamline the codebase.
- Introduced the Agent API to facilitate AI agent interactions, including chat completions with real-time streaming capabilities.
- Implemented endpoints for GET and POST requests to handle chat completions, supporting features like context management and assistant selection.
- Updated the README to include comprehensive documentation for the Agent API, detailing its functionalities, key endpoints, and usage examples.
- Enhanced the OpenAPI structure to integrate the new agent handlers into the existing routing system, ensuring OAuth protection for all endpoints.
- Added LocalPath method to the Manager and Storage interfaces to retrieve the absolute path and content type of files.
- Enhanced local and S3 storage implementations to support LocalPath, including handling gzipped files and content type detection.
- Introduced comprehensive tests for LocalPath functionality, covering various file types, non-existent files, and gzipped content.
- Updated AddFile API to utilize LocalPath for retrieving file information, improving error handling and response consistency.
- Refactored the attachment manager to support file uploads with a new storage path and improved metadata handling.
- Implemented chunked uploads and direct content retrieval, enhancing performance and flexibility.
- Updated the file management API to include comprehensive operations for file uploads, downloads, and metadata management.
- Added support for multiple storage backends, including local and S3, with improved error handling and validation.
- Enhanced test coverage for file operations, ensuring reliability and consistency across different storage implementations.
- Added AddFile, AddText, AddURL, and AddSegments functions to handle file and segment uploads, including JSON request parsing and validation.
- Integrated error handling for invalid requests and uninitialized Knowledge Base instances, ensuring robust API responses.
- Enhanced response structures for success and error cases, improving consistency across the API.
- Updated AutoDetectConverter function to use a single content type parameter instead of multiple content types.
- Added a new test for resolving environment variables during configuration parsing, ensuring that environment variables are correctly substituted in the configuration.
- Refactored the configuration handling to resolve environment variables immediately after parsing, improving the clarity and usability of the configuration structure.
- Enhanced the collection management API by standardizing error responses and success responses using a custom response structure, improving consistency across endpoints.
- Implemented cleanup logic for test collections to ensure proper resource management during testing.
- Deleted obsolete test files for various OpenAPI components, including config_test.go, dsl_test.go, hello_test.go, oauth_test.go, oauth_token_test.go, and openapi_test.go, to streamline the codebase.
- Refactored OAuth response handling by integrating response methods from the response package, ensuring consistent error and success responses across OAuth endpoints.
- Enhanced error handling and response structure for improved clarity and maintainability, aligning with best practices for API responses.
- Introduced the Knowledge Base (KB) API by creating a new kb package with various endpoints for collection and document management, segment management, and search functionalities.
- Updated the OpenAPI router to attach the KB API, ensuring OAuth protection for all endpoints.
- Implemented logging for configuration validation to enhance error handling and user feedback.
- Deleted the existing hello world handlers from hello.go to streamline the codebase.
- Updated openapi.go to attach the new hello package for handling hello world routes, ensuring OAuth protection is applied correctly.
- Enhanced the README.md to reflect the new structure and provide comprehensive documentation for the hello world API endpoints.
- Updated response methods to standardize Content-Type header to "application/json" across OAuth endpoints, enhancing compliance with JSON standards.
- Refactored error and success response methods to streamline response generation without unnecessary wrappers, improving clarity and maintainability.
- Enhanced security by ensuring all responses include appropriate OAuth security headers, aligning with best practices for sensitive endpoints.
- Simplified test assertions for Content-Type in OAuth tests, ensuring consistency in response validation.
- Updated OAuth token handling to include PKCE (Proof Key for Code Exchange) parameters, improving security for authorization code grants.
- Refactored token management tests to incorporate PKCE code verifier and challenge, ensuring compliance with OAuth 2.1 standards.
- Enhanced refresh token handling to validate requested scopes against originally granted scopes, improving security and compliance.
- Updated various methods to support optional scope parameters, streamlining token management and validation processes.
- Improved test coverage for token introspection and exchange scenarios, ensuring robust validation of token handling logic.
- Added the JWKS endpoint to return JSON Web Key Set in compliance with RFC 7517, including necessary security headers.
- Refactored the JWKS generation logic to retrieve signing certificates and construct the JWK from the RSA public key.
- Introduced comprehensive tests for the JWKS endpoint, validating response format, compliance, and security headers.
- Updated go.mod to include the MongoDB driver as a required dependency.
