Here is the formatted and organized version of your document for the AI Model Hub Service:

AI Model Hub Service

This project is a central hub service designed to route requests to various Large Language Models (LLMs) and manage associated data. It integrates a relational database (PostgreSQL) for metadata with a vector store (FAISS) for efficient similarity search.

High-Level Architecture

The service consists of several main components that work together to process user requests and manage data:

• API Server A FastAPI application that serves as the primary entry point for all client requests.

• LLM Router / Orchestrator The core logic that determines which LLM handles a request, manages chat history, and orchestrates interactions with the databases.

• Vector Database (FAISS + PostgreSQL) A two-part system for data storage:

  • PostgreSQL: Stores relational data like user sessions, chat messages, document metadata, and more.
  • FAISS: An in-memory index for high-speed vector similarity search.

Database Schema

The schema is designed to store conversations, documents, and the metadata necessary to link them together.

1. sessions

Stores individual conversations with a user.

• Columns: id, user_id, title, model_name, created_at, is_archived

2. messages

Stores each message within a session.

• Columns: id, session_id, sender, content, created_at, model_response_time, token_count, metadata

3. documents

Stores the original text and metadata of a document for RAG (Retrieval-Augmented Generation).

• Columns: id, title, text, source_url, author, status, created_at, user_id

4. vector_metadata

Links a document to its vector representation in the FAISS index.

• Columns: id, document_id, faiss_index, session_id, embedding_model

Technology Stack

• API Framework: FastAPI (Python)
• LLM Integration: LangChain (or similar)
• Relational Database: PostgreSQL
• Vector Store: FAISS
• ORM: SQLAlchemy
• Testing Framework: Pytest

Running Tests

To ensure the database models work correctly, a set of tests is provided using an in-memory SQLite database.

Steps to Run Tests

1. Navigate to the project root.

2. Set up your environment and install dependencies:

   pip install -e .[test]

   This installs the project in editable mode along with the test dependencies defined in pyproject.toml.

3. Run tests using Pytest:

   pytest

   Pytest will automatically discover and run all tests in the tests/ directory.

Let me know if you want this converted to Markdown, PDF, or documentation site format (e.g., Sphinx or MkDocs).