A micro framework for LLM semantic search.
graph TD;
subgraph load[load data]
KB[Knowledge Base] --> Parser[Parser and normaliser]
Parser --> Splitter
Splitter --> Embedder
Embedder --> VectorDatabase
subgraph Embedder
chanks[chanks list] --> llm[LLM SentenceTransformer]
llm --> vectors[vectors list]
end
end
graph TD;
subgraph search[search data]
SR[search request] --> llm[LLM SentenceTransformer]
llm --> VectorDatabase
VectorDatabase --> Render
Render --> User
end
- Python 3.11/3.12
- Ubuntu 24.04
- GPU 4Gb VRAM - CUDA 5, ROCm 6
git lfs install
git clone https://huggingface.co/sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2 st- ePub book
Put ePub book to repo root, for example UpgradePC20.epub
- For CPU
python3 -m venv .venv_llm
source ./.venv_llm/bin/activate
python -m pip install --upgrade pip
pip install -r requirements.txt- For GPU AMD ROCm 6
python3 -m venv .venv_llm
source ./.venv_llm/bin/activate
python -m pip install --upgrade pip
pip install torch --index-url https://download.pytorch.org/whl/rocm6.0
pip install -r requirements.txt- Load ePub to database
python app.py build --epub "Upgrading and Repairing PCs.epub"- Run qery
python app.py search --q "clear CMOS" --k 8 --format pretty