Keyword search works when you know the exact words in a document. Vector search helps when you know what you mean.
A question such as “Can I get my money back?” can find a passage about refunds—even when those exact words never appear.
In this tutorial, you’ll build a PDF search API that extracts text, generates embeddings locally, and returns relevant passages with filenames and page numbers. It is useful for searching manuals, policies, research papers, and personal document collections.
The initial setup downloads an embedding model. After that, indexing and searching can run offline.
What you’ll build
A local API with endpoints to:
· Upload and index PDFs. · Search using natural-language questions. · Return passages with source filenames, page numbers, and similarity scores. · List indexed documents. · Delete a document and its searchable chunks. · Preserve the index across restarts.
The processing pipeline is:
PDF → page text → overlapping chunks → local embeddings → SQLite
Question → local embedding → cosine similarity → ranked passages
We’ll use:
Component Purpose FastAPI Upload and search endpoints pypdf Extract selectable PDF text Sentence Transformers Generate embeddings locally NumPy Calculate vector similarity SQLite Persist text, metadata, and embeddings Docker Compose Run the service with persistent storage
This service retrieves passages. It does not generate answers.
Prerequisites
You’ll need:
· Python 3.12. · Basic Python and HTTP knowledge. · curl and a Bash-compatible terminal. · Docker with Docker Compose, if you want the container setup. · Approximately 4 GB of RAM and a few GB of free disk space. · Internet access for installing dependencies and downloading the model.
No GPU or paid API key is required.
The commands below target Linux or WSL. The default model is suitable for English documents. Multilingual collections may need a different embedding model.
Project structure
pdf-search/
├── app.py
├── requirements.txt
├── .env.example
├── .env
├── .gitignore
├── Dockerfile
├── compose.yaml
├── .dockerignore
├── data/ # Local index, created automatically
│ └── index.sqlite3
└── .cache/ # Local model cache, created automatically
Docker uses named volumes for the index and model cache instead of these local directories.
Step 1: Create the project and install dependencies
mkdir pdf-search
cd pdf-search
python3.12 -m venv .venv
source .venv/bin/activate
Create requirements.txt:
fastapi==0.143.1
uvicorn==0.54.0
python-multipart==0.0.32
python-dotenv==1.2.4
pypdf==6.20.1
numpy==2.5.4
sentence-transformers==6.1.0
transformers==5.19.0
torch==2.14.1
Install the CPU build of PyTorch first:
python -m pip install \
--index-url https://download.pytorch.org/whl/cpu \
torch==2.14.1
python -m pip install -r requirements.txt
Installing PyTorch from its CPU index avoids downloading unnecessary CUDA dependencies.
If you plan to run everything through Docker, you can skip the local dependency installation.
Step 2: Configure environment variables
Create .env.example:
MODEL_ID=sentence-transformers/all-MiniLM-L6-v2
MODEL_REVISION=1110a243fdf4706b3f48f1d95db1a4f5529b4d41
DATA_DIR=./data
HF_HOME=./.cache/huggingface
HF_HUB_OFFLINE=0
HF_HUB_DISABLE_TELEMETRY=1
TOKENIZERS_PARALLELISM=false
OMP_NUM_THREADS=2
CHUNK_TOKENS=180
CHUNK_OVERLAP=30
MAX_UPLOAD_MB=20
MAX_PAGES=250
MAX_TEXT_CHARS=2000000
MAX_DOCUMENT_CHUNKS=2000
MAX_INDEX_CHUNKS=20000
Copy it to .env:
cp .env.example .env
The model revision pins the exact model files. Both document chunks and search queries must use the same embedding model.
We’ll also record the model and chunking configuration inside SQLite. The application will refuse to open an index built with incompatible settings.
Create .gitignore:
.env
.venv/
.cache/
data/
__pycache__/
*.pdf
The index contains extracted document text. Treat it as part of your private document collection.
Step 3: Implement the API
Create app.py:
import hashlib
import json
import os
import re
import sqlite3
from contextlib import asynccontextmanager, contextmanager
from io import BytesIO
from pathlib import Path
from threading import Lock
from dotenv import load_dotenv
load_dotenv() # Load cache/offline settings before importing the model library.
import numpy as np
from fastapi import FastAPI, File, HTTPException, UploadFile
from pydantic import BaseModel, Field
from pypdf import PdfReader
from sentence_transformers import SentenceTransformer
MODEL_ID = os.environ["MODEL_ID"]
MODEL_REVISION = os.environ["MODEL_REVISION"]
DATA_DIR = Path(os.getenv("DATA_DIR", "./data"))
DB_PATH = DATA_DIR / "index.sqlite3"
CHUNK_TOKENS = int(os.getenv("CHUNK_TOKENS", "180"))
CHUNK_OVERLAP = int(os.getenv("CHUNK_OVERLAP", "30"))
MAX_BYTES = int(os.getenv("MAX_UPLOAD_MB", "20")) * 1024 * 1024
MAX_PAGES = int(os.getenv("MAX_PAGES", "250"))
MAX_TEXT_CHARS = int(os.getenv("MAX_TEXT_CHARS", "2000000"))
MAX_DOCUMENT_CHUNKS = int(os.getenv("MAX_DOCUMENT_CHUNKS", "2000"))
MAX_INDEX_CHUNKS = int(os.getenv("MAX_INDEX_CHUNKS", "20000"))
MODEL_LOCK = Lock()
@contextmanager
def database():
connection = sqlite3.connect(DB_PATH, timeout=30)
connection.row_factory = sqlite3.Row
connection.execute("PRAGMA foreign_keys = ON")
try:
with connection:
yield connection
finally:
connection.close()
@asynccontextmanager
async def lifespan(app: FastAPI):
if not 0 <= CHUNK_OVERLAP < CHUNK_TOKENS:
raise RuntimeError("Require 0 <= CHUNK_OVERLAP < CHUNK_TOKENS")
if min(
MAX_BYTES,
MAX_PAGES,
MAX_TEXT_CHARS,
MAX_DOCUMENT_CHUNKS,
MAX_INDEX_CHUNKS,
) <= 0:
raise RuntimeError("All resource limits must be positive")
DATA_DIR.mkdir(parents=True, exist_ok=True)
model = SentenceTransformer(
MODEL_ID,
revision=MODEL_REVISION,
device="cpu",
trust_remote_code=False,
local_files_only=os.getenv("HF_HUB_OFFLINE", "0") == "1",
)
token_budget = (
model.max_seq_length
- model.tokenizer.num_special_tokens_to_add()
)
if not model.tokenizer.is_fast or CHUNK_TOKENS > token_budget:
raise RuntimeError(
"Use a fast tokenizer and chunks within the model token budget"
)
app.state.model = model
app.state.token_budget = token_budget
fingerprint = json.dumps(
{
"model": MODEL_ID,
"revision": MODEL_REVISION,
"dimension": model.get_embedding_dimension(),
"chunk_tokens": CHUNK_TOKENS,
"overlap": CHUNK_OVERLAP,
"format": "page-offsets-v1",
},
sort_keys=True,
)
with database() as db:
db.execute("PRAGMA journal_mode = WAL")
db.executescript("""
CREATE TABLE IF NOT EXISTS settings (
key TEXT PRIMARY KEY,
value TEXT NOT NULL
);
CREATE TABLE IF NOT EXISTS documents (
id TEXT PRIMARY KEY,
filename TEXT NOT NULL,
pages INTEGER NOT NULL,
indexed_pages INTEGER NOT NULL,
chunks INTEGER NOT NULL,
created_at TEXT NOT NULL DEFAULT CURRENT_TIMESTAMP
);
CREATE TABLE IF NOT EXISTS chunks (
id INTEGER PRIMARY KEY,
document_id TEXT NOT NULL
REFERENCES documents(id) ON DELETE CASCADE,
page INTEGER NOT NULL,
text TEXT NOT NULL,
embedding BLOB NOT NULL
);
CREATE INDEX IF NOT EXISTS chunks_document
ON chunks(document_id);
""")
existing = db.execute(
"SELECT value FROM settings WHERE key = 'fingerprint'"
).fetchone()
if existing and existing["value"] != fingerprint:
raise RuntimeError(
"Index configuration changed: "
"use a new DATA_DIR and re-upload PDFs"
)
db.execute(
"INSERT OR IGNORE INTO settings VALUES ('fingerprint', ?)",
(fingerprint,),
)
yield
app = FastAPI(title="Local PDF Search", lifespan=lifespan)
def embed(texts: list[str]) -> np.ndarray:
# One process owns the model; serialize its inference calls.
with MODEL_LOCK:
return app.state.model.encode(
texts,
batch_size=32,
normalize_embeddings=True,
convert_to_numpy=True,
show_progress_bar=False,
).astype("<f4")
def extract_chunks(data: bytes):
tokenizer = app.state.model.tokenizer
chunks = []
skipped = []
characters = 0
try:
reader = PdfReader(BytesIO(data))
if reader.is_encrypted:
raise HTTPException(422, "Encrypted PDFs are not supported")
if len(reader.pages) > MAX_PAGES:
raise HTTPException(413, "PDF exceeds MAX_PAGES")
for page_number, page in enumerate(reader.pages, start=1):
text = re.sub(r"\s+", " ", page.extract_text() or "").strip()
characters += len(text)
if characters > MAX_TEXT_CHARS:
raise HTTPException(413, "PDF exceeds MAX_TEXT_CHARS")
if not text:
skipped.append(page_number)
continue
with MODEL_LOCK:
offsets = tokenizer(
text,
add_special_tokens=False,
return_offsets_mapping=True,
truncation=False,
verbose=False,
)["offset_mapping"]
step = CHUNK_TOKENS - CHUNK_OVERLAP
for start in range(0, len(offsets), step):
end = min(start + CHUNK_TOKENS, len(offsets))
chunk = text[offsets[start][0]:offsets[end - 1][1]]
# Substrings can tokenize differently at a word boundary.
with MODEL_LOCK:
tokens = tokenizer.encode(
chunk,
add_special_tokens=False,
verbose=False,
)
if len(tokens) > app.state.token_budget:
raise HTTPException(
422, "Reduce CHUNK_TOKENS and rebuild the index"
)
chunks.append((page_number, chunk))
if len(chunks) > MAX_DOCUMENT_CHUNKS:
raise HTTPException(
413, "PDF exceeds MAX_DOCUMENT_CHUNKS"
)
if end == len(offsets):
break
except HTTPException:
raise
except Exception as exc:
raise HTTPException(
422, "Cannot extract this PDF; check that it is readable"
) from exc
if not chunks:
raise HTTPException(
422, "No extractable text: run OCR first for scanned PDFs"
)
return len(reader.pages), skipped, chunks
@app.get("/health")
def health():
with database() as db:
count = db.execute("SELECT COUNT(*) FROM chunks").fetchone()[0]
return {"status": "ok", "model": MODEL_ID, "chunks": count}
@app.get("/documents")
def list_documents():
with database() as db:
rows = db.execute(
"SELECT * FROM documents ORDER BY created_at DESC, id"
).fetchall()
return {"documents": [dict(row) for row in rows]}
@app.post("/documents")
def upload_document(file: UploadFile = File(...)):
try:
data = file.file.read(MAX_BYTES + 1)
finally:
file.file.close()
if len(data) > MAX_BYTES:
raise HTTPException(413, "PDF exceeds MAX_UPLOAD_MB")
if b"%PDF-" not in data[:1024]:
raise HTTPException(422, "Upload a valid PDF")
document_id = hashlib.sha256(data).hexdigest()
filename = (
(file.filename or "")
.replace("\\", "/")
.split("/")[-1][:255]
or "document.pdf"
)
with database() as db:
existing = db.execute(
"SELECT * FROM documents WHERE id = ?", (document_id,)
).fetchone()
if existing:
return {"status": "existing", **dict(existing)}
pages, skipped, chunks = extract_chunks(data)
vectors = embed([text for _, text in chunks])
# Embedding happens outside the transaction to keep write locks short.
with database() as db:
db.execute("BEGIN IMMEDIATE")
existing = db.execute(
"SELECT * FROM documents WHERE id = ?", (document_id,)
).fetchone()
if existing:
return {"status": "existing", **dict(existing)}
count = db.execute("SELECT COUNT(*) FROM chunks").fetchone()[0]
if count + len(chunks) > MAX_INDEX_CHUNKS:
raise HTTPException(
413,
"Index is full; delete documents or raise MAX_INDEX_CHUNKS",
)
db.execute(
"""
INSERT INTO documents
(id, filename, pages, indexed_pages, chunks)
VALUES (?, ?, ?, ?, ?)
""",
(
document_id,
filename,
pages,
pages - len(skipped),
len(chunks),
),
)
db.executemany(
"""
INSERT INTO chunks (document_id, page, text, embedding)
VALUES (?, ?, ?, ?)
""",
[
(document_id, page, text, vector.tobytes())
for (page, text), vector in zip(chunks, vectors)
],
)
return {
"status": "indexed",
"id": document_id,
"filename": filename,
"pages": pages,
"indexed_pages": pages - len(skipped),
"chunks": len(chunks),
"skipped_pages": skipped,
}
class SearchRequest(BaseModel):
query: str = Field(min_length=1, max_length=1000)
top_k: int = Field(default=5, ge=1, le=20)
document_id: str | None = Field(
default=None, pattern=r"^[a-f0-9]{64}$"
)
@app.post("/search")
def search(request: SearchRequest):
query = request.query.strip()
if not query:
raise HTTPException(422, "Query must contain non-whitespace text")
with MODEL_LOCK:
tokens = app.state.model.tokenizer.encode(
query,
add_special_tokens=False,
verbose=False,
)
if len(tokens) > app.state.token_budget:
raise HTTPException(
422, "Query exceeds the model token budget; shorten it"
)
with database() as db:
if request.document_id and not db.execute(
"SELECT 1 FROM documents WHERE id = ?",
(request.document_id,),
).fetchone():
raise HTTPException(404, "Document not found")
sql = """
SELECT c.id, c.document_id, d.filename,
c.page, c.text, c.embedding
FROM chunks c
JOIN documents d ON d.id = c.document_id
"""
parameters = ()
if request.document_id:
sql += " WHERE c.document_id = ?"
parameters = (request.document_id,)
rows = db.execute(
sql + " ORDER BY c.id", parameters
).fetchall()
if not rows:
return {"query": query, "results": []}
query_vector = embed([query])[0]
matrix = np.stack(
[
np.frombuffer(row["embedding"], dtype="<f4")
for row in rows
]
)
# Both sides are normalized, so their dot product is cosine similarity.
scores = matrix @ query_vector
best = np.argsort(-scores, kind="stable")[:request.top_k]
return {
"query": query,
"results": [
{
"chunk_id": rows[int(i)]["id"],
"document_id": rows[int(i)]["document_id"],
"filename": rows[int(i)]["filename"],
"page": rows[int(i)]["page"],
"text": rows[int(i)]["text"],
"score": round(float(scores[i]), 4),
}
for i in best
],
}
@app.delete("/documents/{document_id}")
def delete_document(document_id: str):
with database() as db:
result = db.execute(
"DELETE FROM documents WHERE id = ?", (document_id,)
)
if result.rowcount == 0:
raise HTTPException(404, "Document not found")
return {"status": "deleted", "id": document_id}
How the indexing works
Each PDF page becomes one or more overlapping token windows. Using the model’s tokenizer keeps chunks within its input limit. Character-based chunks can exceed that limit unexpectedly, especially with technical vocabulary.
The tokenizer’s character offsets let us preserve the extracted text instead of reconstructing passages from token IDs.
Chunks stay within individual pages, so every result has an unambiguous page reference. These are physical PDF page numbers starting at one; they may differ from printed page labels.
How the search works
The model converts each chunk into a 384-dimensional vector. It converts the question into the same vector space.
Both vectors are normalized. Their dot product therefore gives cosine similarity:
scores = matrix @ query_vector
SQLite stores the vectors as float32 binary data alongside their source text.
This is an exact search over the stored vectors. It is straightforward and practical for modest collections, though each query reads and compares all candidate chunks.
How consistency works
A SHA-256 hash identifies each PDF by its contents. Uploading identical bytes twice returns the existing document. A modified PDF receives a new ID, so delete the previous version when replacing it.
Embedding happens before the write transaction. Inside the transaction, the application checks duplicates again and inserts the document and its chunks together. This also handles simultaneous uploads of the same PDF.
The shared model and tokenizer are protected by one lock. Run one Uvicorn worker: additional workers load additional model copies and do not share that lock.
Step 4: Run locally
From the project directory, with the virtual environment active:
uvicorn app:app \
--host 127.0.0.1 \
--port 8000 \
--workers 1
The first startup downloads the pinned model and stores it under HF_HOME. Subsequent starts reuse that cache.
In another terminal:
curl -fsS http://127.0.0.1:8000/health
Before indexing any documents, the response should be:
{
"status": "ok",
"model": "sentence-transformers/all-MiniLM-L6-v2",
"chunks": 0
}
Interactive API documentation is available at:
http://127.0.0.1:8000/docs
The API is intended for a single user on localhost. Add authentication and request limits before making it accessible remotely.
Step 5: Add Docker and Docker Compose
Create Dockerfile:
FROM python:3.12-slim
ENV PYTHONDONTWRITEBYTECODE=1 \
PYTHONUNBUFFERED=1 \
PIP_NO_CACHE_DIR=1
WORKDIR /app
COPY requirements.txt .
RUN python -m pip install \
--index-url https://download.pytorch.org/whl/cpu \
torch==2.14.1 \
&& python -m pip install -r requirements.txt
RUN groupadd --gid 10001 app \
&& useradd --uid 10001 --gid app --create-home app \
&& mkdir -p /app/data /app/model-cache \
&& chown -R app:app /app
COPY --chown=app:app app.py .
USER app
EXPOSE 8000
CMD ["uvicorn", "app:app", "--host", "0.0.0.0", "--port", "8000", "--workers", "1"]
The container runs as a non-root user. Uvicorn listens on the container’s network interface; Compose controls which host interface exposes it.
Create compose.yaml:
services:
api:
build: .
ports:
- "127.0.0.1:8000:8000"
env_file:
- .env
environment:
DATA_DIR: /app/data
HF_HOME: /app/model-cache
volumes:
- pdf-data:/app/data
- model-cache:/app/model-cache
healthcheck:
test:
- CMD
- python
- -c
- "import urllib.request; urllib.request.urlopen('http://127.0.0.1:8000/health', timeout=5)"
interval: 15s
timeout: 10s
retries: 5
start_period: 5m
volumes:
pdf-data:
model-cache:
Create .dockerignore:
.env
.venv
.cache
data
__pycache__
*.pdf
.git
Stop the local Uvicorn process with Ctrl+C before starting Docker on the same port.
docker compose config --quiet
docker compose up --build -d
docker compose logs -f api
Once startup completes:
curl -fsS http://127.0.0.1:8000/health
The named volumes preserve the index and downloaded model when containers are recreated. Local execution and Docker have separate indexes and model caches.
docker compose down retains those volumes. Adding --volumes removes them.
Step 6: Upload PDFs and test search
Use your own searchable PDF, or generate a small test document.
For the repeatable example below, activate your local virtual environment and install this optional PDF-generation dependency:
python -m pip install reportlab==4.4.9
Create handbook.pdf:
python - <<'PY'
from reportlab.pdfgen import canvas
pages = [
[
"Refund policy",
"Customers can request a full refund within 30 days of purchase.",
"After 30 days, refunds require approval from the support team.",
],
[
"Account security",
"Use a unique password and enable two-factor authentication.",
"Recovery codes should be stored somewhere safe.",
],
[], # A blank page demonstrates skipped-page reporting.
]
pdf = canvas.Canvas("handbook.pdf")
for lines in pages:
for index, line in enumerate(lines):
pdf.drawString(50, 780 - index * 24, line)
pdf.showPage()
pdf.save()
PY
ReportLab is only needed to generate this test file. It is not required by the API or Docker image.
Upload the PDF
curl -fsS \
-F "file=@handbook.pdf" \
http://127.0.0.1:8000/documents \
> upload.json
python -m json.tool upload.json
The response includes:
· status: indexed · pages: 3 · indexed_pages: 2 · chunks: 2 · skipped_pages: [3]
It also includes the document’s id and filename.
Save the ID for later commands:
DOCUMENT_ID=$(python -c \
'import json; print(json.load(open("upload.json"))["id"])')
Search with different wording
curl -fsS \
-X POST http://127.0.0.1:8000/search \
-H "Content-Type: application/json" \
-d '{
"query": "Can I get my money back after three weeks?",
"top_k": 2
}' \
| python -m json.tool
The first result should reference handbook.pdf, page 1, and include the refund policy.
The question says “money back” and “three weeks.” The document says “refund” and “30 days.” Embeddings allow the search to connect those meanings.
A result contains:
Field Meaning chunk_id Stored passage identifier document_id Source PDF identifier filename Original upload filename page Physical PDF page number text Retrieved passage score Cosine similarity to the question
Similarity scores rank relevance. They are not confidence percentages. A query unrelated to the collection can still return its nearest passages.
Search within one document
curl -fsS \
-X POST http://127.0.0.1:8000/search \
-H "Content-Type: application/json" \
-d "{
\"query\": \"How should I protect my account?\",
\"top_k\": 3,
\"document_id\": \"$DOCUMENT_ID\"
}" \
| python -m json.tool
The account-security passage on page 2 should rank first.
Verify duplicate detection
Upload the same file again:
curl -fsS \
-F "file=@handbook.pdf" \
http://127.0.0.1:8000/documents \
| python -m json.tool
The status should be existing, and /health should still report two chunks.
curl -fsS http://127.0.0.1:8000/health
List documents
curl -fsS http://127.0.0.1:8000/documents \
| python -m json.tool
Verify persistence
With Docker:
docker compose down
docker compose up -d
Once the API is ready, repeat the list and search requests. The document should still be available.
For local execution, stop and restart Uvicorn using the same .env and DATA_DIR.
Check input validation
An empty query should return HTTP 422:
curl -i \
-X POST http://127.0.0.1:8000/search \
-H "Content-Type: application/json" \
-d '{"query":" "}'
Requesting more than 20 results should also return 422:
curl -i \
-X POST http://127.0.0.1:8000/search \
-H "Content-Type: application/json" \
-d '{"query":"refund policy","top_k":100}'
Delete the document
curl -fsS \
-X DELETE \
"http://127.0.0.1:8000/documents/$DOCUMENT_ID"
SQLite’s foreign-key cascade removes the associated chunks. If this was your only document, searches now return an empty results array.
Step 7: Enable offline operation
After the model has downloaded successfully, change .env:
HF_HUB_OFFLINE=1
Restart the local Uvicorn process.
For Docker, recreate the container so it receives the updated environment:
docker compose up -d --force-recreate
The model must already exist in the cache used by that execution method. Warming the local cache does not populate Docker’s named volume.
Offline mode prevents model-hub access. PDF extraction, embedding generation, and similarity search already happen locally.
Uploaded originals are not retained permanently by the application. The index stores their extracted text, metadata, and embeddings; keep your original PDFs separately.
Design decisions
Decision Reason Tradeoff Local CPU embeddings Keeps document processing local and avoids API costs Indexing is slower than GPU inference SQLite for text and vectors One persistent database with atomic inserts and cascading deletion No approximate vector index Exact cosine search Simple, inspectable ranking without another service Each query scans all candidate vectors Page-local token chunks Prevents oversized inputs and provides precise page references Context cannot cross page boundaries Overlapping chunks Reduces information loss near chunk boundaries Some results may repeat similar text Content hashes Makes repeated uploads idempotent Changed PDFs become separate documents Configuration fingerprint Prevents mixing incompatible embeddings Model or chunk changes require rebuilding One worker and a shared lock Controls model memory and concurrent tokenizer access Embedding calls run sequentially Localhost binding Fits a private, single-user tool Remote access needs additional controls
With the default model, 20,000 float32 vectors occupy about 31 MB before text, database overhead, and model memory. Search requests allocate additional memory when loading those vectors.
Benchmark your actual collection before raising MAX_INDEX_CHUNKS.
Common issues and troubleshooting
“No extractable text”
The PDF may contain scanned page images. pypdf extracts existing text; it does not perform OCR.
Install OCRmyPDF and run:
ocrmypdf --skip-text original.pdf searchable.pdf
Then upload searchable.pdf.
For mixed documents, inspect skipped_pages. Successful indexing does not mean every page contained searchable text.
Search results are weak or repetitive
Inspect the returned text first. PDF extraction can produce incorrect reading order, repeated headers, or broken tables.
Useful improvements include:
· Removing recurring headers and footers. · Splitting around headings and paragraphs. · Combining semantic search with keyword search. · Using a model suited to the document language. · Adding a reranker.
Validate changes with representative questions and expected source pages.
“Index configuration changed”
The stored model, revision, dimension, or chunking settings differ from the current configuration.
Restore the original settings, or build a fresh index and upload the PDFs again.
For local execution, choose a new DATA_DIR. For Docker, use a new data-volume name in Compose and its volumes declaration. Compose overrides the local DATA_DIR value.
Keep the previous index until the replacement works.
Offline startup fails
The model cache is missing or incomplete.
Set HF_HUB_OFFLINE=0, restart with internet access, wait for successful startup, then enable offline mode again.
Check that you are using the same model cache and pinned revision.
HTTP 413
A configured resource limit was exceeded.
The response identifies the relevant limit. Split unusually large PDFs or deliberately raise the appropriate environment variable.
MAX_UPLOAD_MB limits what the application processes after the multipart upload has been received and spooled. A public deployment also needs request-body limits at its proxy. PDF parsing itself can consume substantial resources, so isolate parsing if you accept untrusted uploads.
Port 8000 is already in use
Stop the local server before starting Docker.
Alternatively, change the Compose mapping:
ports:
- "127.0.0.1:8001:8000"
Then send requests to port 8001.
The first Docker startup is slow
The image installs PyTorch, and the first container startup downloads the model.
Check progress with:
docker compose logs -f api
docker compose ps
The cache volume prevents repeated downloads. Increase the healthcheck’s start_period if your connection needs more time.
Indexing blocks other embedding requests
The model lock intentionally serializes inference.
For larger workloads, move indexing into a background job queue and return a job ID. Keep database transactions short, as this implementation does.
Backups miss recent changes
SQLite uses WAL mode, which can create companion files beside the database.
For a simple file-based backup, stop the service before copying the complete data directory or Docker data volume. For live backups, use SQLite’s backup API.
What you learned
You built a persistent PDF search service and learned how to:
· Extract text while preserving page references. · Split passages using the embedding model’s token budget. · Generate embeddings without sending document text to a cloud API. · Rank passages using normalized vectors and cosine similarity. · Keep uploads consistent with transactions and content hashes. · Run a non-root Docker service with persistent storage and offline model loading.
The implementation was checked with semantic queries, concurrent duplicate uploads, long-page chunking, invalid PDFs, deletion, offline restarts, and Docker persistence.
Possible next steps
· Add a search interface. Show excerpts, filenames, and page references in a small web application. · Add hybrid retrieval. Combine SQLite FTS5 keyword search with vector ranking for names, codes, and exact terminology. · Improve result selection. Add a reranker and evaluate relevance thresholds against a labeled question set. · Support more languages. Choose multilingual embeddings and rebuild the index. · Move to a vector index when needed. Benchmark FAISS or Qdrant when full scans become too slow. · Add background indexing. Track upload jobs and isolate PDF parsing for larger or less trusted collections. · Connect a local language model. Use retrieved passages as context, preserve citations, and handle questions unsupported by the documents.