Multimodal Analyst

PDF & Image Q&A

A full-stack Next.js app backed by a Multi-modal RAG pipeline. It extracts both text and images from documents, encodes them, and answers complex structural queries using a Vision LLM. Ideal for analyzing reports, flowcharts, and slide decks.

Download ZIP

Live previewinteractive mock — no backend

localhost:3000Next.js

Vision Analyst

Inspect files visually.

Stack

Frontend

Next.js

Architecture

Multi-modal RAG

Framework

raw python

Vector DB

qdrant

Corpus

medium

Complexity

High

How it works

Browser→

Next.js /api/chat (SSE)→

FastAPI /query→

Multi-modal RAG

The frontend talks only to the backend; your API keys + pipeline URL stay server-side.

Get started

Option A · Docker Compose (one command)

cp .env.example .env        # add your API key(s)
docker compose up --build

Option B · Backend (manual)

cd backend
python -m venv .venv && source .venv/bin/activate
pip install -r requirements.txt
cp .env.example .env        # add your API key(s)
python pipeline.py ingest ./corpus
uvicorn serve:app --reload  # http://localhost:8000

Option B · Frontend (Next.js)

cd frontend
npm install
cp .env.local.example .env.local
npm run dev                 # http://localhost:3000

The downloaded README.md has the full guide — vector DB setup, API keys, and deployment to Render/Railway + Vercel.

Project structure25 backend · 14 frontend · 43 files total

backend/

.github/

workflows/

deploy.yml

eval/

__init__.py

README.md

run_eval.sh

synthetic_qa.py

test_answer_quality.py

test_retrieval.py

ingestion/

__init__.py

chunker.py

embedder.py

image_extractor.py

loader.py

observability/

__init__.py

tracing.py

retrieval/

__init__.py

retriever.py

.env.example

config.yaml

docker-compose.yml

Dockerfile

pipeline.py

README.md

render.yaml

requirements.txt

serve.py

frontend/

app/

api/

chat/

route.ts

globals.css

layout.tsx

page.tsx

components/

Chat.tsx

ThemeToggle.tsx

.env.local.example

.gitignore

Dockerfile

next.config.mjs

package.json

postcss.config.js

tailwind.config.ts

tsconfig.json

.env.example

.gitignore

docker-compose.yml

README.md

Ready to build?

Download the full monorepo and follow the README.

Download ZIP