EvidenceDesk
RAG support copilot with human review
Turns a support ticket into a source-backed draft note that a person reviews before anything is saved. It shows how retrieval, structured AI output and human approval fit together in a complete product.
- Role
- Independent · sole engineer
- Status
- Open source (MIT) · live demo
- Built
- 2026
- Stack
- Python, FastAPI, LangGraph, PostgreSQL + pgvector, Next.js
The problem
Support agents lose time searching documentation, and a generic chatbot answer is hard to trust. EvidenceDesk drafts a reply note grounded in retrieved documentation, shows which sources support each claim and leaves the final decision to a person.
Architecture
Retrieve
PostgreSQL full-text search and pgvector similarity search over the documentation.
Fuse
Reciprocal rank fusion merges both result lists into one ranked context.
Draft
A bounded LangGraph workflow produces a structured note, with Cloudflare Workers AI for embeddings and generation.
Verify
Citation and quotation checks tie generated claims to the retrieved sources.
Review
A person approves or rejects the proposal in the Next.js and TypeScript interface.
Save
Approval stores the exact reviewed proposal in one database transaction; retry handling prevents duplicate notes.
Guardrails
- Model output can never write to the database directly. Only an explicit human approval can.
- Session and tenant authorization restrict access to records.
- Retries are designed to be idempotent, so a repeated request cannot create a second note.
Testing and evaluation
- Automated API, database, browser and failure-scenario tests run in CI/CD.
- Retrieval evaluation is published separately from answer-quality assessment.
- Limitations are documented: the demo uses fictional support cases, and human review of semantic answer quality is still pending.