E-Judgment: Legal Research Assistant
Source-grounded AI research assistant over 8,200 Ghanaian court judgments
Overview
E-Judgment is a legal research assistant over about 8,200 Ghanaian judgments, built so that a lawyer can trust and check every statement it makes. An ingestion pipeline turns a messy export of 8,803 records into a verified corpus of 159,000 passages, each an exact span of its judgment and tied to its source, extraction method, licence and original GhaLII link. Search fuses exact citation lookup, Postgres full-text search and pgvector embedding search with Reciprocal Rank Fusion, reranks with a cross-encoder and groups results by case, so one long judgment can't crowd out the rest. Answers come from a local LLM (Ollama, or OpenAI when explicitly enabled) as structured claims, and the server keeps only the claims it can verify against the passages; when nothing survives, the system declines. Around this sits production work: invite-only accounts with Argon2id passwords and lockout, hashed audit logs with retention, a Next.js research workspace with a side-by-side evidence panel and streamed, cancellable answers, an in-app tool for lawyers to review test questions, reproducible evaluation runs, and a Docker Compose deployment with backups, restore drills and CI gates.
Context
Ghanaian judgments are published by GhaLII, but finding the authority for a legal issue or a set of facts still means trying keywords and reading judgments one by one. General AI chatbots answer fluently but can invent cases, quotes and page numbers, which makes them unusable for legal work.
Problem
The source data was messy: about 8,800 records from a local export of GhaLII, with PDFs, scans, Word files and placeholder pages saved under misleading names, unreliable file paths, duplicates published under several citations, and viewer boilerplate instead of text. Lawyers need two things a plain chatbot can't give: exact lookup by citation or case name, and answers they can check against the judgment itself.
Approach
Built a retrieval-augmented generation (RAG) system that treats evidence as the product. An offline pipeline verifies, cleans, OCRs and chunks every judgment while keeping its provenance. Search combines exact citation lookup, keyword search and meaning-based search, then reranks the results. Answers keep only the claims that quote a retrieved passage verbatim and that a local entailment model confirms the passage supports. Citations, links and page numbers come from the database, never from the model. It runs on local models by default, with the corpus used non-commercially under GhaLII's CC BY-NC 4.0 licence and GhaLII credited in every view.
Outcome
A working pilot behind invite-only sign-in: search by citation, case name, phrase or facts, and cited answers that stream with visible progress. On a 60-question test set (engineer-drafted, pending lawyer review, so provisional), hybrid search with reranking finds a correct case in the top 20 for 88% of the questions that have one (MRR@10 0.63), and citation and case-name lookups 100%. The system declines to answer when the evidence isn't there.
Challenges
Untrustworthy Source Files
The export saved every download as .pdf, but by file signature the files were PDFs, Word documents, scans and 553 copies of the same placeholder web page. File names were truncated citations, so some paths pointed at another judgment's file, and about 10% of judgments were re-published under different citations.
Exact Lookup and Meaning in One Search
Lawyers search both ways: by citation or case name, where embeddings can't reliably tell [2019] GHASC 12 from [2019] GHASC 21, and by describing a legal issue or facts, where keyword search found almost none of the right cases.
Answers That Can Be Trusted
Language models can invent cases, quotes and page numbers, and a quote that exists doesn't prove the claim built on it. Legal answers had to be checkable down to the passage, with page numbers only when they are certain.
Measuring Quality Honestly
Changes to search and generation needed numbers, not impressions, but the only test questions available at first were written by an engineer rather than a lawyer, so any numbers from them had to be clearly marked as provisional.
Solutions
Verified Ingestion with Provenance
Each source file is identified by its signature and hash and cross-checked against the record's text with character n-gram matching before it is used. Records that can't be trusted are quarantined with a reason code. Word files are converted and scans are OCR'd page by page with Tesseract, with confidence scores. Text is split into passages of 300–400 tokens, each an exact span of its source. Every stage is idempotent, so re-running it changes nothing.
Impact: Of 8,803 records, 8,242 judgments are searchable; 561 were quarantined rather than indexed (553 of them placeholder pages with no judgment text). 1,476 scanned pages were recovered by OCR.
Hybrid Retrieval with Case-Level Results
Exact citation and case-name lookup run first. Postgres full-text search and pgvector embeddings (bge-small) are fused with Reciprocal Rank Fusion and reranked by a cross-encoder. Results are grouped by case, and re-publications of the same judgment collapse into one result. Filters are strict and never silently widened, and a degraded search (e.g. a model missing) says so.
Impact: On the provisional test set, correct cases appear in the top 20 for 88% of answerable questions, against 39% for keyword search alone, and citation and case-name questions score 100%.
Verified, Server-Built Answers
The model returns structured claims, each with a quote and its sources. A claim is kept only if the quote appears verbatim in a cited passage, the claim adds no citation or page number of its own, and an NLI cross-encoder judges the passage to entail it. The server builds the answer from the surviving claims and adds citations, GhaLII links and page numbers from the database, the latter only for pages it has verified. Answers stream with live progress and can be cancelled.
Impact: Answers carry no model-written citations, links or page numbers; each statement links to the exact passage behind it, and the system declines to answer when no claim survives.
Reproducible Evaluation and Cost Control
Every evaluation run stores the models, settings and a fingerprint of the corpus, so results can be reproduced. Lawyers review test questions in the app, with versioned edits and history, and metrics are reported separately for approved questions. OpenAI is opt-in, with a usage ledger and a per-run budget stop, and a protocol is ready for comparing search quality with GhaLII's own site search.
Impact: A 20-question pilot on an OpenAI model cost under 1 cent, and every reported number is tied to a stored, reproducible run.