CompliSense

Problem

A small business in India is subject to more regulation than it can read. The rules live in long documents, they change, and missing one is expensive. Large companies pay a compliance team. Smaller ones guess.

This is my dissertation project, built with Parth and supervised by Dr. Ashish Joshi.

Approach

Three pieces, each doing one job.

Named-entity recognition, or NER, means pulling the specific things out of a document: which entity, which obligation, which deadline. A BERT model does that, and reaches 84.8% macro F1.

XGBoost scores risk, and SHAP explains the score. SHAP attributes a prediction to the individual inputs that drove it, which matters here because “you are non-compliant” is not an acceptable answer on its own. A business needs to know which clause caused it.

Regulations reference each other, so the relationships go in Neo4j, a graph database, while the documents and their embeddings go in PostgreSQL 16 with pgvector for similarity search.

System

regulation documents  ->  BERT NER  ->  entities, obligations, deadlines
                                              |
                          Postgres 16 + pgvector (documents, embeddings)
                                              |
                              Neo4j 5 (which rule references which)
                                              |
                              XGBoost risk score  ->  SHAP explanation

Results

84.8% macro F1 on the entity types. Macro, not weighted, on purpose: it averages the classes equally, so a rare obligation type counts as much as a common one. A weighted number would have looked better and meant less.

The whole platform runs at zero hosting cost on free tiers: Supabase, Render, Vercel and AuraDB. For the businesses this is aimed at, running cost is part of whether the thing is usable at all.

Limits, and what I would do next

An 84.8% macro F1 means roughly one in six entity decisions is wrong somewhere in the mix. This is a tool for a person reviewing compliance, not a replacement for one, and the interface should keep saying so.

The regulation graph is only as current as the last time someone loaded the documents. Regulation changes, and a stale graph is confidently wrong rather than obviously empty.