SELECTED
WORK.
AGENTIC AI.2025 — NOWOPEN ↗F · 02PDF → FORM
IN 20s.2025OPEN ↗F · 03COURSE
BUILDER.2025OPEN ↗F · 04VOICER —
SIGN → SPEECH.2023OPEN ↗F · 05AIR-WRITING
RECOGNITION.2024OPEN ↗F · 06ON-PREM OCR
+ LLM.2024OPEN ↗F · 07RAG CHATBOT
AT SCALE.2024 — 2025OPEN ↗F · 08WEEPSCOPE —
CRY CLASSIFIER.2023OPEN ↗F · 09SARANSH —
BOOK → AUDIO.2023OPEN ↗F · 10TEXT → SQL
+ DASHBOARDS.2024 — 2025OPEN ↗
PERMISSION-AWARE
AGENTIC AI.
Not a chatbot — a LangGraph agent that runs the whole EHS platform. Intent detection routes each request to RAG search, text-to-SQL, or one of 25+ live tool APIs, and every action is permission-checked first. 500+ concurrent users. No caching possible.
— THE PROBLEM
ONE AGENT.
EVERY TOOL.
EVERY ACTION
PERMISSION-
CHECKED.
Each user has a unique profile drawn from 250+ granular permissions and any subset of 750+ courses. The assistant has to do far more than answer questions: read live data, run reports, create and edit records — but only what that user is allowed to touch. Traditional caching is impossible, and a static index cannot model this.
The system is a LangGraph state machine over LangChain tool-calling agents. Intent detection classifies each turn, then routes it to the right capability: vector RAG search, Postgres text-to-SQL, one of ~10-15 filtered data-fetch APIs, or one of ~10-15 CRUD APIs (create user, update record, and so on). A permission-filtering layer runs before any tool fires.
- Intent + routing. Every turn is classified, then dispatched to RAG, text-to-SQL, a filtered fetch API, or a CRUD action — wired together as an explicit LangGraph graph.
- Permission gate. A Milvus permission index (top-30) cross-referenced against PostgreSQL lets GPT-4o decide full access, none, or partial (compound requests split) — re-validated every turn.
- 25+ tools. The platform's data-fetch and CRUD endpoints are exposed as LangChain tools, so the agent fetches live data and triggers workflows end-to-end without ever exceeding the user's rights.
PDF → FORM
IN 20s.
Three LLMs in a single user flow — each chosen empirically for what it’s best at, glued together by a Pydantic-validated intermediate schema.
— THE DESIGN MOVE
BUILD AN
INTERMEDIATE
JSON THAT
LLMs CAN WRITE
RELIABLY.
The platform’s native form schema was too deeply nested for any single LLM to emit in one pass — wrapper objects, type-specific aliases, radio-driven conditional containers nested arbitrarily.
The fix was a simplified intermediate JSON schema, validated by Pydantic, plus a deterministic conversion layer to the native form. That decoupled LLM understanding from structural correctness.
- PDF converter. GPT-4o reads PDF → intermediate JSON → Pydantic → native. ~20s, ~7K tokens. Bulk: 100 PDFs in 5-10 min.
- Conversational builder. GPT-4o suggests title/type; Qwen3-Coder writes blueprint; user iterates; GPT-4o-mini converts.
- Model rationale. Each picked from head-to-head testing — not vibes.
COURSE
BUILDER.
Search 10,000 clips, let Gemini direct, narrate with ElevenLabs, render on serverless. End-to-end training course in one pass.
— THE PIPELINE
RAG, BUT THE
CHUNKS ARE
10-SECOND
VIDEO CLIPS.
Source videos cut into 10-second clips with FFmpeg, each analyzed by Gemini 2.5 Pro for visual content, embedded with Cohere embed-english-v3, indexed in Milvus.
At generation time the user uploads a course doc; AI returns an outline, then in parallel: RAG retrieves 80-100s of candidate clips per ~60s lesson, Cohere reranks, Gemini selects + orders + writes narration in a single inference.
Heavy rendering — cut, merge, overlay, audio mix, transitions — offloaded to AWS Lambda with FFmpeg, so the EC2 app server never feels it.
VOICER —
SIGN → SPEECH.
Ten IMU rings, one wrist controller, a 12-class neural net quantized to fit in 2 KB of RAM — inferring in 2 ms, offline. Samsung Top 10 of 70,000. Featured on CNN News18.
— THE CONSTRAINT
A 12-CLASS
NEURAL NET
HAS TO FIT
IN A CALCULATOR.
Indian Sign Language. Ten IMUs (one per finger) over I²C into a NodeMCU, sampled at 60 Hz, windowed to 2 seconds — 7,200 raw values per gesture. Spectral feature extraction compresses to 234.
A progressively narrowing Dense network (128→64→32→16) quantized to INT8, exported via Edge Impulse, fits the NodeMCU’s tight budget. Bluetooth sends the predicted label to a phone for audio output.
- Custom dataset: 10 participants × 12 classes × 20 repetitions = 2,400 samples.
- Team learned ISL formally so gestures matched real-world usage.
- National Top 10 of 70,000 · Samsung Solve for Tomorrow · CNN News18.
AIR-WRITING
RECOGNITION.
A wearable air-writing system — 62 output classes from a single wrist-mounted IMU — and the discipline to evaluate it honestly.
— THE WORK
SWEEP
A THOUSAND
MODELS.
REPORT THE
HONEST DELTA.
ESP32 + MPU6050 wristband at 60 Hz over Wi-Fi (chose Wi-Fi over Bluetooth to escape Linux-only library lock-in). 20 participants × 62 classes × 10 repetitions = 12,400 labeled samples. Z-score normalized per channel.
Swept Dense / 1D-CNN / LSTM / BiLSTM / hybrids across 1,000+ configurations. Final: CNN-BiLSTM. Reported user-dependent and LOSO-CV user-independent numbers separately — the 18-point gap is the truth.
ON-PREM OCR
+ LLM.
Aadhaar, PAN, GST — data could not leave the client’s 250 GB server. Hybrid OCR + on-prem LLM with zero external API calls.
— THE BRIEF
DATA CAN’T
LEAVE THE
SERVER.
BUILD IT
ANYWAY.
Hybrid OCR combining DocLink (character accuracy) and DocTR (structure), merged by Llama-3.1-Nemotron-70b hosted on-prem via vLLM. Entity extraction layered on top with markdown / JSON output.
The hard constraint: a dedicated 250 GB on-prem server, every model self-hosted, zero external API calls — Aadhaar, PAN and GST numbers never leave the building. Entity extraction held ~90% accuracy across documents of varying scan quality, emitted as markdown or JSON.
RAG CHATBOT
AT SCALE.
A production RAG platform serving thousands of clients — 50K+ queries a month across 10+ chatbot instances. Inherited a 45% MRR stack and rebuilt it into a 95% one.
— THE REBUILD
A 45% MRR
STACK ISN’T
PRODUCTION.
REBUILD
IT.
Inherited a RAG platform with Llama-2-7b loaded inline in the app and FAISS as the vector store — 45% MRR and a hard latency bottleneck. Re-architected it: the LLM became a standalone vLLM-served API (independent scaling, model swaps without redeploy), and the store moved to Milvus after a head-to-head benchmark on production data.
Benchmarked 6+ embedding models (BGE small/base/large, Stella, SPLADE) and settled on BGE-M3 + a reranker with custom query preprocessing and intent identification. Net result: MRR 45% → 95% and latency 16s → 8s.
- Benchmark. Milvus 95% vs Qdrant 85% vs FAISS 45% MRR on production-scale data (millions of vectors) — Milvus won.
- Serving. Llama-2-7b pulled out of app code into a vLLM API for independent scaling and hot model swaps.
- Quality. BGE-M3 + reranker, intent identification, query preprocessing — latency halved, MRR doubled.
WEEPSCOPE —
CRY CLASSIFIER.
A standalone Arduino device that listens to an infant and names the cry — five Dunstan cry types, on-device, with no phone and no internet. The work became a co-authored research paper.
— THE CONSTRAINT
CLASSIFY A
CRY ON A
MICRO-
CONTROLLER.
NO CLOUD.
Dunstan Baby Language describes five characteristic newborn cries — hunger, discomfort, sleepiness, lower gas, burp — that untrained parents struggle to tell apart. The device captures 2 seconds of audio at 16 kHz from an onboard PDM microphone, extracts MFE (Mel-filterbank energy) features, and runs a 1D-CNN on an Arduino Nano 33 BLE — the result shows on an OLED, no phone required.
No labelled dataset existed, so it was built by hand: 3,615 samples (2+ hours) curated against reference cries. MFE beat MFCC and spectrogram features in testing. The full 1D-CNN + GRU model (88.7%) was too large for the board, so it was compressed to a CNN that fits in 168 KB flash, with 5-cycle majority voting recovering practical accuracy.
- Feature search. MFE vs MFCC vs spectrogram, compared empirically — MFE won on accuracy.
- Fit the board. 1D-CNN + GRU (88.7%) compressed to a CNN-only model to live in 168 KB flash, ~1.2 s inference.
- Majority voting. Five inference cycles per decision to suppress false identifications.
SARANSH —
BOOK → AUDIO.
Point it at a printed page; it scans, summarizes, and reads the gist aloud — in multiple Indian languages, fully self-contained on a Raspberry Pi. 2nd of 200+ teams at Ingenium.
— THE BUILD
TURN A
PRINTED PAGE
INTO A
SPOKEN
SUMMARY.
Students and visually-impaired readers need an accessible way to get the gist of physical books, and most tools are either expensive or skip regional Indian languages. Saransh runs the whole pipeline on a Raspberry Pi: a camera captures the page, OpenCV handles preprocessing and page detection, and Tesseract OCR extracts the text.
A Transformers NLP stage condenses the text into a concise summary, then text-to-speech reads it aloud — with the TTS engine chosen per language after quality varied across fonts and Indian languages. The device is self-contained and portable.
TEXT → SQL
+ DASHBOARDS.
Ask a business question in plain English; get back the SQL, the answer, and two charts — over 100+ SAP tables with cryptic column names, in about 18 seconds.
— THE PIPELINE
SAP COLUMNS
LIKE VBELN.
USERS ASK
IN ENGLISH.
Business analysts waited hours-to-days for SAP developers to pull reports from a database of 100+ tables with cryptic column names (VBELN, KUNNR, AUART). The goal: a conversational bot that answers with a visualisation in under 30 seconds. Each schema — DDL plus five sample rows — is indexed offline into Qdrant with BGE-M3.
At query time a 5-stage pipeline runs: the LLM expands the question into schema-aware searches, a self-refinement step validates and augments the retrieved table set, a dedicated call generates SQL against the refined DDLs, and execution feeds Pandas preprocessing plus two auto-generated chart specs.
- Query expansion. SAP names never appear in plain English — the LLM expands each question into 3-5 schema-aware searches.
- Split the work. Table selection and SQL generation in one prompt degraded accuracy; two focused calls fixed it.
- Tame big results. Pandas summaries + top-50 rows beat dumping everything into context; exactly two charts keep the JSON parseable.