I work on production AI systems — agentic workflows, retrieval, document intelligence and the engineering around making them reliable.
I have architected and shipped AI products built to hold up at scale: assistants used by thousands of people every day, retrieval over tens of thousands of documents, pipelines serving hundreds of clients.
01, Currently
Senior AI Engineer
Stealth product startup
Structured intelligence from complex, inconsistent technical documents. Understand the document type, find the relevant pages and sections, extract the tables and attributes, keep the relationships between them, validate, and only then emit output the next system can trust.
Agents treated as engineered systems. MCP tool calling, harnesses, reusable skills, subflows and stateful orchestration in the LangGraph style, with validation loops, structured outputs, human-in-the-loop checkpoints and model routing designed in from the start.
Evaluate before standardizing. Models are compared on accuracy, task reliability, structured-output quality, latency, cost, context handling, failure patterns and consistency. Extraction tools are compared on real documents. Architectures are compared on what they cost to run, observe and change.
Research-heavy work that took weeks or months, finished in days. Decompose the task, run AI-assisted research and extraction, keep deterministic steps deterministic, put a person at the checkpoints that matter, and evaluate every stage on its own.
02, Systems I've built
Documents flow through classification, relevant-content detection, extraction, structure reconstruction, validation and evaluation into trusted structured output.
Complex technical documents in, trusted structured output out. Every stage is validated and evaluated on its own.
One assistant for the workforce of a global shipping and logistics group: document Q&A, tool use and multimodal features, with knowledge sources that can be added without rebuilding the agent.
7,000+
A request enters a LangGraph agent that calls MCP tools and retrieves from Azure AI Search before Azure OpenAI writes the answer; every step is traced in Databricks.
Cross-cutting: tracing · Databricks.
Ingestion, structured summaries and question answering over a corpus of audit documents, inside a large professional-services firm's tax and audit platform.
40,000+
Audit documents are ingested into an index; retrieval feeds structured summaries and question answering, with tracing across the pipeline.
Cross-cutting: tracing.
Scheduled company-analysis reports for hundreds of enterprise clients: scraping and ETL in, LLM-written analysis out.
600+
Public sources are scraped and transformed, embedded, orchestrated with Databricks Jobs into generated reports, and served through Azure Functions with a Redis cache.
Cross-cutting: cache · Redis.
A RAG assistant for an education information platform (colleges, exams, admissions, careers) over data that changes constantly, fed by a scheduled ETL pipeline.
10,000+
Sources are extracted by Airbyte, scheduled by Airflow and transformed with DBT into an index that a LangGraph RAG flow answers from.
Extraction, retrieval and summarization over incoming insurance documents and medical records, processed in parallel by fault-tolerant workers.
Documents pass through Azure Document Intelligence and extraction into PGVector, then summarization, with Celery workers on RabbitMQ running the stages in parallel.
Cross-cutting: workers · Celery / RabbitMQ.
03, Engineering AI systems
A model is one component. Production AI usually fails at the boundaries around it.
Every component compared on its own dimensions, on real inputs, before it becomes the standard. It touches every other concern.
Where state lives, which steps are deterministic, and where a model is allowed to decide. Most later problems are decided here.
What the model sees decides what it can answer. Hybrid retrieval, reranking and context assembly are engineering problems with measurable outcomes.
Validation loops, structured outputs, retries and human checkpoints. The system should fail visibly and recover on purpose.
Observability, cost, latency and maintainability over time. A demo has none of these constraints; a system has all of them.
04, Evaluate, then choose
The best model or tool is not universally the best one. It is the one that performs best for the constraints of the specific system.
Nothing becomes the standard in a system before it is compared with the alternatives on that system's own inputs. The dimensions below are what the comparison covers.
| Dimension | What I measure | Why it matters in production |
|---|---|---|
| Accuracy | Correctness on a held-out set of the system's real tasks, scored against reference outputs or rubrics. | The headline dimension, but only meaningful on this system's inputs, not on a public benchmark. |
| Task reliability | How often the model completes the whole task as specified, across repeated runs and input variants. | A model that is right most of the time and wrong unpredictably is a bug, not a feature. |
| Structured-output quality | Schema adherence, field completeness and type correctness of typed outputs, before any repair step. | Downstream code consumes the output. Every repair step is latency, cost and a hidden failure mode. |
| Latency | Time to first token and to completion at the prompt sizes the system actually sends, median and tail. | Decides whether a step can be synchronous and how many steps a workflow can afford. |
| Cost | Cost per completed task at production volumes, including retries and validation reruns. | Per-call price is misleading. Per-successful-task cost is what the budget sees. |
| Context handling | Quality at the context lengths the system needs, including retrieval-heavy prompts and long documents. | Many models degrade quietly as context grows. The degradation has to be measured, not assumed. |
| Failure patterns | What goes wrong: invented fields, refusals, truncation, format drift, and how detectable each failure is. | Detectable failures can be caught by validation. Silent ones reach users. |
| Consistency | Variance across repeated runs of the same input and across paraphrased inputs. | Users and downstream systems need the same answer to the same question. |
| Dimension | What I measure | Why it matters in production |
|---|---|---|
| Recall | Whether the passages needed to answer are in the retrieved set, checked against a labeled query set from real usage. | If the right context is not retrieved, no model can answer correctly. |
| Precision | How much of the retrieved set is actually relevant at the context budget the system uses. | Irrelevant passages cost tokens and pull answers off course. |
| Reranking quality | Change in the position of relevant passages, and in answer quality, with and without a reranker at the same context budget. | Reranking costs latency. It has to earn its place with a measured gain. |
| Query behavior | Keyword, vector and hybrid retrieval on the same queries, especially identifiers, codes, rare terms and paraphrases. | Semantic search misses exact terms; keyword search misses meaning. The right mix is system-specific. |
| Latency | End-to-end retrieval time including embedding, search and reranking, at production index size. | Retrieval sits on the critical path of every answer. |
| Failure behaviour | What the system does when nothing relevant exists: abstain, or answer anyway. | An assistant that answers from nothing is worse than one that says it cannot. |
| Dimension | What I measure | Why it matters in production |
|---|---|---|
| Table fidelity | Cell-level accuracy on real tables from the corpus: merged cells, multi-page tables, rotated layouts, footnotes. | Tables carry the numbers. A shifted column is a wrong answer that looks right. |
| Structure preservation | Whether headings, sections, reading order and the relationships between elements survive extraction. | Downstream steps depend on knowing which attribute belongs to which section. |
| Text accuracy | Character- and field-level accuracy on the text the corpus actually contains, including scanned pages. | Extraction errors propagate; nothing downstream can correct a misread value. |
| Edge cases | Behaviour on scans, low resolution, unusual layouts, mixed languages and very long documents. | Production corpora are edge cases. The clean sample is the exception. |
| Latency | Time per page and per document at production volume, including any OCR step. | Decides batch sizing, queue design and what can run interactively. |
| Cost | Cost per page at production volume, including reprocessing of failures. | Document volumes are large. Per-page cost compounds fast. |
| Downstream usability | How much post-processing the output needs before validation can consume it. | A tool that is accurate but needs a parser written around it has moved the work, not removed it. |
| Dimension | What I measure | Why it matters in production |
|---|---|---|
| End-to-end success | Output quality on real tasks, judged after every stage has run, not per component. | Components that score well alone can compose into a system that does not. |
| Complexity | Number of moving parts, model calls, branches and stateful steps needed to complete a task. | Every part is something to observe, test and explain. Complexity is paid for monthly. |
| Observability | Whether every step's inputs, outputs, latency and cost can be traced for a single task after the fact. | You cannot fix what you cannot see. Debugging a black box in production is guesswork. |
| Failure recovery | What happens on a failed validation, a tool timeout or a model outage: retry, fallback, human checkpoint, or crash. | Production is defined by how the system behaves on its worst day. |
| Maintainability | How hard it is to change a model, a prompt, a tool or a skill without touching the rest. | Models change every quarter. The architecture has to outlive the model choice. |
| Latency | End-to-end time per task at production load, median and tail. | Users and upstream systems wait on the whole chain, not on one model call. |
| Cost | Total cost per completed task including retries, evaluation runs and infrastructure, at expected volume. | The architecture decides the cost profile more than the model does. |
| Time-to-production | How long from working prototype to a monitored, validated system users can rely on. | A demo takes a day. Closing the gap to a system people rely on is the actual project. |
05, An agentic workflow, end to end
The shape most of my agentic systems take. Hover or focus a stage for one sentence on what it does and where it fails.
Cross-cutting:observabilityevaluationpersistence
Input
A request, a document batch or an event. Normalized and typed before anything downstream sees it.
Router
Classifies the request and picks a path and a model. A cheap model with a strict schema; a wrong route is caught here, not five steps later.
Agent State
The typed state every step reads and writes. Persisted, so a run can pause, resume or be replayed.
Planner
Decomposes the task into steps and assigns skills. Re-plans when validation sends work back.
Specialized Skills
Reusable, tested units of work: extract this table, verify that field. Deterministic where possible, model-backed where necessary.
MCP / Tools
Typed tool calls over MCP: search, databases, internal APIs. The boundary where the agent touches the real world.
Retrieval
Hybrid retrieval and reranking assemble the context the model is allowed to see. Measured separately from generation.
Model
A swappable component. Chosen per task by evaluation, routed by the Router, never the whole system.
Validation
Schema checks, cross-field consistency, business rules. Failures return to the Planner with the reason attached.
Human Approval
A checkpoint where a person confirms, corrects or rejects. The run pauses; state waits.
Output
Structured, validated output with provenance: which sources, which model, which checks passed.
06, Technical landscape
What the systems above are built with. Emphasis marks what I reach for first.
LangGraphLangChainMCP (Model Context Protocol)Agentic orchestrationAgent harnesses and skillsHuman-in-the-loop workflowsStructured outputsModel routingLLM evaluation
OpenAIAzure OpenAIGeminiClaudeLlamaCohere
RAGHybrid searchRerankingAzure AI SearchPGVectorDatabricks Vector SearchChroma
PythonFastAPIDjangoCeleryRabbitMQRedisPostgreSQLAzure Cosmos DB
DatabricksDatabricks Jobs and Asset BundlesAirflowDBTAirbyteScrapyETL pipelines
AzureAzure FunctionsAzure Container AppsAzure Document IntelligenceAWSGitHub ActionsAzure DevOpsLangFuseLangSmithDatabricks Traces
07, Experience
Stealth product startup
Structured document extraction, agentic workflows, evaluation.
Coditas
4+ years. Enterprise assistants, RAG at scale, data pipelines, document processing.
08, About
I'm Sanskar, a Senior AI Engineer in Pune, India. I build AI systems that have to keep working after the demo: retrieval, agents, document extraction, and the evaluation and operations around them.
The first AI demo is easy. Making it reliable isn't. Most of my work is the second part: deciding what a model is allowed to decide, validating what it produces, measuring every component before it becomes the standard, and keeping the whole system observable once real documents and real users arrive.
I came to AI systems from backend and data engineering. That is probably why I care more about the boundaries around a model than about the model.
09, Elsewhere
Interesting engineering problems are always worth a conversation.