A model that scores 99 percent usually cheated.

Most of data science is not the model. It is the grind before it, loading a messy file, fixing how it is represented, charting it, deciding what kind of problem it even is, choosing an algorithm, and checking the result is real. Everyone wants to build the model. Almost nobody wants the grind. So I built a tool that does the grind, and explains the answer to someone who is not an ML engineer.

AutoDS Copilot takes a spreadsheet and walks the entire workflow, pre-clean, explore, quality checks, task detection, cleaning, a model leaderboard, training, evaluation, tuning, a written analysis, an interactive dashboard, and prediction on new data. It runs locally, and it needs no API key. But automating the steps was the easy part. The part worth writing about is trust.

One engine, three front doors

All of the machine-learning logic lives in one place, a modular scikit-learn pipeline where every stage is a plain, testable function. Three thin interfaces drive that same engine. A web console you click through. A command-line agent that runs the pipeline as cooperating steps. And a set of tools any AI client can call. Nothing is duplicated, so the same capability shows up in a browser, a terminal, and an AI assistant without writing it three times.

That separation is the spine of the project. The interfaces are thin, the engine is where the care goes.

A high score is a suspect, not a trophy

Here is the failure mode that ruins real projects. A model scores ninety-nine percent, everyone celebrates, and it turns out a column in the data quietly contained the answer, or an ID column let the model memorise rows instead of learning. That is data leakage, and it is the single most common way a machine-learning result is a lie.

AutoDS is built against it on two fronts. First, correctness by construction, every bit of preprocessing, the imputing, scaling, and encoding, lives inside a scikit-learn Pipeline that is fit only on the training split, never on the whole dataset, so information from the test set cannot leak backward into training. Second, it audits the data before it models, flagging columns that move almost perfectly with the target, constant columns, ID-like columns, and duplicates, and it pre-selects the risky ones to drop.

Specialised steps, and a human at the gates

The command-line version is a LangGraph state machine. Each stage is a node, a loader, a profiler, an EDA step, a task detector, a cleaner, a model recommender, a trainer, an evaluator, and an insight writer, with a typed state threaded through all of them. At two points, cleaning and model choice, the graph stops and hands control back to a person. It uses a real interrupt that checkpoints the run and resumes exactly where it paused once the human decides.

The hardest bug in the whole project was not machine learning at all. The checkpointer serialises the state so a run can pause and continue, but a pandas DataFrame and a fitted model are not serialisable. My first version broke the moment it tried to save. The fix was to keep those heavy objects in an in-memory store keyed by a run id, and let the graph state carry only that lightweight id. The state stays serialisable, and the pause and resume work. It is a small idea that took real debugging to find, and it is the part I am most quietly proud of.

Advice that is grounded, and that learns from use

The tool does not narrate from thin air. A small knowledge base of data-science guidance is retrieved to ground the written insights, with the source named, so the advice it gives is backed by real material rather than invented. That is ordinary retrieval augmented generation.

The piece I like more is the memory. Every run is saved with a numeric fingerprint of the dataset, its size, shape, missingness, imbalance, and task. When a new dataset arrives, the tool retrieves the most similar past runs and recommends by analogy. The more it is used, the better its suggestions get, because it actually remembers what happened before. And everything is grounded and works offline, the language-model step is optional and only polishes the wording.

A browser, a terminal, and an AI assistant

The same engine is exposed as a Flask web console with a live pipeline and an interactive dashboard, as the LangGraph agent you run from the command line, and as a Model Context Protocol server with nine tools, packaged as an installable plugin. That last one means you can drop it into an AI client like Claude Desktop and run the whole pipeline by chatting, profile a dataset, detect the task, run the leaderboard, train and evaluate, all as tool calls. One engine, three ways to reach it.

Three things I believe now

Automation is the easy half. Wiring the steps together took a week, making the result trustworthy took the rest of the project. Leakage is the real enemy. A number you cannot explain is a number you cannot use, so the guardrails that hunt for a cheat matter more than any single model. And ground the words. The moment a tool starts explaining itself, every sentence should trace back to the data or to retrieved guidance, never to a confident guess.

It is open source

The full engine, the web console, the agent, the MCP plugin, and an offline test suite are all in the repo, with a written report you can generate in a minute on the bundled sample data.

If you work on applied machine learning, AutoML, or trustworthy AI, I would genuinely value your critique. I am always happy to talk shop, and always building toward something you can actually trust.