Machine learning
& artificial intelligence.

Where ML projects actually fail

Very few machine learning projects fail because the model was not accurate enough. They fail earlier, on problem framing, or later, on deployment — and the modelling in between is the part that goes fine.

Framing failures look like this: the target variable is not what the business actually cares about, or the prediction arrives too late to change any decision, or the feature that makes the model accurate in testing would not exist at prediction time. That last one — target leakage — is the most common way a project produces a model that scores 94% in a notebook and is worthless in production. A churn model containing cancellation_reason is the canonical example; that field is empty until after the customer has left.

Deployment failures look different: the model exists but nothing consumes it, or the feature computation in the serving path quietly diverges from the training pipeline (see feature stores), or accuracy decays and nobody notices because the service still returns HTTP 200. We spend more time on framing and deployment than on modelling, because that is where the risk concentrates.

How we approach model development

We start with a baseline that is deliberately unambitious — a simple statistical rule, or gradient-boosted trees on the obvious features. This is not a formality. It establishes what the problem's achievable accuracy looks like, and a meaningful share of projects should stop there, because a deployed explainable model that solves the business problem beats an interesting architecture that does not ship.

From there we escalate only as the evidence justifies. On tabular data that usually means better features rather than a different model class — see XGBoost vs neural networks for why deep learning is frequently the more expensive route to a slightly worse answer on business data. On perceptual data — images, audio, free text — we start from a pretrained model and adapt it, because transfer learning means a few thousand labelled examples can be enough where training from scratch would need six figures.

Throughout, evaluation is split correctly for the problem: temporally for anything that runs forward in time, and by group where the same customer, patient, or device appears repeatedly. Getting the split wrong is how teams produce confident numbers that do not survive contact with production, and it is a mistake that cannot be detected from the metrics themselves.

What we deliver beyond the model

A model file is not a deliverable. What makes it usable is everything around it: a reproducible training pipeline, so the model can be rebuilt from versioned code and data; an evaluation suite broken out by the slices that matter rather than a single aggregate score; a serving path with measured p95 latency and cost per thousand predictions; and monitoring on input distributions, output distributions, and business outcome.

We also deliver the honest documentation — what the model is not good at, which inputs put it outside its training distribution, what error rate to expect on which segments, and what should trigger a retrain. That document is what lets your team own the system rather than treating it as a black box that someone else has to maintain.

Where explainability is a requirement rather than a nicety — credit, insurance, clinical, employment — we design for it from the start, because retrofitting explanation onto a model chosen purely for accuracy is considerably harder than choosing a model that can explain itself.