What is Unsupervised Learning?

The plain definition

Unsupervised learning is what you do when nobody has told you the right answer. There is no target column. The algorithm's task is to describe the shape of the data: which records resemble each other, which dimensions carry the variation, which points sit far from everything else.

This makes it powerful for exploration and treacherous for evaluation. With supervised learning you can measure whether you were right. With unsupervised learning, correctness is often a judgement call.

The three things it is actually used for

Clustering groups similar records. K-means is the default and assumes roughly spherical, similarly sized clusters — an assumption that is wrong more often than it is stated. DBSCAN and HDBSCAN find arbitrarily shaped clusters and, importantly, are allowed to label a point as noise rather than forcing it into a group. Customer segmentation, document topic grouping, and behavioural cohorts all live here.

Dimensionality reduction compresses many correlated columns into a few informative ones. PCA is the linear workhorse. UMAP and t-SNE produce two-dimensional maps that are excellent for looking at data and genuinely dangerous for drawing conclusions from — the distances between clusters in a t-SNE plot are not meaningful, and the same data will produce visibly different pictures at different perplexity settings.

Density and novelty estimation models what normal looks like so that abnormal becomes measurable. This is the engine behind most anomaly detection, and it is often the right framing for fraud, equipment failure, and data-quality monitoring — precisely because you cannot collect labelled examples of failures that have not happened yet.

How to validate something with no ground truth

You cannot compute accuracy, so you need other evidence. Internal metrics such as silhouette score or the Davies-Bouldin index measure whether clusters are compact and well separated; they are useful for choosing the number of clusters, and they are not evidence that the clusters mean anything.

Stability is a stronger signal. Re-run the clustering on bootstrap resamples of the data. If the groupings survive resampling, they reflect structure. If they reshuffle every run, you have found noise and given it names.

Downstream utility is the strongest signal of all. A segmentation is good if acting on it changes an outcome — if the segments respond differently to a campaign, or predict retention, or let an operations team route work more effectively. That is a testable claim, and it is the only one that should justify putting a segmentation into production.

Where it fits in a modern stack

Unsupervised methods increasingly act as a preprocessing stage rather than a final answer. Embeddings — dense vectors learned without explicit labels — are unsupervised representations, and they now underpin search, deduplication, recommendation, and retrieval. Self-supervised pretraining, the technique behind every large language model, is unsupervised learning wearing a different hat: the labels are generated from the data itself by masking or predicting the next token.

In practice most production systems are hybrids. Cluster to understand and to generate candidate features, then use a supervised model for the decision that carries business consequences.