What is Reinforcement Learning?

The plain definition

Reinforcement learning has no answer key. Instead there is an agent, an environment, a set of actions, and a reward signal. The agent acts, the environment changes and returns a reward, and over many episodes the agent learns a policy — a rule for choosing actions — that maximises reward accumulated over time.

The phrase 'over time' carries the difficulty. A move that looks bad now may be excellent three hundred steps later. Sacrificing a piece in chess, spending on acquisition to earn retention, throttling throughput to avoid a queue collapse. Assigning credit for a delayed outcome to the specific decision that caused it is the central technical problem, and it is called the credit assignment problem.

Exploration versus exploitation

An agent that always takes the action it currently believes is best will never discover a better one. An agent that always experiments never banks its winnings. Every RL algorithm is, at heart, a scheme for managing that trade-off — epsilon-greedy, upper confidence bounds, Thompson sampling, entropy bonuses.

This is also the cleanest bridge into commercial use. A multi-armed bandit is reinforcement learning stripped to a single decision with no state, and it is genuinely practical: it beats fixed-split A/B testing for ongoing content, pricing, and creative selection because it reallocates traffic toward winners while still sampling alternatives. If someone in your organisation wants RL, a contextual bandit is usually the version that ships.

Why full RL rarely reaches production

Reinforcement learning needs to fail a great many times to learn. In a simulator that is free. In the real world it means a pricing engine that loses money for a quarter, a logistics agent that misroutes real freight, or a control system that damages equipment. That is why RL's public triumphs are overwhelmingly in domains with cheap, high-fidelity simulators: board games, video games, and closed-loop physical control such as datacentre cooling.

The remaining obstacles are just as real. Reward specification is notoriously error-prone — agents optimise the reward you wrote, not the outcome you meant, and reward hacking is the norm rather than the exception. Sample efficiency is poor; millions of episodes is routine. Non-stationarity undermines the whole premise: if customer behaviour or competitor pricing shifts, the environment the policy learned no longer exists.

Before choosing RL, apply this test. If you can frame the problem as predicting an outcome from historical data, use supervised learning — it is cheaper, faster, and far easier to validate. RL earns its cost only when your decisions genuinely change the future state of the system, and when you can either simulate that system faithfully or afford to learn from live consequences.

RLHF and why the term now means two things

Reinforcement learning from human feedback (RLHF) is the reason most people encounter the term today. A reward model is trained on human preference comparisons — which of these two responses is better — and then a policy is optimised against that learned reward. This is the alignment step that turns a raw next-token predictor into a usable assistant, and it is the final stage in producing most modern large language models.

It is worth keeping the two senses distinct. Classical RL learns to act in an environment over long horizons. RLHF applies RL machinery to a single-step generation problem in order to align outputs with human taste. Related methods such as DPO now achieve much the same result without the reinforcement learning loop at all.