ML in finance is not ML in tech.

A model that achieves 0.65 AUC on ImageNet-scale data is unremarkable. A model that achieves 0.65 AUC on next-day equity direction is either a discovery or a bug — and the prior on “bug” should be very high.

Financial markets are close to efficient, adversarial (someone is on the other side of every trade, and they adapt), and non-stationary at every timescale that matters. Standard ML practice — i.i.d. assumptions, random train/test splits, feature importance from a single fitted model — was not built for this environment, and applying it naively is the single most reliable way to produce a strategy that looks great in-sample and loses money live.

This is causal inference and time series under the specific stress of a P&L: it is not enough for a model to be right about a cause or a dependency in general — it has to be right about the one causing tomorrow's return, net of costs, out of sample.

Financial data structures: sampling in event time

Most ML pipelines start from time bars — one row per fixed interval (minute, hour, day). Time bars oversample quiet periods and undersample the bursts of information flow that actually move markets, and they produce return distributions further from normal than the alternative. Dollar bars and volume bars — sampling a new bar every time a fixed amount of dollar value or volume trades — synchronize sampling with information arrival instead of the clock, and empirically produce returns closer to i.i.d. Gaussian, which is exactly the property most downstream statistics assume.

Labeling: from fixed horizons to the triple-barrier method

The naive label — did the price rise over the next $k$ bars? — ignores path and ignores risk. A security that dips 8% and recovers to +1% gets the same label as one that rises steadily to +1%, even though no risk manager would treat those as equivalent outcomes.

The triple-barrier method labels each observation by which of three barriers is touched first: an upper barrier (profit-take), a lower barrier (stop-loss), or a vertical barrier (a maximum holding period). This ties the label directly to a realistic trading rule instead of an arbitrary fixed horizon.

Meta-labeling

Meta-labeling splits the problem in two: a primary model decides direction (long, short, or flat — often from a simple, interpretable rule or an existing signal), and a secondary model predicts whether the primary model's call will be correct, which sets position size. This decoupling means the secondary model only needs to learn when to trust a signal, a much lower-variance target than direction itself — and it directly improves the F1-score of the combined system by filtering out the primary model's low-confidence calls.

Fractional differentiation

Standard practice differences a series once to achieve stationarity — but integer differencing removes all long-memory signal along with the trend, throwing away exactly the information an ML model needs to find. Fractional differentiation generalizes the difference operator to a non-integer order $d\in (0,1)$, applying the binomial series expansion of $(1-L{)}^d$ (where $L$ is the lag operator) to find the minimum $d$ that passes an ADF stationarity test while preserving the maximum possible memory. It is the direct answer to the over-differencing pitfall covered in the time series essay.

Cross-validation for non-i.i.d. data

Random K-fold cross-validation assumes exchangeability. Financial labels overlap in time (a triple-barrier label spans multiple future bars), so a random split puts temporally adjacent, highly correlated observations in both the training and test folds — leaking information and inflating out-of-sample performance estimates.

Purged K-fold cross-validation removes training observations whose label period overlaps with the test fold. Embargo adds a further buffer after each test fold before training resumes, to remove serial-correlation leakage that purging alone misses. Neither is optional for a result to be trustworthy.

Feature importance under substitution effects

Standard feature importance (e.g. from a single random forest fit) is unstable in finance because financial features are highly correlated — two redundant features can each show low importance individually while jointly carrying real signal, because each substitutes for the other across trees.

• MDI (Mean Decrease Impurity). Fast, in-sample, but biased toward high-cardinality features and distorted by substitution effects.
• MDA (Mean Decrease Accuracy). Out-of-sample, permutation-based — shuffle a feature and measure the drop in performance. More robust, more expensive.
• SFI (Single Feature Importance). Train a model on each feature in isolation. Immune to substitution effects entirely, at the cost of ignoring interaction effects.

No single method is sufficient on its own; using at least two (typically MDA plus SFI) and requiring agreement is standard practice for a feature set that will inform live capital.

Sample weighting and uniqueness

Overlapping labels violate the i.i.d. assumption most models are trained under in a second way: identical, highly-overlapping observations get equal weight in the loss function, which over-represents whatever regime produced the most overlapping labels. Weighting each observation by its average uniqueness — how much its label period overlaps with others — and by time-decay (recent observations matter more, since markets are non-stationary) corrects for both.

Common pitfalls

• Backtest overfitting. Testing enough strategy variants against the same historical data eventually finds one that works by chance. The deflated Sharpe ratio adjusts the significance threshold for the number of trials; the probability of backtest overfitting (PBO) estimates directly how likely the selected strategy is to underperform out-of-sample.
• Data snooping via shared research infrastructure. Every researcher using the same public dataset to search for signals is running one enormous, uncoordinated multiple-testing procedure — most published anomalies do not survive out-of-sample.
• Survivorship bias. Training and backtesting only on instruments that exist today silently excludes the failures — delistings, bankruptcies, mergers — that a live strategy would have had to survive.
• Ignoring transaction costs and market impact. A signal profitable in a frictionless backtest can be unprofitable the moment realistic slippage, fees, and impact are included.

The path forward

A production-oriented financial ML workflow, in order:

1. Sample in event time (dollar/volume bars), not clock time.
2. Label with the triple-barrier method; separate direction (primary model) from confidence and sizing (meta-labeling).
3. Achieve stationarity via fractional differentiation, not naive integer differencing, to preserve memory.
4. Validate with purged, embargoed cross-validation; report the deflated Sharpe ratio and PBO, not a single backtest Sharpe.

This workflow does not manufacture alpha. It removes the specific, well-documented ways that standard ML practice manufactures fake alpha — which, in a domain this adversarial, is most of the value a rigorous process can add.