What is data leakage in machine learning? Explain different types with examples and a prevention strategy.
Interview preparation resource from Gate Smashers.
Data leakage occurs when training uses information that would not be legitimately available when the model makes a real prediction, or when held-out data influences training. Typical cases include target leakage, future leakage, preprocessing before splitting, duplicate entities across splits and test-set-driven tuning. I prevent it by defining the prediction timestamp, splitting raw data first, fitting transformations inside training-only pipelines and auditing every feature’s source and availability.
| Leakage type | Example | Prevention |
|---|---|---|
| Target / post-outcome | Using a field recorded after the outcome | Enforce a prediction-time feature cutoff |
| Future / look-ahead | Future observations enter a rolling feature | Use time-aware windows and tests |
| Preprocessing | Scaler fitted on all data | Fit transformations inside each training fold |
| Entity overlap | Same customer appears in train and test | Group-aware splitting |
| Test tuning | Threshold selected using test results | Tune on validation data only |
from sklearn.pipeline import Pipeline
from sklearn.impute import SimpleImputer
from sklearn.preprocessing import StandardScaler
from sklearn.linear_model import LogisticRegression
model = Pipeline([
("imputer", SimpleImputer(strategy="median")),
("scaler", StandardScaler()),
("classifier", LogisticRegression())
])
# The pipeline learns imputation and scaling only from X_train.
model.fit(X_train, y_train)
score = model.score(X_validation, y_validation)Leakage is an information-boundary failure
A feature may be strongly correlated with the target and still be invalid. The key question is whether that value exists, in the same form, at the moment the production prediction is made. Leakage can also happen without an obviously suspicious feature when preprocessing or data selection uses held-out information.
Types and examples
- Target leakage: A hospital readmission model uses a discharge field that is recorded after the prediction should occur.
- Look-ahead leakage: A demand forecast uses a rolling average that accidentally includes future days.
- Train–test contamination: The same customer or near-duplicate document appears in both sets.
- Transformation leakage: A scaler, imputer or feature selector is fitted on the complete dataset.
- Label leakage through proxies: A feature such as ‘refund processed’ nearly encodes the fraud label because it happens after investigation.
- Evaluation leakage: Test performance guides model selection or decision-threshold tuning.
Prevention checklist
- Write down the prediction event, prediction timestamp and allowed data sources.
- Split at the entity and time level before fitting any learned transformation.
- Put preprocessing, feature selection and the estimator in one reproducible pipeline.
- Compute aggregates using only data available before each prediction timestamp.
- Search for duplicates and near-duplicates across splits.
- Ask domain experts when every feature becomes available operationally.
- Keep a final untouched test set and reproduce features using production-like code.
How leakage differs from overfitting
Overfitting learns unstable patterns from legitimate training information. Leakage gives the model illegitimate information. Both may create a train–validation mismatch, but regularization cannot repair a leaked evaluation design.
Common mistake
Dropping one suspicious column is not a complete leakage audit. Leakage may be hidden in timestamps, aggregates, preprocessing state, labels, sampling rules or duplicated entities.