leakR

Welcome to leakR, an R package designed to help researchers, data scientists, and machine learning practitioners rigorously detect and diagnose data leakage in their workflows.

Data leakage is a pervasive yet often overlooked issue that undermines the integrity and reproducibility of predictive models by allowing unintended information to “leak” between training and testing phases. leakR provides a modular, extensible toolkit for detecting the most common and impactful forms of leakage, starting with tabular data contamination, target leakage, and temporal misalignments, while laying the foundation for a universal leakage detection framework across diverse data domains.

Installation

From CRAN (Recommended)

install.packages("leakr")

From GitHub (Development Version)

For the latest features and bug fixes:

# Install devtools if you don't have it
install.packages("devtools")

# Install leakR from GitHub
devtools::install_github("cherylisabella/leakR")

Quick Start

library(leakr)

# Basic audit of your dataset
report <- leakr_audit(iris, target = "Species")

# View summary of issues found
leakr_summarise(report)

# Generate diagnostic visualizations
leakr_plot(report)

# Access detailed results
print(report)

Main Functions

Function	Purpose
`leakr_audit()`	Main auditing function - detects leakage across your dataset
`leakr_summarise()`	Generate human-readable summaries of detected issues
`leakr_plot()`	Create diagnostic visualizations highlighting problems
`leakr_from_caret()`	Import and audit caret workflow objects
`leakr_from_tidymodels()`	Import and audit tidymodels workflow objects
`leakr_from_mlr3()`	Import and audit mlr3 workflow objects

Learn More

Get started with the comprehensive vignettes:

# Getting started guide
vignette("getting-started", package = "leakr")

# Advanced detection techniques
vignette("advanced-detection", package = "leakr") 

# Framework integration examples
vignette("framework-integration", package = "leakr")

Why leakR?

Automates leakage detection, filling a key methodological gap
Designed for clarity, reproducibility, and transparent ML research
Modular architecture supports gradual expansion (time series, NLP, images)
Useful for both academic and industry workflows

What leakR Detects

Train/test contamination - Overlapping records between training and test sets
Target leakage - Features that contain information about the target variable that wouldn’t be available at prediction time
Duplicate rows/records - Exact and near-duplicate observations that can inflate performance metrics
Temporal misalignments - Time-based data leaks in time series analysis

Key Features

Visual summaries of suspicious patterns and leakage hotspots
Detailed leakage reports suitable for audits, peer review, or publications
Clean APIs for seamless integration into existing ML workflows
Example vignettes demonstrating real leakage phenomena with code illustrations
Framework integration with caret, tidymodels, and mlr3

Development Roadmap

Phase 1: Core tabular leakage detectors ✓
Phase 2: Time series leakage detection (in progress)
Phase 3: Domain-specific extensions (NLP, image pipelines)
Phase 4: Pipeline integration and multi-language support

Citation

If you use leakR in your research, please cite:

@Manual{leakr2025,
  title = {leakR: Data Leakage Detection Tools for Machine Learning},
  author = {Cheryl Isabella Lim},
  year = {2025},
  note = {R package version 0.1.0},
  url = {https://github.com/cherylisabella/leakR},
}

License

This project is licensed under the MIT License - see the LICENSE file for details.

leakR is currently under development. Feedback and contributions are welcome from the community!