Title:	A Pipeline Toolkit for Reproducible Computation at Scale
Version:	7.13.11
Description:	A general-purpose computational engine for data analysis, drake rebuilds intermediate data objects when their dependencies change, and it skips work when the results are already up to date. Not every execution starts from scratch, there is native support for parallel and distributed computing, and completed projects have tangible evidence that they are reproducible. Extensive documentation, from beginner-friendly tutorials to practical examples and more, is available at the reference website https://docs.ropensci.org/drake/ and the online manual https://books.ropensci.org/drake/.
License:	GPL-3
URL:	https://github.com/ropensci/drake, https://docs.ropensci.org/drake/, https://books.ropensci.org/drake/
BugReports:	https://github.com/ropensci/drake/issues
Depends:	R (≥ 3.3.0)
Imports:	base64url, digest (≥ 0.6.21), igraph (≥ 2.0.0), methods, parallel, rlang (≥ 0.2.0), storr (≥ 1.1.0), tidyselect (≥ 1.0.0), txtq (≥ 0.2.3), utils, vctrs (≥ 0.2.0)
Suggests:	abind, bindr, callr, cli (≥ 1.1.0), clustermq (≥ 0.9.1), crayon, curl (≥ 2.7), data.table, datasets, disk.frame, downloader, fst, future (≥ 1.3.0), ggplot2, ggraph, grDevices, keras, knitr, lubridate, networkD3, prettycode, progress (≥ 1.2.2), qs (≥ 0.20.2), Rcpp, rmarkdown, rstudioapi, stats, styler (≥ 1.2.0), testthat (≥ 2.1.0), tibble, txtplot, usethis, visNetwork (≥ 2.0.9), webshot
Encoding:	UTF-8
Language:	en-US
RoxygenNote:	7.3.2
NeedsCompilation:	no
Packaged:	2024-12-04 11:19:57 UTC; C240390
Author:	William Michael Landau [aut, cre], Alex Axthelm [ctb], Jasper Clarkberg [ctb], Kirill Müller [ctb], Ben Bond-Lamberty [ctb], Tristan Mahr [ctb], Miles McBain [ctb], Noam Ross [ctb], Ellis Hughes [ctb], Matthew Mark Strasiotto [ctb], Ben Marwick [rev], Peter Slaughter [rev], Eli Lilly and Company [cph]
Maintainer:	William Michael Landau <will.landau.oss@gmail.com>
Repository:	CRAN
Date/Publication:	2024-12-04 11:30:06 UTC

drake: A pipeline toolkit for reproducible computation at scale.

Description

drake is a pipeline toolkit (⁠https://github.com/pditommaso/awesome-pipeline⁠) and a scalable, R-focused solution for reproducibility and high-performance computing.

Author(s)

William Michael Landau will.landau@gmail.com

References

⁠https://github.com/ropensci/drake⁠

Examples

## Not run: 
isolate_example("Quarantine side effects.", {
if (suppressWarnings(require("knitr"))) {
library(drake)
load_mtcars_example() # Get the code with drake_example("mtcars").
make(my_plan) # Build everything.
plot(my_plan) # fast call to vis_drake_graph()
make(my_plan) # Nothing is done because everything is already up to date.
reg2 = function(d) { # Change one of your functions.
  d$x3 = d$x^3
  lm(y ~ x3, data = d)
}
make(my_plan) # Only the pieces depending on reg2() get rebuilt.
# Write a flat text log file this time.
make(my_plan, cache_log_file = TRUE)
# Read/load from the cache.
readd(small)
loadd(large)
head(large)
}
# Dynamic branching
# Get the mean mpg for each cyl in the mtcars dataset.
plan <- drake_plan(
  raw = mtcars,
  group_index = raw$cyl,
  munged = target(raw[, c("mpg", "cyl")], dynamic = map(raw)),
  mean_mpg_by_cyl = target(
    data.frame(mpg = mean(munged$mpg), cyl = munged$cyl[1]),
    dynamic = group(munged, .by = group_index)
  )
)
make(plan)
readd(mean_mpg_by_cyl)
})

## End(Not run)

Default Makefile recipe

Description

2019-01-03

Usage

Makefile_recipe(
  recipe_command = drake::default_recipe_command(),
  target = "your_target",
  cache_path = NULL
)

Arguments

Value

A character scalar

analyses

Description

2019-02-15

Usage

analyses(...)

Arguments

Show the analysis wildcard used in plan_summaries().

Description

Deprecated on 2019-01-12.

Usage

analysis_wildcard()

Details

Used to generate workflow plan data frames.

Value

The analysis wildcard used in plan_summaries().

See Also

plan_summaries()

as_drake_filename

Description

2019-02-15

Usage

as_drake_filename(...)

Arguments

as_file

Description

2019-02-15

Usage

as_file(...)

Arguments

List the available hash algorithms for drake caches.

Description

Deprecated on 2018-12-12.

Usage

available_hash_algos()

Value

A character vector of names of available hash algorithms.

backend

Description

2019-02-15

Usage

backend(...)

Arguments

Row-bind together drake plans

Description

Combine drake plans together in a way that correctly fills in missing entries.

Usage

bind_plans(...)

Arguments

See Also

drake_plan(), make()

Examples

# You might need to refresh your data regularly (see ?triggers).
download_plan <- drake_plan(
  data = target(
    command = download_data(),
    trigger = "always"
  )
)
# But if the data don't change, the analyses don't need to change.
analysis_plan <- drake_plan(
  usage = get_usage_metrics(data),
  topline = scrape_topline_table(data)
)
your_plan <- bind_plans(download_plan, analysis_plan)
your_plan

Function build_drake_graph

Description

Use drake_config() instead.

Usage

build_drake_graph(
  plan,
  targets = plan$target,
  envir = parent.frame(),
  verbose = 1L,
  jobs = 1,
  console_log_file = NULL,
  trigger = drake::trigger(),
  cache = NULL
)

Arguments

Details

Deprecated on 2018-11-02.

Value

An igraph object.

build_graph

Description

2019-02-15

Usage

build_graph(...)

Arguments

See the time it took to build each target.

Description

Applies to targets in your plan, not imports or files.

Usage

build_times(
  ...,
  path = NULL,
  search = NULL,
  digits = 3,
  cache = drake::drake_cache(path = path),
  targets_only = NULL,
  verbose = NULL,
  jobs = 1,
  type = c("build", "command"),
  list = character(0)
)

Arguments

Details

Times for dynamic targets (⁠https://books.ropensci.org/drake/dynamic.html⁠) only reflect the time it takes to post-process the sub-targets (typically very fast) and exclude the time it takes to build the sub-targets themselves. Sub-targets build times are listed individually.

Value

A data frame of times, each from system.time().

See Also

predict_runtime()

Examples

## Not run: 
isolate_example("Quarantine side effects.", {
if (suppressWarnings(require("knitr"))) {
if (requireNamespace("lubridate")) {
# Show the build times for the mtcars example.
load_mtcars_example() # Get the code with drake_example("mtcars").
make(my_plan) # Build all the targets.
print(build_times()) # Show how long it took to build each target.
}
}
})

## End(Not run)

List all the built targets (non-imports) in the cache.

Description

Deprecated on 2019-01-08.

Usage

built(
  path = getwd(),
  search = TRUE,
  cache = drake::get_cache(path = path, search = search, verbose = verbose),
  verbose = 1L,
  jobs = 1
)

Arguments

Details

Targets are listed in the workflow plan data frame (see drake_plan().

Value

Character vector naming the built targets in the cache.

See Also

cached(), loadd()

List all the storr cache namespaces used by drake.

Description

Deprecated on 2019-01-12.

Usage

cache_namespaces(default = storr::storr_environment()$default_namespace)

Arguments

Details

Ordinary users do not need to worry about this function. It is just another window into drake's internals.

Value

A character vector of storr namespaces used for drake.

See Also

make()

Return the file path where the cache is stored, if applicable.

Description

Deprecated on 2019-01-12.

Usage

cache_path(cache = NULL)

Arguments

Details

Currently only works with storr::storr_rds() file system caches.

Value

File path where the cache is stored.

List targets in the cache.

Description

Tip: read/load a cached item with readd() or loadd().

Usage

cached(
  ...,
  list = character(0),
  no_imported_objects = FALSE,
  path = NULL,
  search = NULL,
  cache = drake::drake_cache(path = path),
  verbose = NULL,
  namespace = NULL,
  jobs = 1,
  targets_only = TRUE
)

Arguments

Value

Either a named logical indicating whether the given targets or cached or a character vector listing all cached items, depending on whether any targets are specified.

See Also

cached_planned(), cached_unplanned(), readd(), loadd(), drake_plan(), make()

Examples

## Not run: 
isolate_example("Quarantine side effects.", {
if (suppressWarnings(require("knitr"))) {
if (requireNamespace("lubridate")) {
load_mtcars_example() # Load drake's canonical example.
make(my_plan) # Run the project, build all the targets.
cached()
cached(targets_only = FALSE)
}
}
})

## End(Not run)

List targets in both the plan and the cache.

Description

Includes dynamic sub-targets as well. See examples for details.

Usage

cached_planned(
  plan,
  path = NULL,
  cache = drake::drake_cache(path = path),
  namespace = NULL,
  jobs = 1
)

Arguments

Value

A character vector of target and sub-target names.

See Also

cached(), cached_unplanned

Examples

## Not run: 
isolate_example("cache_planned() example", {
plan <- drake_plan(w = 1)
make(plan)
cached_planned(plan)
plan <- drake_plan(
  x = seq_len(2),
  y = target(x, dynamic = map(x))
)
cached_planned(plan)
make(plan)
cached_planned(plan)
cached()
})

## End(Not run)

List targets in the cache but not the plan.

Description

Includes dynamic sub-targets as well. See examples for details.

Usage

cached_unplanned(
  plan,
  path = NULL,
  cache = drake::drake_cache(path = path),
  namespace = NULL,
  jobs = 1
)

Arguments

Value

A character vector of target and sub-target names.

See Also

cached(), cached_planned

Examples

## Not run: 
isolate_example("cache_unplanned() example", {
plan <- drake_plan(w = 1)
make(plan)
cached_unplanned(plan)
plan <- drake_plan(
  x = seq_len(2),
  y = target(x, dynamic = map(x))
)
cached_unplanned(plan)
make(plan)
cached_unplanned(plan)
# cached_unplanned() helps clean superfluous targets.
cached()
clean(list = cached_unplanned(plan))
cached()
})

## End(Not run)

Cancel a target mid-build

Description

Cancel a target mid-build. Upon cancellation, drake halts the current target and moves to the next one. The target's previous value and metadata, if they exist, remain in the cache.

Usage

cancel(allow_missing = TRUE)

Arguments

Value

Nothing.

See Also

cancel_if

Examples

## Not run: 
isolate_example("cancel()", {
f <- function(x) {
  cancel()
  Sys.sleep(2) # Does not run.
}
g <- function(x) f(x)
plan <- drake_plan(y = g(1))
make(plan)
# Does not exist.
# readd(y)
})

## End(Not run)

Cancel a target mid-build under some condition

Description

Cancel a target mid-build if some logical condition is met. Upon cancellation, drake halts the current target and moves to the next one. The target's previous value and metadata, if they exist, remain in the cache.

Usage

cancel_if(condition, allow_missing = TRUE)

Arguments

Value

Nothing.

See Also

cancel

Examples

## Not run: 
isolate_example("cancel_if()", {
f <- function(x) {
  cancel_if(x > 1)
  Sys.sleep(2) # Does not run if x > 1.
}
g <- function(x) f(x)
plan <- drake_plan(y = g(2))
make(plan)
# Does not exist.
# readd(y)
})

## End(Not run)

check

Description

2019-02-15

Usage

check(...)

Arguments

Check a workflow plan data frame for obvious errors.

Description

Deprecated on 2019-01-12.

Usage

check_plan(
  plan = NULL,
  targets = NULL,
  envir = parent.frame(),
  cache = drake::get_cache(verbose = verbose),
  verbose = 1L,
  jobs = 1
)

Arguments

Details

Possible obvious errors include circular dependencies and missing input files.

Value

Invisibly return plan.

See Also

drake_plan(), make()

Invalidate and deregister targets.

Description

Force targets to be out of date and remove target names from the data in the cache. Be careful and run which_clean() before clean(). That way, you know beforehand which targets will be compromised.

Usage

clean(
  ...,
  list = character(0),
  destroy = FALSE,
  path = NULL,
  search = NULL,
  cache = drake::drake_cache(path = path),
  verbose = NULL,
  jobs = NULL,
  force = FALSE,
  garbage_collection = FALSE,
  purge = FALSE
)

Arguments

Details

By default, clean() invalidates all targets, so be careful. clean() always:

1. Forces targets to be out of date so the next make() does not skip them.

2. Deregisters targets so loadd(your_target) and readd(your_target) no longer work.

By default, clean() does not actually remove the underlying data. Even old targets from the distant past are still in the cache and recoverable via drake_history() and make(recover = TRUE). To actually remove target data from the cache, as well as any file_out() files from any targets you are currently cleaning, run clean(garbage_collection = TRUE). Garbage collection is slow, but it reduces the storage burden of the cache.

Value

Invisibly return NULL.

See Also

which_clean(), drake_gc()

Examples

## Not run: 
isolate_example("Quarantine side effects.", {
if (suppressWarnings(require("knitr"))) {
load_mtcars_example() # Get the code with drake_example("mtcars").
make(my_plan) # Run the project, build the targets.
# Show all registered targets in the cache.
cached()
# Deregister 'summ_regression1_large' and 'small' in the cache.
clean(summ_regression1_large, small)
# Those objects are no longer registered as targets.
cached()
# Rebuild the invalidated/outdated targets.
make(my_plan)
# Clean everything.
clean()
# But the data objects and files are not actually gone!
file.exists("report.md")
drake_history()
make(my_plan, recover = TRUE)
# You need garbage collection to actually remove the data
# and any file_out() files of any uncleaned targets.
clean(garbage_collection = TRUE)
drake_history()
make(my_plan, recover = TRUE)
}
})

## End(Not run)

Deprecated: clean the main example from drake_example("main")

Description

This function deletes files. Use at your own risk. Destroys the ⁠.drake/⁠ cache and the report.Rmd file in the current working directory. Your working directory (getcwd()) must be the folder from which you first ran load_main_example() and make(my_plan).

Usage

clean_main_example()

Details

Deprecated 2018-12-31.

Value

Nothing.

Clean the mtcars example from drake_example("mtcars")

Description

This function deletes files. Use at your own risk. Destroys the ⁠.drake/⁠ cache and the report.Rmd file in the current working directory. Your working directory (getcwd()) must be the folder from which you first ran load_mtcars_example() and make(my_plan).

Usage

clean_mtcars_example()

Value

nothing

See Also

load_mtcars_example(), clean()

Examples

## Not run: 
isolate_example("Quarantine side effects.", {
if (suppressWarnings(require("knitr"))) {
# Populate your workspace and write 'report.Rmd'.
load_mtcars_example() # Get the code: drake_example("mtcars")
# Check the dependencies of an imported function.
deps_code(reg1)
# Check the dependencies of commands in the workflow plan.
deps_code(my_plan$command[1])
deps_code(my_plan$command[4])
# Plot the interactive network visualization of the workflow.
outdated(my_plan) # Which targets are out of date?
# Run the workflow to build all the targets in the plan.
make(my_plan)
outdated(my_plan) # Everything should be up to date.
# For the reg2() model on the small dataset,
# the p-value is so small that there may be an association
# between weight and fuel efficiency after all.
readd(coef_regression2_small)
# Clean up the example.
clean_mtcars_example()
}
})

## End(Not run)

Auxiliary storr namespaces

Description

2019-02-13

Usage

cleaned_namespaces(default = storr::storr_environment()$default_namespace)

Arguments

Value

A character vector of storr namespaces that are cleaned during clean().

Build a target using the clustermq backend

Description

For internal use only

Usage

cmq_build(target, meta, deps, spec, config_tmp, config)

Arguments

Turn a script into a function.

Description

code_to_function() is a quick (and very dirty) way to retrofit drake to an existing script-based project. It parses individual ⁠\*.R/\*.RMD⁠ files into functions so they can be added into the drake workflow.

Usage

code_to_function(path, envir = parent.frame())

Arguments

Details

Most data science workflows consist of imperative scripts. drake, on the other hand, assumes you write functions. code_to_function() allows for pre-existing workflows to incorporate drake as a workflow management tool seamlessly for cases where re-factoring is unfeasible. So drake can monitor dependencies, the targets are passed as arguments of the dependent functions.

Value

A function to be input into the drake plan

See Also

file_in(), file_out(), knitr_in(), ignore(), no_deps(), code_to_plan(), plan_to_code(), plan_to_notebook()

Examples

## Not run: 
isolate_example("contain side effects", {
if (requireNamespace("ggplot2", quietly = TRUE)) {
# The `code_to_function()` function creates a function that makes it
# available for drake to process as part of the workflow.
# The main purpose is to allow pre-existing workflows to incorporate drake
# into the workflow seamlessly for cases where re-factoring is unfeasible.
#

script1 <- tempfile()
script2 <- tempfile()
script3 <- tempfile()
script4 <- tempfile()

writeLines(c(
  "data <- mtcars",
  "data$make <- do.call('c',",
  "lapply(strsplit(rownames(data), split=\" \"), `[`, 1))",
  "saveRDS(data, \"mtcars_alt.RDS\")"
 ),
  script1
)

writeLines(c(
  "data <- readRDS(\"mtcars_alt.RDS\")",
  "mtcars_lm <- lm(mpg~cyl+disp+vs+gear+make,data=data)",
  "saveRDS(mtcars_lm, \"mtcars_lm.RDS\")"
  ),
  script2
)
writeLines(c(
  "mtcars_lm <- readRDS(\"mtcars_lm.RDS\")",
  "lm_summary <- summary(mtcars_lm)",
  "saveRDS(lm_summary, \"mtcars_lm_summary.RDS\")"
  ),
  script3
)
writeLines(c(
  "data<-readRDS(\"mtcars_alt.RDS\")",
  "gg <- ggplot2::ggplot(data)+",
  "ggplot2::geom_point(ggplot2::aes(",
  "x=disp, y=mpg, shape=as.factor(vs), color=make))",
  "ggplot2::ggsave(\"mtcars_plot.png\", gg)"
 ),
  script4
)


do_munge <- code_to_function(script1)
do_analysis <- code_to_function(script2)
do_summarize <- code_to_function(script3)
do_vis <- code_to_function(script4)

plan <- drake_plan(
  munged   = do_munge(),
  analysis = do_analysis(munged),
  summary  = do_summarize(analysis),
  plot     = do_vis(munged)
 )

plan
# drake knows  "script1" is the first script to be evaluated and ran,
# because it has no dependencies on other code and a dependency of
# `analysis`. See for yourself:

make(plan)

# See the connections that the sourced scripts create:
if (requireNamespace("visNetwork", quietly = TRUE)) {
  vis_drake_graph(plan)
}
}
})

## End(Not run)

Turn an R script file or knitr / R Markdown report into a drake plan.

Description

code_to_plan(), plan_to_code(), and plan_to_notebook() together illustrate the relationships between drake plans, R scripts, and R Markdown documents.

Usage

code_to_plan(path)

Arguments

Details

This feature is easy to break, so there are some rules for your code file:

1. Stick to assigning a single expression to a single target at a time. For multi-line commands, please enclose the whole command in curly braces. Conversely, compound assignment is not supported (e.g. target_1 <- target_2 <- target_3 <- get_data()).

2. Once you assign an expression to a variable, do not modify the variable any more. The target/command binding should be permanent.

3. Keep it simple. Please use the assignment operators rather than assign() and similar functions.

See Also

drake_plan(), make(), plan_to_code(), plan_to_notebook()

Examples

plan <- drake_plan(
  raw_data = read_excel(file_in("raw_data.xlsx")),
  data = raw_data,
  hist = create_plot(data),
  fit = lm(Ozone ~ Temp + Wind, data)
)
file <- tempfile()
# Turn the plan into an R script a the given file path.
plan_to_code(plan, file)
# Here is what the script looks like.
cat(readLines(file), sep = "\n")
# Convert back to a drake plan.
code_to_plan(file)

config

Description

2019-02-15

Usage

config(...)

Arguments

Configure the hash algorithms, etc. of a drake cache.

Description

The purpose of this function is to prepare the cache to be called from make(). drake only uses a single hash algorithm now, so we no longer need this configuration step.

Usage

configure_cache(
  cache = drake::get_cache(verbose = verbose),
  short_hash_algo = drake::default_short_hash_algo(cache = cache),
  long_hash_algo = drake::default_long_hash_algo(cache = cache),
  log_progress = FALSE,
  overwrite_hash_algos = FALSE,
  verbose = 1L,
  jobs = 1,
  init_common_values = FALSE
)

Arguments

Details

Deprecated on 2018-12-12.

Value

A drake/storr cache.

dataframes_graph

Description

2019-02-15

Usage

dataframes_graph(...)

Arguments

Show the dataset wildcard used in plan_analyses() and plan_summaries().

Description

Deprecated on 2019-01-12.

Usage

dataset_wildcard()

Details

Used to generate workflow plan data frames.

Value

The dataset wildcard used in plan_analyses() and plan_summaries().

See Also

plan_analyses()

Run a function in debug mode.

Description

Internal function for drake_debug(). Not for general use.

Usage

debug_and_run(f)

Arguments

Value

The return value of f.

See Also

drake_debug()

Default arguments of Makefile parallelism

Description

2019-01-03

Usage

default_Makefile_args(jobs, verbose)

Arguments

Value

args for system2(command, args)

Default Makefile command

Description

2019-01-03

Usage

default_Makefile_command()

Value

A character scalar

Return the default title for graph visualizations

Description

For internal use only.

Usage

default_graph_title()

Value

A character scalar with the default graph title.

Examples

default_graph_title()

Return the default long hash algorithm for make().

Description

Deprecated. drake now only uses one hash algorithm per cache.

Usage

default_long_hash_algo(cache = NULL)

Arguments

Details

Deprecated on 2018-12-12

Value

A character vector naming a hash algorithm.

Default parallel backend

Description

2019-01-02

Usage

default_parallelism()

Value

character

Default Makefile recipe command

Description

2019-01-02

Usage

default_recipe_command()

Value

A character scalar with the default recipe command.

Return the default short hash algorithm for make().

Description

Deprecated. drake now only uses one hash algorithm per cache.

Usage

default_short_hash_algo(cache = NULL)

Arguments

Details

Deprecated on 2018-12-12

Value

A character vector naming a hash algorithm.

default_system2_args

Description

2019-02-15

Usage

default_system2_args(...)

Arguments

Default verbosity

Description

Deprecated on 2019-01-01

Usage

default_verbose()

Value

1

States of the dependencies of a target

Description

Deprecated on 2019-02-14.

Usage

dependency_profile(target, config, character_only = FALSE)

Arguments

Value

A data frame of the old hashes and new hashes of the data frame, along with an indication of which hashes changed since the last make().

deprecate_wildcard

Description

2019-02-15

Usage

deprecate_wildcard(...)

Arguments

deps

Description

2019-05-16

Usage

deps(...)

Arguments

List the dependencies of a function or command

Description

Functions are assumed to be imported, and language/text are assumed to be commands in a plan.

Usage

deps_code(x)

Arguments

Value

A data frame of the dependencies.

See Also

deps_target(), deps_knitr()

Examples

# Your workflow likely depends on functions in your workspace.
f <- function(x, y) {
  out <- x + y + g(x)
  saveRDS(out, "out.rds")
}
# Find the dependencies of f. These could be R objects/functions
# in your workspace or packages. Any file names or target names
# will be ignored.
deps_code(f)
# Define a workflow plan data frame that uses your function f().
my_plan <- drake_plan(
  x = 1 + some_object,
  my_target = x + readRDS(file_in("tracked_input_file.rds")),
  return_value = f(x, y, g(z + w))
)
# Get the dependencies of workflow plan commands.
# Here, the dependencies could be R functions/objects from your workspace
# or packages, imported files, or other targets in the workflow plan.
deps_code(my_plan$command[[1]])
deps_code(my_plan$command[[2]])
deps_code(my_plan$command[[3]])
# You can also supply expressions or text.
deps_code(quote(x + y + 123))
deps_code("x + y + 123")

Find the drake dependencies of a dynamic knitr report target.

Description

Dependencies in knitr reports are marked by loadd() and readd() in active code chunks.

Usage

deps_knitr(path)

Arguments

Value

A data frame of dependencies.

See Also

deps_code(), deps_target()

Examples

## Not run: 
isolate_example("Quarantine side effects.", {
load_mtcars_example() # Get the code with drake_example("mtcars").
deps_knitr("report.Rmd")
})

## End(Not run)

Find out why a target is out of date.

Description

The dependency profile can give you a hint as to why a target is out of date. It can tell you if

• the command changed (deps_profile() reports the hash of the command, not the command itself)

• at least one input file changed,

• at least one output file changed,

• or a non-file dependency changed. For this last part, the imports need to be up to date in the cache, which you can do with outdated() or make(skip_targets = TRUE).

• the pseudo-random number generator seed changed. Unfortunately, deps_profile() does not currently get more specific than that.

Usage

deps_profile(target, ..., character_only = FALSE, config = NULL)

Arguments

Value

A data frame of old and new values for each of the main triggers, along with an indication of which values changed since the last make().

See Also

diagnose(), deps_code(), make(), drake_config()

Examples

## Not run: 
isolate_example("Quarantine side effects.", {
if (suppressWarnings(require("knitr"))) {
load_mtcars_example() # Load drake's canonical example.
make(my_plan) # Run the project, build the targets.
# Get some example dependency profiles of targets.
deps_profile(small, my_plan)
# Change a dependency.
simulate <- function(x) {}
# Update the in-memory imports in the cache
# so deps_profile can detect changes to them.
# Changes to targets are already cached.
make(my_plan, skip_targets = TRUE)
# The dependency hash changed.
deps_profile(small, my_plan)
}
})

## End(Not run)

Internal function with a drake_config() argument

Description

Not a user-side function.

Usage

deps_profile_impl(target, config, character_only = FALSE)

Arguments

List the dependencies of a target

Description

Intended for debugging and checking your project. The dependency structure of the components of your analysis decides which targets are built and when.

Usage

deps_target(target, ..., character_only = FALSE, config = NULL)

Arguments

Value

A data frame with the dependencies listed by type (globals, files, etc).

See Also

deps_code(), deps_knitr()

Examples

## Not run: 
isolate_example("Quarantine side effects.", {
load_mtcars_example() # Get the code with drake_example("mtcars").
deps_target(regression1_small, my_plan)
})

## End(Not run)

Internal function with a drake_config() argument

Description

Not a user-side function.

Usage

deps_target_impl(target, config, character_only = FALSE)

Arguments

See the dependencies of a target

Description

Use deps_target() (singular) instead.

Usage

deps_targets(targets, config, reverse = FALSE)

Arguments

Details

Deprecated on 2018-08-30.

Value

Names of dependencies listed by type (object, input file, etc).

Get diagnostic metadata on a target.

Description

Diagnostics include errors, warnings, messages, runtimes, and other context/metadata from when a target was built or an import was processed. If your target's last build succeeded, then diagnose(your_target) has the most current information from that build. But if your target failed, then only diagnose(your_target)$error, diagnose(your_target)$warnings, and diagnose(your_target)$messages correspond to the failure, and all the other metadata correspond to the last build that completed without an error.

Usage

diagnose(
  target = NULL,
  character_only = FALSE,
  path = NULL,
  search = NULL,
  cache = drake::drake_cache(path = path),
  verbose = 1L
)

Arguments

Value

Either a character vector of target names or an object of class "error".

See Also

drake_failed(), drake_progress(), readd(), drake_plan(), make()

Examples

## Not run: 
isolate_example("Quarantine side effects.", {
diagnose() # List all the targets with recorded error logs.
# Define a function doomed to failure.
f <- function() {
  stop("unusual error")
}
# Create a workflow plan doomed to failure.
bad_plan <- drake_plan(my_target = f())
# Running the project should generate an error
# when trying to build 'my_target'.
try(make(bad_plan), silent = FALSE)
drake_failed() # List the failed targets from the last make() (my_target).
# List targets that failed at one point or another
# over the course of the project (my_target).
# drake keeps all the error logs.
diagnose()
# Get the error log, an object of class "error".
error <- diagnose(my_target)$error # See also warnings and messages.
str(error) # See what's inside the error log.
error$calls # View the traceback. (See the rlang::trace_back() function).
})

## End(Not run)

Do the prework in the prework argument to make().

Description

For internal use only. The only reason this function is exported is to set up parallel socket (PSOCK) clusters without too much fuss.

Usage

do_prework(config, verbose_packages)

Arguments

Value

Inivisibly returns NULL.

Examples

## Not run: 
isolate_example("Quarantine side effects.", {
if (suppressWarnings(require("knitr"))) {
load_mtcars_example() # Get the code with drake_example("mtcars").
# Create a main internal configuration list with prework.
con <- drake_config(my_plan, prework = c("library(knitr)", "x <- 1"))
# Do the prework. Usually done at the beginning of `make()`,
# and for distributed computing backends like "future_lapply",
# right before each target is built.
do_prework(config = con, verbose_packages = TRUE)
# The `eval` element is the environment where the prework
# and the commands in your workflow plan data frame are executed.
identical(con$eval$x, 1) # Should be TRUE.
}
})

## End(Not run)

doc_of_function_call

Description

2019-02-15

Usage

doc_of_function_call(...)

Arguments

Get a template file for execution on a cluster.

Description

Deprecated. Use drake_hpc_template_file() instead.

Usage

drake_batchtools_tmpl_file(
  example = drake::drake_hpc_template_files(),
  to = getwd(),
  overwrite = FALSE
)

Arguments

Details

Deprecated on 2018-06-27.

Build/process a single target or import.

Description

Not valid for dynamic branching.

Usage

drake_build(
  target,
  ...,
  meta = NULL,
  character_only = FALSE,
  replace = FALSE,
  config = NULL
)

Arguments

Value

The value of the target right after it is built.

See Also

drake_debug()

Examples

## Not run: 
isolate_example("Quarantine side effects.", {
if (suppressWarnings(require("knitr"))) {
# This example is not really a user-side demonstration.
# It just walks through a dive into the internals.
# Populate your workspace and write 'report.Rmd'.
load_mtcars_example() # Get the code with drake_example("mtcars").
out <- drake_build(small, my_plan)
# Now includes `small`.
cached()
head(readd(small))
# `small` was invisibly returned.
head(out)
}
})

## End(Not run)

Internal function with a drake_config() argument

Description

Not a user-side function.

Usage

drake_build_impl(
  target,
  config = NULL,
  meta = NULL,
  character_only = FALSE,
  replace = FALSE
)

Arguments

Get the cache of a drake project.

Description

make() saves the values of your targets so you rarely need to think about output files. By default, the cache is a hidden folder called ⁠.drake/⁠. You can also supply your own storr cache to the cache argument of make(). The drake_cache() function retrieves this cache.

Usage

drake_cache(path = NULL, verbose = NULL, console_log_file = NULL)

Arguments

Details

drake_cache() actually returns a decorated storr, an object that contains a storr (plus bells and whistles). To get the actual inner storr, use drake_cache()$storr. Most methods are delegated to the inner storr. Some methods and objects are new or overwritten. Here are the ones relevant to users.

• history: drake's history (which powers drake_history()) is a txtq. Access it with drake_cache()$history.

• import(): The import() method is a function that can import targets, function dependencies, etc. from one decorated storr to another. History is not imported. For that, you have to work with the history txtqs themselves, Arguments to import():

  • ... and list: specify targets to import just like with loadd(). Leave these blank to import everything.

  • from: the decorated storr from which to import targets.

  • jobs: number of local processes for parallel computing.

  • gc: TRUE or FALSE, whether to run garbage collection for memory after importing each target. Recommended, but slow.

• export(): Same as import(), except the from argument is replaced by to: the decorated storr where the targets end up.

Value

A drake/storr cache in a folder called ⁠.drake/⁠, if available. NULL otherwise.

See Also

new_cache(), drake_config()

Examples

## Not run: 
isolate_example("Quarantine side effects.", {
if (suppressWarnings(require("knitr"))) {
clean(destroy = TRUE)
# No cache is available.
drake_cache() # NULL
load_mtcars_example() # Get the code with drake_example("mtcars").
make(my_plan) # Run the project, build the targets.
x <- drake_cache() # Now, there is a cache.
y <- storr::storr_rds(".drake") # Nearly equivalent.
# List the objects readable from the cache with readd().
x$list()
# drake_cache() actually returns a *decorated* storr.
# The *real* storr is inside.
drake_cache()$storr
}
# You can import and export targets to and from decorated storrs.
plan1 <- drake_plan(w = "w", x = "x")
plan2 <- drake_plan(a = "a", x = "x2")
cache1 <- new_cache("cache1")
cache2 <- new_cache("cache2")
make(plan1, cache = cache1)
make(plan2, cache = cache2)
cache1$import(cache2, a)
cache1$get("a")
cache1$get("x")
cache1$import(cache2)
cache1$get("x")
# With txtq >= 0.1.6.9002, you can import history from one cache into
# another.
# nolint start
# drake_history(cache = cache1)
# cache1$history$import(cache2$history)
# drake_history(cache = cache1)
# nolint end
})

## End(Not run)

Get the state of the cache.

Description

Get the fingerprints of all the targets in a data frame. This functionality is like make(..., cache_log_file = TRUE), but separated and more customizable. Hopefully, this functionality is a step toward better data versioning tools.

Usage

drake_cache_log(
  path = NULL,
  search = NULL,
  cache = drake::drake_cache(path = path),
  verbose = 1L,
  jobs = 1,
  targets_only = FALSE
)

Arguments

Details

A hash is a fingerprint of an object's value. Together, the hash keys of all your targets and imports represent the state of your project. Use drake_cache_log() to generate a data frame with the hash keys of all the targets and imports stored in your cache. This function is particularly useful if you are storing your drake project in a version control repository. The cache has a lot of tiny files, so you should not put it under version control. Instead, save the output of drake_cache_log() as a text file after each make(), and put the text file under version control. That way, you have a changelog of your project's results. See the examples below for details. Depending on your project's history, the targets may be different than the ones in your workflow plan data frame. Also, the keys depend on the hash algorithm of your cache. To define your own hash algorithm, you can create your own storr cache and give it a hash algorithm (e.g. storr_rds(hash_algorithm = "murmur32"))

Value

Data frame of the hash keys of the targets and imports in the cache

See Also

cached(), drake_cache()

Examples

## Not run: 
isolate_example("Quarantine side effects.", {
if (suppressWarnings(require("knitr"))) {
# Load drake's canonical example.
load_mtcars_example() # Get the code with drake_example()
# Run the project, build all the targets.
make(my_plan)
# Get a data frame of all the hash keys.
# If you want a changelog, be sure to do this after every make().
cache_log <- drake_cache_log()
head(cache_log)
# Suppress partial arg match warnings.
suppressWarnings(
  # Save the hash log as a flat text file.
  write.table(
    x = cache_log,
    file = "drake_cache.log",
    quote = FALSE,
    row.names = FALSE
  )
)
# At this point, put drake_cache.log under version control
# (e.g. with 'git add drake_cache.log') alongside your code.
# Now, every time you run your project, your commit history
# of hash_lot.txt is a changelog of the project's results.
# It shows which targets and imports changed on every commit.
# It is extremely difficult to track your results this way
# by putting the raw '.drake/' cache itself under version control.
}
})

## End(Not run)

Generate a flat text log file to represent the state of the cache.

Description

Deprecated on 2019-03-09.

Usage

drake_cache_log_file(
  file = "drake_cache.log",
  path = getwd(),
  search = TRUE,
  cache = drake::get_cache(path = path, search = search, verbose = verbose),
  verbose = 1L,
  jobs = 1L,
  targets_only = FALSE
)

Arguments

Details

Calling this function to create a log file and later calling make() makes the log file out of date. Therefore, we recommend using make() with the cache_log_file argument to create the cache log. This way ensures that the log is always up to date with make() results.

Value

There is no return value, but a log file is generated.

See Also

drake_cache_log(), make(), get_cache()

List cancelled targets.

Description

List the targets that were cancelled in the current or previous call to make() using cancel() or cancel_if().

Usage

drake_cancelled(cache = drake::drake_cache(path = path), path = NULL)

Arguments

Value

A character vector of target names.

See Also

drake_running(), drake_failed(), make()

Examples

## Not run: 
isolate_example("contain side effects", {
plan <- drake_plan(x = 1, y = cancel_if(x > 0))
make(plan)
drake_cancelled()
})

## End(Not run)

Ending of _drake.R for r_make() and friends

Description

Call this function inside the ⁠_drake.R⁠ script for r_make() and friends. All non-deprecated function arguments are the same between make() and drake_config().

Usage

drake_config(
  plan,
  targets = NULL,
  envir = parent.frame(),
  verbose = 1L,
  hook = NULL,
  cache = drake::drake_cache(),
  fetch_cache = NULL,
  parallelism = "loop",
  jobs = 1L,
  jobs_preprocess = 1L,
  packages = rev(.packages()),
  lib_loc = NULL,
  prework = character(0),
  prepend = NULL,
  command = NULL,
  args = NULL,
  recipe_command = NULL,
  timeout = NULL,
  cpu = Inf,
  elapsed = Inf,
  retries = 0,
  force = FALSE,
  log_progress = TRUE,
  graph = NULL,
  trigger = drake::trigger(),
  skip_targets = FALSE,
  skip_imports = FALSE,
  skip_safety_checks = FALSE,
  lazy_load = "eager",
  session_info = NULL,
  cache_log_file = NULL,
  seed = NULL,
  caching = c("main", "master", "worker"),
  keep_going = FALSE,
  session = NULL,
  pruning_strategy = NULL,
  makefile_path = NULL,
  console_log_file = NULL,
  ensure_workers = NULL,
  garbage_collection = FALSE,
  template = list(),
  sleep = function(i) 0.01,
  hasty_build = NULL,
  memory_strategy = "speed",
  spec = NULL,
  layout = NULL,
  lock_envir = NULL,
  history = TRUE,
  recover = FALSE,
  recoverable = TRUE,
  curl_handles = list(),
  max_expand = NULL,
  log_build_times = TRUE,
  format = NULL,
  lock_cache = TRUE,
  log_make = NULL,
  log_worker = FALSE
)

Arguments

Details

In drake, make() has two stages:

1. Configure a workflow to your environment and plan.

2. Build targets. The drake_config() function just does step (1), which is a common requirement for not only make(), but also utility functions like vis_drake_graph() and outdated(). That is why drake_config() is a requirement for the ⁠_drake.R⁠ script, which powers r_make(), r_outdated(), r_vis_drake_graph(), etc.

Value

A configured drake workflow.

Recovery

make(recover = TRUE, recoverable = TRUE) powers automated data recovery. The default of recover is FALSE because targets recovered from the distant past may have been generated with earlier versions of R and earlier package environments that no longer exist.

How it works: if recover is TRUE, drake tries to salvage old target values from the cache instead of running commands from the plan. A target is recoverable if

1. There is an old value somewhere in the cache that shares the command, dependencies, etc. of the target about to be built.

2. The old value was generated with make(recoverable = TRUE).

If both conditions are met, drake will

1. Assign the most recently-generated admissible data to the target, and

2. skip the target's command.

See Also

make(), drake_plan(), vis_drake_graph()

Examples

## Not run: 
isolate_example("quarantine side effects", {
if (requireNamespace("knitr", quietly = TRUE)) {
writeLines(
  c(
    "library(drake)",
    "load_mtcars_example()",
    "drake_config(my_plan, targets = c(\"small\", \"large\"))"
  ),
  "_drake.R" # default value of the `source` argument
)
cat(readLines("_drake.R"), sep = "\n")
r_outdated()
r_make()
r_outdated()
}
})

## End(Not run)

Run a single target's command in debug mode.'

Description

Not valid for dynamic branching. drake_debug() loads a target's dependencies and then runs its command in debug mode (see browser(), debug(), and debugonce()). This function does not store the target's value in the cache (see ⁠https://github.com/ropensci/drake/issues/587⁠).

Usage

drake_debug(
  target = NULL,
  ...,
  character_only = FALSE,
  replace = FALSE,
  verbose = TRUE,
  config = NULL
)

Arguments

Value

The value of the target right after it is built.

See Also

drake_build()

Examples

## Not run: 
isolate_example("Quarantine side effects.", {
if (suppressWarnings(require("knitr"))) {
# This example is not really a user-side demonstration.
# It just walks through a dive into the internals.
# Populate your workspace and write 'report.Rmd'.
load_mtcars_example() # Get the code with drake_example("mtcars").
# out <- drake_debug(small, my_plan)
# `small` was invisibly returned.
# head(out)
}
})

## End(Not run)

drake_deps helper

Description

Static code analysis.

Usage

drake_deps(expr, exclude = character(0), restrict = NULL)

Arguments

Value

A drake_deps object.

Examples

if (FALSE) { # stronger than roxygen dontrun
expr <- quote({
  a <- base::list(1)
  b <- seq_len(10)
  file_out("abc")
  file_in("xyz")
  x <- "123"
  loadd(abc)
  readd(xyz)
})
drake_deps(expr)
}

drake_deps_ht helper

Description

Static code analysis.

Usage

drake_deps_ht(expr, exclude = character(0), restrict = NULL)

Arguments

Value

A drake_deps_ht object.

Examples

if (FALSE) { # stronger than roxygen dontrun
expr <- quote({
  a <- base::list(1)
  b <- seq_len(10)
  file_out("abc")
  file_in("xyz")
  x <- "123"
  loadd(abc)
  readd(xyz)
})
drake_deps_ht(expr)
}

List done targets.

Description

List the targets that completed in the current or previous call to make().

Usage

drake_done(cache = drake::drake_cache(path = path), path = NULL)

Arguments

Value

A character vector of target names.

See Also

drake_running(), drake_failed(), drake_cancelled(), drake_progress(), make()

Examples

## Not run: 
isolate_example("contain side effects", {
plan <- drake_plan(x = 1, y = x)
make(plan)
drake_done()
})

## End(Not run)

Get the environment where drake builds targets

Description

Call this function inside the commands in your plan to get the environment where drake builds targets. Advanced users can use it to strategically remove targets from memory while make() is running.

Usage

drake_envir(which = c("targets", "dynamic", "subtargets", "imports"))

Arguments

Details

drake manages in-memory targets in 4 environments: one with sub-targets, one with whole dynamic targets, one with static targets, and one with imported global objects and functions. This last environment is usually the environment from which you call make(). Select the appropriate environment for your use case with the which argument of drake_envir().

Value

The environment where drake builds targets.

Keywords

drake_plan() understands special keyword functions for your commands. With the exception of target(), each one is a proper function with its own help file.

• target(): give the target more than just a command. Using target(), you can apply a transformation (examples: ⁠https://books.ropensci.org/drake/plans.html#large-plans⁠), # nolint supply a trigger (⁠https://books.ropensci.org/drake/triggers.html⁠), # nolint or set any number of custom columns.

• file_in(): declare an input file dependency.

• file_out(): declare an output file to be produced when the target is built.

• knitr_in(): declare a knitr file dependency such as an R Markdown (⁠*.Rmd⁠) or R LaTeX (⁠*.Rnw⁠) file.

• ignore(): force drake to entirely ignore a piece of code: do not track it for changes and do not analyze it for dependencies.

• no_deps(): tell drake to not track the dependencies of a piece of code. drake still tracks the code itself for changes.

• id_chr(): Get the name of the current target.

• drake_envir(): get the environment where drake builds targets. Intended for advanced custom memory management.

See Also

from_plan()

Examples

## Not run: 
isolate_example("contain side effects", {
plan <- drake_plan(
  large_data_1 = sample.int(1e4),
  large_data_2 = sample.int(1e4),
  subset = c(large_data_1[seq_len(10)], large_data_2[seq_len(10)]),
  summary = {
    print(ls(envir = parent.env(drake_envir())))
    # We don't need the large_data_* targets in memory anymore.
    rm(large_data_1, large_data_2, envir = drake_envir("targets"))
    print(ls(envir = drake_envir("targets")))
    mean(subset)
  }
)
make(plan, cache = storr::storr_environment(), session_info = FALSE)
})

## End(Not run)

Download the files of an example drake project.

Description

The drake_example() function downloads a folder from ⁠https://github.com/wlandau/drake-examples⁠. By default, it creates a new folder with the example name in your current working directory. After the files are written, have a look at the enclosed README file. Other instructions are available in the files at ⁠https://github.com/wlandau/drake-examples⁠.

Usage

drake_example(
  example = "main",
  to = getwd(),
  destination = NULL,
  overwrite = FALSE,
  quiet = TRUE
)

Arguments

Value

NULL

See Also

drake_examples(), make()

Examples

## Not run: 
isolate_example("Quarantine side effects.", {
if (requireNamespace("downloader")) {
drake_examples() # List all the drake examples.
# Sets up the same example from load_mtcars_example()
drake_example("mtcars")
# Sets up the SLURM example.
drake_example("slurm")
}
})

## End(Not run)

List the names of all the drake examples.

Description

You can find the code files of the examples at ⁠https://github.com/wlandau/drake-examples⁠. The drake_examples() function downloads the list of examples from ⁠https://wlandau.github.io/drake-examples/examples.md⁠, so you need an internet connection.

Usage

drake_examples(quiet = TRUE)

Arguments

Value

Names of all the drake examples.

See Also

drake_example(), make()

Examples

## Not run: 
isolate_example("Quarantine side effects.", {
if (requireNamespace("downloader")) {
drake_examples() # List all the drake examples.
# Sets up the example from load_mtcars_example()
drake_example("mtcars")
# Sets up the SLURM example.
drake_example("slurm")
}
})

## End(Not run)

List failed targets.

Description

List the targets that quit in error during make().

Usage

drake_failed(cache = drake::drake_cache(path = path), path = NULL)

Arguments

Value

A character vector of target names.

See Also

drake_done(), drake_running(), drake_cancelled(), drake_progress(), make()

Examples

## Not run: 
isolate_example("contain side effects", {
if (suppressWarnings(require("knitr"))) {
# Build a plan doomed to fail:
bad_plan <- drake_plan(x = function_doesnt_exist())
cache <- storr::storr_environment() # optional
try(
  make(bad_plan, cache = cache, history = FALSE),
  silent = TRUE
) # error
drake_failed(cache = cache) # "x"
e <- diagnose(x, cache = cache) # Retrieve the cached error log of x.
names(e)
e$error
names(e$error)
}
})

## End(Not run)

Do garbage collection on the drake cache.

Description

Garbage collection removes obsolete target values from the cache.

Usage

drake_gc(
  path = NULL,
  search = NULL,
  verbose = NULL,
  cache = drake::drake_cache(path = path),
  force = FALSE
)

Arguments

Details

Caution: garbage collection actually removes data so it is no longer recoverable with drake_history() or make(recover = TRUE). You cannot undo this operation. Use at your own risk.

Value

NULL

See Also

clean()

Examples

## Not run: 
isolate_example("Quarantine side effects.", {
if (suppressWarnings(require("knitr"))) {
load_mtcars_example() # Get the code with drake_example("mtcars").
make(my_plan) # Run the project, build the targets.
# At this point, check the size of the '.drake/' cache folder.
# Clean without garbage collection.
clean(garbage_collection = FALSE)
# The '.drake/' cache folder is still about the same size.
drake_gc() # Do garbage collection on the cache.
# The '.drake/' cache folder should have gotten much smaller.
}
})

## End(Not run)

Session info of the last call to make().

Description

By default, session info is saved during make() to ensure reproducibility. Your loaded packages and their versions are recorded, for example.

Usage

drake_get_session_info(
  path = NULL,
  search = NULL,
  cache = drake::drake_cache(path = path),
  verbose = 1L
)

Arguments

Value

sessionInfo() of the last call to make()

See Also

diagnose(), cached(), readd(), drake_plan(), make()

Examples

## Not run: 
isolate_example("Quarantine side effects.", {
if (suppressWarnings(require("knitr"))) {
load_mtcars_example() # Get the code with drake_example("mtcars").
make(my_plan) # Run the project, build the targets.
drake_get_session_info() # Get the cached sessionInfo() of the last make().
}
})

## End(Not run)

Visualize the workflow with ggraph/ggplot2

Description

This function requires packages ggplot2 and ggraph. Install them with install.packages(c("ggplot2", "ggraph")).

Usage

drake_ggraph(
  ...,
  build_times = "build",
  digits = 3,
  targets_only = FALSE,
  main = NULL,
  from = NULL,
  mode = c("out", "in", "all"),
  order = NULL,
  subset = NULL,
  make_imports = TRUE,
  from_scratch = FALSE,
  full_legend = FALSE,
  group = NULL,
  clusters = NULL,
  show_output_files = TRUE,
  label_nodes = FALSE,
  transparency = TRUE,
  config = NULL
)

Arguments

Value

A ggplot2 object, which you can modify with more layers, show with plot(), or save as a file with ggsave().

See Also

vis_drake_graph(), sankey_drake_graph(), render_drake_ggraph(), text_drake_graph()

Examples

## Not run: 
isolate_example("Quarantine side effects.", {
load_mtcars_example() # Get the code with drake_example("mtcars").
# Plot the network graph representation of the workflow.
if (requireNamespace("ggraph", quietly = TRUE)) {
  drake_ggraph(my_plan) # Save to a file with `ggplot2::ggsave()`.
}
})

## End(Not run)

Internal function with a drake_config() argument

Description

Not a user-side function.

Usage

drake_ggraph_impl(
  config,
  build_times = "build",
  digits = 3,
  targets_only = FALSE,
  main = NULL,
  from = NULL,
  mode = c("out", "in", "all"),
  order = NULL,
  subset = NULL,
  make_imports = TRUE,
  from_scratch = FALSE,
  full_legend = FALSE,
  group = NULL,
  clusters = NULL,
  show_output_files = TRUE,
  label_nodes = FALSE,
  transparency = TRUE
)

Arguments

Prepare the workflow graph for visualization

Description

With the returned data frames, you can plot your own custom visNetwork graph.

Usage

drake_graph_info(
  ...,
  from = NULL,
  mode = c("out", "in", "all"),
  order = NULL,
  subset = NULL,
  build_times = "build",
  digits = 3,
  targets_only = FALSE,
  font_size = 20,
  from_scratch = FALSE,
  make_imports = TRUE,
  full_legend = FALSE,
  group = NULL,
  clusters = NULL,
  show_output_files = TRUE,
  hover = FALSE,
  on_select_col = NULL,
  config = NULL
)

Arguments

Value

A list of three data frames: one for nodes, one for edges, and one for the legend nodes. The list also contains the default title of the graph.

See Also

vis_drake_graph()

Examples

## Not run: 
isolate_example("Quarantine side effects.", {
if (requireNamespace("visNetwork", quietly = TRUE)) {
if (suppressWarnings(require("knitr"))) {
load_mtcars_example() # Get the code with drake_example("mtcars").
vis_drake_graph(my_plan)
# Get a list of data frames representing the nodes, edges,
# and legend nodes of the visNetwork graph from vis_drake_graph().
raw_graph <- drake_graph_info(my_plan)
# Choose a subset of the graph.
smaller_raw_graph <- drake_graph_info(
  my_plan,
  from = c("small", "reg2"),
  mode = "in"
)
# Inspect the raw graph.
str(raw_graph)
# Use the data frames to plot your own custom visNetwork graph.
# For example, you can omit the legend nodes
# and change the direction of the graph.
library(visNetwork)
graph <- visNetwork(nodes = raw_graph$nodes, edges = raw_graph$edges)
visHierarchicalLayout(graph, direction = 'UD')
}
}
})

## End(Not run)

Internal function

Description

Not a user-side function.

Usage

drake_graph_info_impl(
  config,
  from = NULL,
  mode = c("out", "in", "all"),
  order = NULL,
  subset = NULL,
  build_times = "build",
  digits = 3,
  targets_only = FALSE,
  font_size = 20,
  from_scratch = FALSE,
  make_imports = TRUE,
  full_legend = FALSE,
  group = NULL,
  clusters = NULL,
  show_output_files = TRUE,
  hover = FALSE,
  on_select_col = NULL
)

Arguments

History and provenance

Description

See the history and provenance of your targets: what you ran, when you ran it, the function arguments you used, and how to get old data back.

Usage

drake_history(cache = NULL, history = NULL, analyze = TRUE, verbose = NULL)

Arguments

Details

drake_history() returns a data frame with the following columns.

• target: the name of the target.

• current: logical, whether the row describes the data actually assigned to the target name in the cache, e.g. what you get with loadd(target) and readd(target). Does NOT tell you if the target is up to date.

• built: when the target's value was stored in the cache. This is the true creation date of the target's value, not the recovery date from make(recover = TRUE).

• exists: logical, whether the target's historical value still exists in the cache. Garbage collection via (clean(garbage_collection = TRUE) and drake_cache()$gc()) remove these historical values, but clean() under the default settings does not.

• hash: fingerprint of the target's historical value in the cache. If the value still exists, you can read it with drake_cache()$get_value(hash).

• command: the drake_plan() command executed to build the target.

• seed: random number generator seed.

• runtime: the time it took to execute the drake_plan() command. Does not include overhead due to drake's processing.

If analyze is TRUE, various other columns are included to show the explicitly-named length-1 arguments to function calls in the commands. See the "Provenance" section for more details.

Value

A data frame of target history.

Provenance

If analyze is TRUE, drake scans your drake_plan() commands for function arguments and mentions them in the history. A function argument shows up if and only if: 1. It has length 1.
2. It is atomic, i.e. a base type: logical, integer, real, complex, character, or raw.
3. It is explicitly named in the function call, For example, x is detected as 1 in fn(list(x = 1)) but not f(list(1)). The exceptions are file_out(), file_in(), and knitr_in(). For example, filename is detected as "my_file.csv" in process_data(filename = file_in("my_file.csv")). NB: in process_data(filename = file_in("a", "b")) filename is not detected because the value must be atomic.

Examples

## Not run: 
isolate_example("contain side-effects", {
if (requireNamespace("knitr", quietly = TRUE)) {
# First, let's iterate on a drake workflow.
load_mtcars_example()
make(my_plan, history = TRUE, verbose = 0L)
# Naturally, we'll make updates to our targets along the way.
reg2 <- function(d) {
  d$x2 <- d$x ^ 3
  lm(y ~ x2, data = d)
}
Sys.sleep(0.01)
make(my_plan, history = TRUE, verbose = 0L)
# The history is a data frame about all the recorded runs of your targets.
out <- drake_history(analyze = TRUE)
print(out)
# Let's use the history to recover the oldest version
# of our regression2_small target.
oldest_reg2_small <- max(which(out$target == "regression2_small"))
hash_oldest_reg2_small <- out[oldest_reg2_small, ]$hash
cache <- drake_cache()
cache$get_value(hash_oldest_reg2_small)
# If you run clean(), drake can still find all the targets.
clean(small)
drake_history()
# But if you run clean() with garbage collection,
# older versions of your targets may be gone.
clean(large, garbage_collection = TRUE)
drake_history()
invisible()
}
})

## End(Not run)

Write a template file for deploying work to a cluster / job scheduler.

Description

See the example files from drake_examples() and drake_example() for example usage.

Usage

drake_hpc_template_file(
  file = drake::drake_hpc_template_files(),
  to = getwd(),
  overwrite = FALSE
)

Arguments

Value

NULL is returned, but a batchtools template file is written.

See Also

drake_hpc_template_files(), drake_examples(), drake_example(), shell_file()

Examples

## Not run: 
plan <- drake_plan(x = rnorm(1e7), y = rnorm(1e7))
# List the available template files.
drake_hpc_template_files()
# Write a SLURM template file.
out <- file.path(tempdir(), "slurm_batchtools.tmpl")
drake_hpc_template_file("slurm_batchtools.tmpl", to = tempdir())
cat(readLines(out), sep = "\n")
# library(future.batchtools) # nolint
# future::plan(batchtools_slurm, template = out) # nolint
# make(plan, parallelism = "future", jobs = 2) # nolint

## End(Not run)

List the available example template files for deploying work to a cluster / job scheduler.

Description

See the example files from drake_examples() and drake_example() for example usage.

Usage

drake_hpc_template_files()

Value

A character vector of example template files that you can write with drake_hpc_template_file().

See Also

drake_hpc_template_file(), drake_examples(), drake_example(), shell_file()

Examples

## Not run: 
plan <- drake_plan(x = rnorm(1e7), y = rnorm(1e7))
# List the available template files.
drake_hpc_template_files()
# Write a SLURM template file.
out <- file.path(tempdir(), "slurm_batchtools.tmpl")
drake_hpc_template_file("slurm_batchtools.tmpl", to = tempdir())
cat(readLines(out), sep = "\n")
# library(future.batchtools) # nolint
# future::plan(batchtools_slurm, template = out) # nolint
# make(plan, parallelism = "future", jobs = 2) # nolint

## End(Not run)

Compute the initial pre-build metadata of a target or import.

Description

Deprecated on 2019-01-12.

Usage

drake_meta(target, config)

Arguments

Details

The metadata helps determine if the target is up to date or outdated. The metadata of imports is used to compute the metadata of targets. Target metadata is computed with drake_meta(), and then drake:::store_outputs() completes the metadata after the target is built. In other words, the output of drake_meta() corresponds to the state of the target immediately before make() builds it. See diagnose() to read the final metadata of a target, including any errors, warnings, and messages in the last build.

Value

A list of metadata on a target. Does not include the file modification time if the target is a file. That piece is computed later in make() by drake:::store_outputs().

See Also

diagnose(), deps_profile(), make()

Show drake's color palette.

Description

Deprecated on 2019-01-12.

Usage

drake_palette()

Details

This function is used in both the console and graph visualizations. Your console must have the crayon package enabled. This palette applies to console output (internal functions console() and console_many_targets()) and the node colors in the graph visualizations. So if you want to contribute improvements to the palette, please both drake_palette() and visNetwork::visNetwork(nodes = legend_nodes())

Value

There is a console message, but the actual return value is NULL.

Create a drake plan for the plan argument of make().

Description

A drake plan is a data frame with columns "target" and "command". Each target is an R object produced in your workflow, and each command is the R code to produce it.

Usage

drake_plan(
  ...,
  list = NULL,
  file_targets = NULL,
  strings_in_dots = NULL,
  tidy_evaluation = NULL,
  transform = TRUE,
  trace = FALSE,
  envir = parent.frame(),
  tidy_eval = TRUE,
  max_expand = NULL
)

Arguments

Details

Besides "target" and "command", drake_plan() understands a special set of optional columns. For details, visit ⁠https://books.ropensci.org/drake/plans.html#special-custom-columns-in-your-plan⁠ # nolint

Value

A data frame of targets, commands, and optional custom columns.

Columns

drake_plan() creates a special data frame. At minimum, that data frame must have columns target and command with the target names and the R code chunks to build them, respectively.

You can add custom columns yourself, either with target() (e.g. drake_plan(y = target(f(x), transform = map(c(1, 2)), format = "fst"))) or by appending columns post-hoc (e.g. plan$col <- vals).

Some of these custom columns are special. They are optional, but drake looks for them at various points in the workflow.

• transform: a call to map(), split(), cross(), or combine() to create and manipulate large collections of targets. Details: (⁠https://books.ropensci.org/drake/plans.html#large-plans⁠). # nolint

• format: set a storage format to save big targets more efficiently. See the "Formats" section of this help file for more details.

• trigger: rule to decide whether a target needs to run. It is recommended that you define this one with target(). Details: ⁠https://books.ropensci.org/drake/triggers.html⁠.

• hpc: logical values (TRUE/FALSE/NA) whether to send each target to parallel workers. Visit ⁠https://books.ropensci.org/drake/hpc.html#selectivity⁠ to learn more.

• resources: target-specific lists of resources for a computing cluster. See ⁠https://books.ropensci.org/drake/hpc.html#advanced-options⁠ for details.

• caching: overrides the caching argument of make() for each target individually. Possible values:

  • "main": tell the main process to store the target in the cache.

  • "worker": tell the HPC worker to store the target in the cache.

  • NA: default to the caching argument of make().

• elapsed and cpu: number of seconds to wait for the target to build before timing out (elapsed for elapsed time and cpu for CPU time).

• retries: number of times to retry building a target in the event of an error.

• seed: an optional pseudo-random number generator (RNG) seed for each target. drake usually comes up with its own unique reproducible target-specific seeds using the global seed (the seed argument to make() and drake_config()) and the target names, but you can overwrite these automatic seeds. NA entries default back to drake's automatic seeds.

• max_expand: for dynamic branching only. Same as the max_expand argument of make(), but on a target-by-target basis. Limits the number of sub-targets created for a given target.

Formats

Specialized target formats increase efficiency and flexibility. Some allow you to save specialized objects like keras models, while others increase the speed while conserving storage and memory. You can declare target-specific formats in the plan (e.g. drake_plan(x = target(big_data_frame, format = "fst"))) or supply a global default format for all targets in make(). Either way, most formats have specialized installation requirements (e.g. R packages) that are not installed with drake by default. You will need to install them separately yourself. Available formats:

• "file": Dynamic files. To use this format, simply create local files and directories yourself and then return a character vector of paths as the target's value. Then, drake will watch for changes to those files in subsequent calls to make(). This is a more flexible alternative to file_in() and file_out(), and it is compatible with dynamic branching. See ⁠https://github.com/ropensci/drake/pull/1178⁠ for an example.

• "fst": save big data frames fast. Requires the fst package. Note: this format strips non-data-frame attributes such as the

• "fst_tbl": Like "fst", but for tibble objects. Requires the fst and tibble packages. Strips away non-data-frame non-tibble attributes.

• "fst_dt": Like "fst" format, but for data.table objects. Requires the fst and data.table packages. Strips away non-data-frame non-data-table attributes.

• "diskframe": Stores disk.frame objects, which could potentially be larger than memory. Requires the fst and disk.frame packages. Coerces objects to disk.frames. Note: disk.frame objects get moved to the drake cache (a subfolder of ⁠.drake/⁠ for most workflows). To ensure this data transfer is fast, it is best to save your disk.frame objects to the same physical storage drive as the drake cache, as.disk.frame(your_dataset, outdir = drake_tempfile()).

• "keras": save Keras models as HDF5 files. Requires the keras package.

• "qs": save any R object that can be properly serialized with the qs package. Requires the qs package. Uses qsave() and qread(). Uses the default settings in qs version 0.20.2.

• "rds": save any R object that can be properly serialized. Requires R version >= 3.5.0 due to ALTREP. Note: the "rds" format uses gzip compression, which is slow. "qs" is a superior format.

Keywords

drake_plan() understands special keyword functions for your commands. With the exception of target(), each one is a proper function with its own help file.

• target(): give the target more than just a command. Using target(), you can apply a transformation (examples: ⁠https://books.ropensci.org/drake/plans.html#large-plans⁠), # nolint supply a trigger (⁠https://books.ropensci.org/drake/triggers.html⁠), # nolint or set any number of custom columns.

• file_in(): declare an input file dependency.

• file_out(): declare an output file to be produced when the target is built.

• knitr_in(): declare a knitr file dependency such as an R Markdown (⁠*.Rmd⁠) or R LaTeX (⁠*.Rnw⁠) file.

• ignore(): force drake to entirely ignore a piece of code: do not track it for changes and do not analyze it for dependencies.

• no_deps(): tell drake to not track the dependencies of a piece of code. drake still tracks the code itself for changes.

• id_chr(): Get the name of the current target.

• drake_envir(): get the environment where drake builds targets. Intended for advanced custom memory management.

Transformations

drake has special syntax for generating large plans. Your code will look something like ⁠drake_plan(y = target(f(x), transform = map(x = c(1, 2, 3)))⁠ You can read about this interface at ⁠https://books.ropensci.org/drake/plans.html#large-plans⁠. # nolint

Static branching

In static branching, you define batches of targets based on information you know in advance. Overall usage looks like ⁠drake_plan(<x> = target(<...>, transform = <call>)⁠, where

• ⁠<x>⁠ is the name of the target or group of targets.

• ⁠<...>⁠ is optional arguments to target().

• ⁠<call>⁠ is a call to one of the transformation functions.

Transformation function usage:

• map(..., .data, .names, .id, .tag_in, .tag_out)

• split(..., slices, margin = 1L, drop = FALSE, .names, .tag_in, .tag_out) # nolint

• cross(..., .data, .names, .id, .tag_in, .tag_out)

• combine(..., .by, .names, .id, .tag_in, .tag_out)

Dynamic branching

• map(..., .trace)

• cross(..., .trace)

• group(..., .by, .trace)

map() and cross() create dynamic sub-targets from the variables supplied to the dots. As with static branching, the variables supplied to map() must all have equal length. group(f(data), .by = x) makes new dynamic sub-targets from data. Here, data can be either static or dynamic. If data is dynamic, group() aggregates existing sub-targets. If data is static, group() splits data into multiple subsets based on the groupings from .by.

Differences from static branching:

• ... must contain unnamed symbols with no values supplied, and they must be the names of targets.

• Arguments .id, .tag_in, and .tag_out no longer apply.

See Also

make, drake_config, transform_plan, map, split, cross, combine

Examples

## Not run: 
isolate_example("contain side effects", {
# For more examples, visit
# https://books.ropensci.org/drake/plans.html.

# Create drake plans:
mtcars_plan <- drake_plan(
  write.csv(mtcars[, c("mpg", "cyl")], file_out("mtcars.csv")),
  value = read.csv(file_in("mtcars.csv"))
)
if (requireNamespace("visNetwork", quietly = TRUE)) {
  plot(mtcars_plan) # fast simplified call to vis_drake_graph()
}
mtcars_plan
make(mtcars_plan) # Makes `mtcars.csv` and then `value`
head(readd(value))
# You can use knitr inputs too. See the top command below.

load_mtcars_example()
head(my_plan)
if (requireNamespace("knitr", quietly = TRUE)) {
  plot(my_plan)
}
# The `knitr_in("report.Rmd")` tells `drake` to dive into the active
# code chunks to find dependencies.
# There, `drake` sees that `small`, `large`, and `coef_regression2_small`
# are loaded in with calls to `loadd()` and `readd()`.
deps_code("report.Rmd")

# Formats are great for big data: https://github.com/ropensci/drake/pull/977
# Below, each target is 1.6 GB in memory.
# Run make() on this plan to see how much faster fst is!
n <- 1e8
plan <- drake_plan(
  data_fst = target(
    data.frame(x = runif(n), y = runif(n)),
    format = "fst"
  ),
  data_old = data.frame(x = runif(n), y = runif(n))
)

# Use transformations to generate large plans.
# Read more at
# `https://books.ropensci.org/drake/plans.html#create-large-plans-the-easy-way`. # nolint
drake_plan(
  data = target(
    simulate(nrows),
    transform = map(nrows = c(48, 64)),
    custom_column = 123
  ),
  reg = target(
    reg_fun(data),
   transform = cross(reg_fun = c(reg1, reg2), data)
  ),
  summ = target(
    sum_fun(data, reg),
   transform = cross(sum_fun = c(coef, residuals), reg)
  ),
  winners = target(
    min(summ),
    transform = combine(summ, .by = c(data, sum_fun))
  )
)

# Split data among multiple targets.
drake_plan(
  large_data = get_data(),
  slice_analysis = target(
    analyze(large_data),
    transform = split(large_data, slices = 4)
  ),
  results = target(
    rbind(slice_analysis),
    transform = combine(slice_analysis)
  )
)

# Set trace = TRUE to show what happened during the transformation process.
drake_plan(
  data = target(
    simulate(nrows),
    transform = map(nrows = c(48, 64)),
    custom_column = 123
  ),
  reg = target(
    reg_fun(data),
   transform = cross(reg_fun = c(reg1, reg2), data)
  ),
  summ = target(
    sum_fun(data, reg),
   transform = cross(sum_fun = c(coef, residuals), reg)
  ),
  winners = target(
    min(summ),
    transform = combine(summ, .by = c(data, sum_fun))
  ),
  trace = TRUE
)

# You can create your own custom columns too.
# See ?triggers for more on triggers.
drake_plan(
  website_data = target(
    command = download_data("www.your_url.com"),
    trigger = "always",
    custom_column = 5
  ),
  analysis = analyze(website_data)
)

# Tidy evaluation can help generate super large plans.
sms <- rlang::syms(letters) # To sub in character args, skip this.
drake_plan(x = target(f(char), transform = map(char = !!sms)))

# Dynamic branching
# Get the mean mpg for each cyl in the mtcars dataset.
plan <- drake_plan(
  raw = mtcars,
  group_index = raw$cyl,
  munged = target(raw[, c("mpg", "cyl")], dynamic = map(raw)),
  mean_mpg_by_cyl = target(
    data.frame(mpg = mean(munged$mpg), cyl = munged$cyl[1]),
    dynamic = group(munged, .by = group_index)
  )
)
make(plan)
readd(mean_mpg_by_cyl)
})

## End(Not run)

Show the code required to produce a given drake plan

Description

You supply a plan, and drake_plan_source() supplies code to generate that plan. If you have the prettycode package, installed, you also get nice syntax highlighting in the console when you print it.

Usage

drake_plan_source(plan)

Arguments

Value

a character vector of lines of text. This text is a call to drake_plan() that produces the plan you provide.

See Also

drake_plan()

Examples

plan <- drake::drake_plan(
  small_data = download_data("https://some_website.com"),
  large_data_raw = target(
    command = download_data("https://lots_of_data.com"),
    trigger = trigger(
      change = time_last_modified("https://lots_of_data.com"),
      command = FALSE,
      depend = FALSE
    ),
    timeout = 1e3
  )
)
print(plan)
if (requireNamespace("styler", quietly = TRUE)) {
  source <- drake_plan_source(plan)
  print(source) # Install the prettycode package for syntax highlighting.
  file <- tempfile() # Path to an R script to contain the drake_plan() call.
  writeLines(source, file) # Save the code to an R script.
}

Get the build progress of your targets

Description

Objects that drake imported, built, or attempted to build are listed as "done" or "running". Skipped objects are not listed.

Usage

drake_progress(
  ...,
  list = character(0),
  cache = drake::drake_cache(path = path),
  path = NULL,
  progress = NULL
)

Arguments

Value

The build progress of each target reached by the current make() so far.

See Also

diagnose(), drake_get_session_info(), cached(), readd(), drake_plan(), make()

Examples

## Not run: 
isolate_example("Quarantine side effects.", {
if (suppressWarnings(require("knitr"))) {
load_mtcars_example() # Get the code with drake_example("mtcars").
make(my_plan) # Run the project, build the targets.
# Watch the changing drake_progress() as make() is running.
drake_progress() # List all the targets reached so far.
drake_progress(small, large) # Just see the progress of some targets.
drake_progress(list = c("small", "large")) # Same as above.
}
})

## End(Not run)

Put quotes around each element of a character vector.

Description

Deprecated on 2019-01-01

Usage

drake_quotes(x = NULL, single = FALSE)

Arguments

Value

Character vector with quotes around it.

List running targets.

Description

List the targets that either

1. Are currently being built during a call to make(), or

2. Were in progress when make() was interrupted.

Usage

drake_running(cache = drake::drake_cache(path = path), path = NULL)

Arguments

Value

A character vector of target names.

See Also

drake_done(), drake_failed(), drake_cancelled(), drake_progress(), make()

Examples

## Not run: 
isolate_example("Quarantine side effects.", {
if (suppressWarnings(require("knitr"))) {
load_mtcars_example() # Get the code with drake_example("mtcars").
make(my_plan) # Run the project, build the targets.
drake_running() # Everything should be done.
# nolint start
# Run make() in one R session...
# slow_plan <- drake_plan(x = Sys.sleep(2))
# make(slow_plan)
# and see the progress in another session.
# drake_running()
# nolint end
}
})

## End(Not run)

Write an example ⁠_drake.R⁠ script to the current working directory.

Description

A ⁠_drake.R⁠ file is required for r_make() and friends. See the r_make() help file for details.

Usage

drake_script(code = NULL)

Arguments

Value

Nothing.

Examples

## Not run: 
isolate_example("contain side-effects", {
drake_script({
  library(drake)
  plan <- drake_plan(x = 1)
  drake_config(plan, lock_cache = FALSE)
})
cat(readLines("_drake.R"), sep = "\n")
r_make()
})

## End(Not run)

Session info of the last call to make().

Description

Deprecated. Use drake_get_session_info() instead.

Usage

drake_session(
  path = getwd(),
  search = TRUE,
  cache = drake::get_cache(path = path, search = search, verbose = verbose),
  verbose = 1L
)

Arguments

Details

Deprecated on 2018-12-06.

Value

sessionInfo() of the last call to make()

drake_settings helper

Description

List of class drake_settings.

Usage

drake_settings(
  cache_log_file = NULL,
  curl_handles = list(),
  garbage_collection = FALSE,
  jobs = 1L,
  jobs_preprocess = 1L,
  keep_going = TRUE,
  lazy_load = "eager",
  lib_loc = character(0),
  lock_cache = TRUE,
  lock_envir = TRUE,
  log_build_times = TRUE,
  log_progress = TRUE,
  memory_strategy = "speed",
  parallelism = "loop",
  recover = TRUE,
  recoverable = TRUE,
  seed = 0L,
  session_info = TRUE,
  skip_imports = FALSE,
  skip_safety_checks = FALSE,
  skip_targets = FALSE,
  sleep = function(i) 0.01,
  template = list(),
  log_worker = FALSE
)

Value

A drake_settings object.

Examples

if (FALSE) { # stronger than roxygen dontrun
drake_settings()
}

Take a strategic subset of a dataset.

Description

drake_slice() is similar to split(). Both functions partition data into disjoint subsets, but whereas split() returns all the subsets, drake_slice() returns just one. In other words, drake_slice(..., index = i) returns split(...)[[i]]. Other features: 1. drake_slice() works on vectors, data frames, matrices, lists, and arbitrary arrays. 2. Like parallel::splitIndices(), drake_slice() tries to distribute the data uniformly across subsets. See the examples to learn why splitting is useful in drake.

Usage

drake_slice(data, slices, index, margin = 1L, drop = FALSE)

Arguments

Value

A subset of data.

Examples

# Simple usage
x <- matrix(seq_len(20), nrow = 5)
x
drake_slice(x, slices = 3, index = 1)
drake_slice(x, slices = 3, index = 2)
drake_slice(x, slices = 3, index = 3)
drake_slice(x, slices = 3, margin = 2, index = 1)
# In drake, you can split a large dataset over multiple targets.
## Not run: 
isolate_example("contain side effects", {
plan <- drake_plan(
  large_data = mtcars,
  data_split = target(
    drake_slice(large_data, slices = 32, index = i),
    transform = map(i = !!seq_len(32))
  )
)
plan
cache <- storr::storr_environment()
make(plan, cache = cache, session_info = FALSE, verbose = FALSE)
readd(data_split_1L, cache = cache)
readd(data_split_2L, cache = cache)
})

## End(Not run)

Turn valid expressions into character strings.

Description

Deprecated on 2019-01-01

Usage

drake_strings(...)

Arguments

Value

A character vector.

drake tempfile

Description

Create the path to a temporary file inside drake's cache.

Usage

drake_tempfile(path = NULL, cache = drake::drake_cache(path = path))

Arguments

Details

This function is just like the tempfile() function in base R except that the path points to a special location inside drake's cache. This ensures that if the file needs to be copied to persistent storage in the cache, drake does not need to copy across physical storage media. Example: the "diskframe" format. See the "Formats" and "Columns" sections of the drake_plan() help file. Unless you supply the cache or the path to the cache (see drake_cache()) drake will assume the cache folder is named ⁠.drake/⁠ and it is located either in your working directory or an ancestor of your working directory.

See Also

drake_cache(), new_cache()

Examples

cache <- new_cache(tempfile())
# No need to supply a cache if a .drake/ folder exists.
drake_tempfile(cache = cache)
drake_plan(
  x = target(
    as.disk.frame(large_data, outdir = drake_tempfile()),
    format = "diskframe"
  )
)

Output a random tip about drake.

Description

Deprecated on 2019-01-12.

Usage

drake_tip()

Details

Tips are usually related to news and usage.

Value

A character scalar with a tip on how to use drake.

drake_triggers helper

Description

Triggers of a target.

Usage

drake_triggers(
  command = TRUE,
  depend = TRUE,
  file = TRUE,
  seed = TRUE,
  format = TRUE,
  condition = FALSE,
  change = NULL,
  mode = c("whitelist", "blacklist", "condition")
)

Arguments

Remove leading and trailing escaped quotes from character strings.

Description

Deprecated on 2019-01-01

Usage

drake_unquote(x = NULL)

Arguments

Value

Character vector without leading or trailing escaped quotes around the elements.

evaluate

Description

2019-02-15

Usage

evaluate(...)

Arguments

Use wildcard templating to create a workflow plan data frame from a template data frame.

Description

Deprecated on 2019-05-16. Use drake_plan() transformations instead. See ⁠https://books.ropensci.org/drake/plans.html#large-plans⁠ for the details.

Usage

evaluate_plan(
  plan,
  rules = NULL,
  wildcard = NULL,
  values = NULL,
  expand = TRUE,
  rename = expand,
  trace = FALSE,
  columns = "command",
  sep = "_"
)

Arguments

Details

The commands in workflow plan data frames can have wildcard symbols that can stand for datasets, parameters, function arguments, etc. These wildcards can be evaluated over a set of possible values using evaluate_plan().

Specify a single wildcard with the wildcard and values arguments. In each command, the text in wildcard will be replaced by each value in values in turn. Specify multiple wildcards with the rules argument, which overrules wildcard and values if not NULL. Here, rules should be a list with wildcards as names and vectors of possible values as list elements.

Value

A workflow plan data frame with the wildcards evaluated.

See Also

drake_plan()

example_drake

Description

2019-02-15

Usage

example_drake(...)

Arguments

examples_drake

Description

2019-02-15

Usage

examples_drake(...)

Arguments

expand

Description

2019-02-15

Usage

expand(...)

Arguments

Deprecated: create replicates of targets.

Description

Deprecated on 2019-05-16. Use drake_plan() transformations instead. See ⁠https://books.ropensci.org/drake/plans.html#large-plans⁠ for the details.

Usage

expand_plan(plan, values = NULL, rename = TRUE, sep = "_", sanitize = TRUE)

Arguments

Details

Duplicates the rows of a workflow plan data frame. Prefixes are appended to the new target names so targets still have unique names.

Value

An expanded workflow plan data frame (with replicated targets).

See Also

drake_plan()

Deprecated: expose package functions and objects for analysis with drake.

Description

Deprecated on 2020-06-24.

Usage

expose_imports(
  package,
  character_only = FALSE,
  envir = parent.frame(),
  jobs = 1
)

Arguments

Details

Deprecated. This function assigns the objects and functions from the package environment to the user's environment (usually global) so drake can watch them for changes. This used to be the standard way to make drake compatible with workflows implemented as custom analysis packages. Now, the recommendation is to supply getNamespace("yourPackage") to the envir argument of make() and friends. Read ⁠https://github.com/ropensci/drake/issues/1286⁠, especially ⁠https://github.com/ropensci/drake/issues/1286#issuecomment-649088321⁠, # nolint for details.

Value

The environment that the exposed imports are loaded into. Defaults to your R workspace.

Examples

# nolint start
## Not run: 
isolate_example("contain side effects", {
# Consider a simple plan that depends on the biglm package.
# library(biglm)
plan <- drake_plan(model = biglm(y ~ x, data = huge_dataset))
# Even if you load the biglm package, drake still ignores
# the biglm() function as a dependency. The function is missing
# from the graph:
# vis_drake_graph(plan)
# And if you install an updated version of biglm with a revised
# biglm() function, this will not cause drake::make(plan)
# to rerun the model.
# This is because biglm() is not in your environment.
# ls()
# biglm() exists in its own special package environment,
# which drake does not scan.
# ls("package:biglm")
# To depend on biglm(), use expose_imports(biglm)
# to bring the objects and functions in biglm into
# your own (non-package) environment.
# expose_imports(biglm)
# Now, the biglm() function should be in your environment.
# ls()
# biglm() now appears in the graph.
# vis_drake_graph(plan)
# And subsequent make()s respond to changes to biglm()
# and its dependencies.
})

## End(Not run)
# nolint end

List failed targets.

Description

Deprecated on 2020-03-23. Use drake_failed() instead.

Usage

failed(
  path = NULL,
  search = NULL,
  cache = drake::drake_cache(path = path),
  verbose = 1L,
  upstream_only = NULL
)

Arguments

Value

A character vector of target names.

Declare input files and directories.

Description

file_in() marks individual files (and whole directories) that your targets depend on.

Usage

file_in(...)

Arguments

Value

A character vector of declared input file or directory paths.

URLs

As of drake 7.4.0, file_in() and file_out() have support for URLs. If the file name begins with "http://", "https://", or "ftp://", make() attempts to check the ETag to see if the data changed from last time. If no ETag can be found, drake simply uses the ETag from last make() and registers the file as unchanged (which prevents your workflow from breaking if you lose internet access). If your file_in() URLs require authentication, see the curl_handles argument of make() and drake_config() to learn how to supply credentials.

Keywords

drake_plan() understands special keyword functions for your commands. With the exception of target(), each one is a proper function with its own help file.

• target(): give the target more than just a command. Using target(), you can apply a transformation (examples: ⁠https://books.ropensci.org/drake/plans.html#large-plans⁠), # nolint supply a trigger (⁠https://books.ropensci.org/drake/triggers.html⁠), # nolint or set any number of custom columns.

• file_in(): declare an input file dependency.

• file_out(): declare an output file to be produced when the target is built.

• knitr_in(): declare a knitr file dependency such as an R Markdown (⁠*.Rmd⁠) or R LaTeX (⁠*.Rnw⁠) file.

• ignore(): force drake to entirely ignore a piece of code: do not track it for changes and do not analyze it for dependencies.

• no_deps(): tell drake to not track the dependencies of a piece of code. drake still tracks the code itself for changes.

• id_chr(): Get the name of the current target.

• drake_envir(): get the environment where drake builds targets. Intended for advanced custom memory management.

See Also

file_out(), knitr_in(), ignore(), no_deps()

Examples

## Not run: 
isolate_example("contain side effects", {
# The `file_out()` and `file_in()` functions
# just takes in strings and returns them.
file_out("summaries.txt")
# Their main purpose is to orchestrate your custom files
# in your workflow plan data frame.
plan <- drake_plan(
  out = write.csv(mtcars, file_out("mtcars.csv")),
  contents = read.csv(file_in("mtcars.csv"))
)
plan
# drake knows "\"mtcars.csv\"" is the first target
# and a dependency of `contents`. See for yourself:

make(plan)
file.exists("mtcars.csv")

# You may use `.id_chr` inside `file_out()` and `file_in()`
# to refer  to the current target. This works inside
# static `map()`, `combine()`, `split()`, and `cross()`.

plan <- drake::drake_plan(
  data = target(
    write.csv(data, file_out(paste0(.id_chr, ".csv"))),
    transform = map(data = c(airquality, mtcars))
  )
)
plan

# You can also work with entire directories this way.
# However, in `file_out("your_directory")`, the directory
# becomes an entire unit. Thus, `file_in("your_directory")`
# is more appropriate for subsequent steps than
# `file_in("your_directory/file_inside.txt")`.
plan <- drake_plan(
  out = {
    dir.create(file_out("dir"))
    write.csv(mtcars, "dir/mtcars.csv")
  },
  contents = read.csv(file.path(file_in("dir"), "mtcars.csv"))
)
plan

make(plan)
file.exists("dir/mtcars.csv")

# See the connections that the file relationships create:
if (requireNamespace("visNetwork", quietly = TRUE)) {
  vis_drake_graph(plan)
}
})

## End(Not run)

Declare output files and directories.

Description

file_out() marks individual files (and whole directories) that your targets create.

Usage

file_out(...)

Arguments

Value

A character vector of declared output file or directory paths.

Keywords

drake_plan() understands special keyword functions for your commands. With the exception of target(), each one is a proper function with its own help file.

• target(): give the target more than just a command. Using target(), you can apply a transformation (examples: ⁠https://books.ropensci.org/drake/plans.html#large-plans⁠), # nolint supply a trigger (⁠https://books.ropensci.org/drake/triggers.html⁠), # nolint or set any number of custom columns.

• file_in(): declare an input file dependency.

• file_out(): declare an output file to be produced when the target is built.

• knitr_in(): declare a knitr file dependency such as an R Markdown (⁠*.Rmd⁠) or R LaTeX (⁠*.Rnw⁠) file.

• ignore(): force drake to entirely ignore a piece of code: do not track it for changes and do not analyze it for dependencies.

• no_deps(): tell drake to not track the dependencies of a piece of code. drake still tracks the code itself for changes.

• id_chr(): Get the name of the current target.

• drake_envir(): get the environment where drake builds targets. Intended for advanced custom memory management.

See Also

file_in(), knitr_in(), ignore(), no_deps()

Examples

## Not run: 
isolate_example("contain side effects", {
# The `file_out()` and `file_in()` functions
# just takes in strings and returns them.
file_out("summaries.txt")
# Their main purpose is to orchestrate your custom files
# in your workflow plan data frame.
plan <- drake_plan(
  out = write.csv(mtcars, file_out("mtcars.csv")),
  contents = read.csv(file_in("mtcars.csv"))
)
plan
# drake knows "\"mtcars.csv\"" is the first target
# and a dependency of `contents`. See for yourself:

make(plan)
file.exists("mtcars.csv")

 # You may use `.id_chr` inside `file_out()` and `file_in()`
 # to refer  to the current target. This works inside `map()`,
 # `combine()`, `split()`, and `cross()`.

plan <- drake::drake_plan(
  data = target(
    write.csv(data, file_out(paste0(.id_chr, ".csv"))),
    transform = map(data = c(airquality, mtcars))
  )
)

plan

# You can also work with entire directories this way.
# However, in `file_out("your_directory")`, the directory
# becomes an entire unit. Thus, `file_in("your_directory")`
# is more appropriate for subsequent steps than
# `file_in("your_directory/file_inside.txt")`.
plan <- drake_plan(
  out = {
    dir.create(file_out("dir"))
    write.csv(mtcars, "dir/mtcars.csv")
  },
  contents = read.csv(file.path(file_in("dir"), "mtcars.csv"))
)
plan

make(plan)
file.exists("dir/mtcars.csv")

# See the connections that the file relationships create:
if (requireNamespace("visNetwork", quietly = TRUE)) {
  vis_drake_graph(plan)
}
})

## End(Not run)

Show a file's encoded representation in the cache

Description

This function simply wraps literal double quotes around the argument x so drake knows it is the name of a file. Use when you are calling functions like deps_code(): for example, deps_code(file_store("report.md")). See the examples for details. Internally, drake wraps the names of file targets/imports inside literal double quotes to avoid confusion between files and generic R objects.

Usage

file_store(x)

Arguments

Value

A single-quoted character string: i.e., a filename understandable by drake.

Examples

# Wraps the string in single quotes.
file_store("my_file.rds") # "'my_file.rds'"
## Not run: 
isolate_example("contain side effects", {
if (suppressWarnings(require("knitr"))) {
load_mtcars_example() # Get the code with drake_example("mtcars").
make(my_plan) # Run the workflow to build the targets
list.files() # Should include input "report.Rmd" and output "report.md".
head(readd(small)) # You can use symbols for ordinary objects.
# But if you want to read cached info on files, use `file_store()`.
readd(file_store("report.md"), character_only = TRUE) # File fingerprint.
deps_code(file_store("report.Rmd"))
config <- drake_config(my_plan)
deps_profile(
  file_store("report.Rmd"),
  plan = my_plan,
  character_only = TRUE
)
}
})

## End(Not run)

Search up the file system for the nearest drake cache.

Description

Only works if the cache is a file system in a hidden folder named ⁠.drake/⁠ (default).

Usage

find_cache(path = getwd(), dir = NULL, directory = NULL)

Arguments

Value

File path of the nearest drake cache or NULL if no cache is found.

See Also

drake_plan(), make(),

Examples

## Not run: 
isolate_example("Quarantine side effects.", {
if (suppressWarnings(require("knitr"))) {
load_mtcars_example() # Get the code with drake_example("mtcars").
make(my_plan) # Run the project, build the target.
# Find the file path of the project's cache.
# Search up through parent directories if necessary.
find_cache()
}
})

## End(Not run)

find_knitr_doc

Description

2019-02-15

Usage

find_knitr_doc(...)

Arguments

Search up the file system for the nearest root path of a drake project.

Description

Deprecated on 2019-01-08.

Usage

find_project(path = getwd())

Arguments

Details

Only works if the cache is a file system in a folder named .drake (default).

Value

File path of the nearest drake project or NULL if no drake project is found.

See Also

drake_plan(), make()

from_plan

Description

The from_plan() function is now defunct in order to reduce the demands on memory usage.

Usage

from_plan(column)

Arguments

Details

2019-03-28

See Also

drake_envir()

Task passed to individual futures in the "future" backend

Description

For internal use only. Only exported to make available to futures.

Usage

future_build(target, meta, config, spec, config_tmp, protect)

Arguments

Value

Either the target value or a list of build results.

gather

Description

2019-02-15

Usage

gather(...)

Arguments

Gather multiple groupings of targets

Description

Deprecated on 2019-05-16. Use drake_plan() transformations instead. See ⁠https://books.ropensci.org/drake/plans.html#large-plans⁠ for the details.

Usage

gather_by(
  plan,
  ...,
  prefix = "target",
  gather = "list",
  append = TRUE,
  filter = NULL,
  sep = "_"
)

Arguments

Details

Perform several calls to gather_plan() based on groupings from columns in the plan, and then row-bind the new targets to the plan.

Value

A workflow plan data frame.

See Also

drake_plan()

Combine targets

Description

Deprecated on 2019-05-16. Use drake_plan() transformations instead. See ⁠https://books.ropensci.org/drake/plans.html#large-plans⁠ for the details.

Usage

gather_plan(plan = NULL, target = "target", gather = "list", append = FALSE)

Arguments

Details

Creates a new workflow plan to aggregate existing targets in the supplied plan.

Value

A workflow plan data frame that aggregates multiple prespecified targets into one additional target downstream.

See Also

drake_plan()

The default cache of a drake project.

Description

Use drake_cache() instead.

Usage

get_cache(
  path = getwd(),
  search = TRUE,
  verbose = 1L,
  force = FALSE,
  fetch_cache = NULL,
  console_log_file = NULL
)

Arguments

Details

Deprecated on 2019-05-25.

Deprecated, get a trace of a dynamic target's value.

Description

Deprecated on 2019-12-10. Use read_trace() instead.

Usage

get_trace(trace, value)

Arguments

Value

The dynamic trace of one target in another: a vector of values from a grouping variable.

Name of the current target

Description

id_chr() gives you the name of the current target while make() is running. For static branching in drake_plan(), use the .id_chr symbol instead. See the examples for details.

Usage

id_chr()

Value

The name of the current target.

Keywords

drake_plan() understands special keyword functions for your commands. With the exception of target(), each one is a proper function with its own help file.

• target(): give the target more than just a command. Using target(), you can apply a transformation (examples: ⁠https://books.ropensci.org/drake/plans.html#large-plans⁠), # nolint supply a trigger (⁠https://books.ropensci.org/drake/triggers.html⁠), # nolint or set any number of custom columns.

• file_in(): declare an input file dependency.

• file_out(): declare an output file to be produced when the target is built.

• knitr_in(): declare a knitr file dependency such as an R Markdown (⁠*.Rmd⁠) or R LaTeX (⁠*.Rnw⁠) file.

• ignore(): force drake to entirely ignore a piece of code: do not track it for changes and do not analyze it for dependencies.

• no_deps(): tell drake to not track the dependencies of a piece of code. drake still tracks the code itself for changes.

• id_chr(): Get the name of the current target.

• drake_envir(): get the environment where drake builds targets. Intended for advanced custom memory management.

Examples

try(id_chr()) # Do not use outside the plan.
## Not run: 
isolate_example("id_chr()", {
plan <- drake_plan(x = id_chr())
make(plan)
readd(x)
# Dynamic branching
plan <- drake_plan(
  x = seq_len(4),
  y = target(id_chr(), dynamic = map(x))
)
make(plan)
readd(y, subtargets = 1)
# Static branching
plan <- drake_plan(
  y = target(c(x, .id_chr), transform = map(x = !!seq_len(4)))
)
plan
})

## End(Not run)

Ignore code

Description

Ignore sections of commands and imported functions.

Usage

ignore(x = NULL)

Arguments

Details

In user-defined functions and drake_plan() commands, you can wrap code chunks in ignore() to

1. Tell drake to not search for dependencies (targets etc. mentioned in the code) and

2. Ignore changes to the code so downstream targets remain up to date. To enforce (1) without (2), use no_deps().

Value

The argument.

Keywords

drake_plan() understands special keyword functions for your commands. With the exception of target(), each one is a proper function with its own help file.

• target(): give the target more than just a command. Using target(), you can apply a transformation (examples: ⁠https://books.ropensci.org/drake/plans.html#large-plans⁠), # nolint supply a trigger (⁠https://books.ropensci.org/drake/triggers.html⁠), # nolint or set any number of custom columns.

• file_in(): declare an input file dependency.

• file_out(): declare an output file to be produced when the target is built.

• knitr_in(): declare a knitr file dependency such as an R Markdown (⁠*.Rmd⁠) or R LaTeX (⁠*.Rnw⁠) file.

• ignore(): force drake to entirely ignore a piece of code: do not track it for changes and do not analyze it for dependencies.

• no_deps(): tell drake to not track the dependencies of a piece of code. drake still tracks the code itself for changes.

• id_chr(): Get the name of the current target.

• drake_envir(): get the environment where drake builds targets. Intended for advanced custom memory management.

See Also

file_in(), file_out(), knitr_in(), no_deps()

Examples

## Not run: 
isolate_example("Contain side effects", {
# Normally, `drake` reacts to changes in dependencies.
x <- 4
make(plan = drake_plan(y = sqrt(x)))
x <- 5
make(plan = drake_plan(y = sqrt(x)))
make(plan = drake_plan(y = sqrt(4) + x))
# But not with ignore().
make(plan = drake_plan(y = sqrt(4) + ignore(x))) # Builds y.
x <- 6
make(plan = drake_plan(y = sqrt(4) + ignore(x))) # Skips y.
make(plan = drake_plan(y = sqrt(4) + ignore(x + 1))) # Skips y.

# ignore() works with functions and multiline code chunks.
f <- function(x) {
  ignore({
    x <- x + 1
    x <- x + 2
  })
  x # Not ignored.
}
make(plan = drake_plan(y = f(2)))
readd(x)
# Changes the content of the ignore() block:
f <- function(x) {
  ignore({
    x <- x + 1
  })
  x # Not ignored.
}
make(plan = drake_plan(x = f(2)))
readd(x)
})

## End(Not run)

List all the imports in the drake cache.

Description

Deprecated on 2019-01-08.

Usage

imported(
  files_only = FALSE,
  path = getwd(),
  search = TRUE,
  cache = drake::get_cache(path = path, search = search, verbose = verbose),
  verbose = 1L,
  jobs = 1
)

Arguments

Details

An import is a non-target object processed by make(). Targets in the workflow plan data frame (see drake_config() may depend on imports.

Value

Character vector naming the imports in the cache.

See Also

cached(), loadd()

List the targets in progress

Description

Deprecated on 2019-01-13.

Usage

in_progress(
  path = getwd(),
  search = TRUE,
  cache = drake::get_cache(path = path, search = search, verbose = verbose),
  verbose = 1L
)

Arguments

Details

Similar to progress().

Value

A character vector of target names.

See Also

diagnose(), drake_get_session_info(), cached(), readd(), drake_plan(), make()

is_function_call

Description

2019-02-15

Usage

is_function_call(...)

Arguments

Isolate the side effects of an example.

Description

Runs code in a temporary directory in a controlled environment with a controlled set of options.

Usage

isolate_example(desc, code)

Arguments

Value

Nothing.

Dependencies of a knitr report

Description

Deprecated on 2019-02-14 knit("your_report.Rmd") or knit("your_report.Rmd", quiet = TRUE).

Usage

knitr_deps(target)

Arguments

Value

Data frame of dependencies

Declare knitr/rmarkdown source files as dependencies.

Description

knitr_in() marks individual knitr/R Markdown reports as dependencies. In drake, these reports are pieces of the pipeline. R Markdown is a great tool for displaying precomputed results, but not for running a large workflow from end to end. These reports should do as little computation as possible.

Usage

knitr_in(...)

Arguments

Details

Unlike file_in() and file_out(), knitr_in() does not work with entire directories.

Value

A character vector of declared input file paths.

Keywords

drake_plan() understands special keyword functions for your commands. With the exception of target(), each one is a proper function with its own help file.

• target(): give the target more than just a command. Using target(), you can apply a transformation (examples: ⁠https://books.ropensci.org/drake/plans.html#large-plans⁠), # nolint supply a trigger (⁠https://books.ropensci.org/drake/triggers.html⁠), # nolint or set any number of custom columns.

• file_in(): declare an input file dependency.

• file_out(): declare an output file to be produced when the target is built.

• knitr_in(): declare a knitr file dependency such as an R Markdown (⁠*.Rmd⁠) or R LaTeX (⁠*.Rnw⁠) file.

• ignore(): force drake to entirely ignore a piece of code: do not track it for changes and do not analyze it for dependencies.

• no_deps(): tell drake to not track the dependencies of a piece of code. drake still tracks the code itself for changes.

• id_chr(): Get the name of the current target.

• drake_envir(): get the environment where drake builds targets. Intended for advanced custom memory management.

See Also

file_in(), file_out(), ignore(), no_deps()

Examples

## Not run: 
isolate_example("contain side effects", {
if (requireNamespace("knitr", quietly = TRUE)) {
# `knitr_in()` is like `file_in()`
# except that it analyzes active code chunks in your `knitr`
# source file and detects non-file dependencies.
# That way, updates to the right dependencies trigger rebuilds
# in your report.
# The mtcars example (`drake_example("mtcars")`)
# already has a demonstration

load_mtcars_example()
make(my_plan)

# Now how did drake magically know that
# `small`, `large`, and `coef_regression2_small` were
# dependencies of the output file `report.md`?
# because the command in the workflow plan had
# `knitr_in("report.Rmd")` in it, so drake knew
# to analyze the active code chunks. There, it spotted
# where `small`, `large`, and `coef_regression2_small`
# were read from the cache using calls to `loadd()` and `readd()`.
}
})

## End(Not run)

Create the nodes data frame used in the legend of the graph visualizations.

Description

Output a visNetwork-friendly data frame of nodes. It tells you what the colors and shapes mean in the graph visualizations.

Usage

legend_nodes(font_size = 20)

Arguments

Value

A data frame of legend nodes for the graph visualizations.

Examples

## Not run: 
# Show the legend nodes used in graph visualizations.
# For example, you may want to inspect the color palette more closely.
if (requireNamespace("visNetwork", quietly = TRUE)) {
# visNetwork::visNetwork(nodes = legend_nodes()) # nolint
}

## End(Not run)

load_basic_example

Description

2019-02-15

Usage

load_basic_example(...)

Arguments

Load the main example.

Description

The main example lives at ⁠https://github.com/wlandau/drake-examples/tree/main/main⁠. Use drake_example("main") to download its code. This function also writes/overwrites the files report.Rmd and raw_data.xlsx.

Usage

load_main_example(
  envir = parent.frame(),
  report_file = "report.Rmd",
  overwrite = FALSE,
  force = FALSE
)

Arguments

Details

Deprecated 2018-12-31.

Value

A drake_config() configuration list.

Load the mtcars example.

Description

Is there an association between the weight and the fuel efficiency of cars? To find out, we use the mtcars example from drake_example("mtcars"). The mtcars dataset itself only has 32 rows, so we generate two larger bootstrapped datasets and then analyze them with regression models. Finally, we summarize the regression models to see if there is an association.

Usage

load_mtcars_example(
  envir = parent.frame(),
  report_file = NULL,
  overwrite = FALSE,
  force = FALSE
)

Arguments

Details

Use drake_example("mtcars") to get the code for the mtcars example. This function also writes/overwrites the file, report.Rmd.

Value

Nothing.

See Also

clean_mtcars_example() drake_examples()

Examples

## Not run: 
isolate_example("Quarantine side effects.", {
if (suppressWarnings(require("knitr"))) {
# Populate your workspace and write 'report.Rmd'.
load_mtcars_example() # Get the code: drake_example("mtcars")
# Check the dependencies of an imported function.
deps_code(reg1)
# Check the dependencies of commands in the workflow plan.
deps_code(my_plan$command[1])
deps_code(my_plan$command[4])
# Plot the interactive network visualization of the workflow.
outdated(my_plan) # Which targets are out of date?
# Run the workflow to build all the targets in the plan.
make(my_plan)
outdated(my_plan) # Everything should be up to date.
# For the reg2() model on the small dataset,
# the p-value is so small that there may be an association
# between weight and fuel efficiency after all.
readd(coef_regression2_small)
# Clean up the example.
clean_mtcars_example()
}
})

## End(Not run)

drake now has just one hash algorithm per cache.

Description

Deprecated on 2018-12-12

Usage

long_hash(cache = drake::get_cache(verbose = verbose), verbose = 1L)

Arguments

Value

A character vector naming a hash algorithm.

Run your project (build the outdated targets).

Description

This is the central, most important function of the drake package. It runs all the steps of your workflow in the correct order, skipping any work that is already up to date. Because of how make() tracks global functions and objects as dependencies of targets, please restart your R session so the pipeline runs in a clean reproducible environment.

Usage

make(
  plan,
  targets = NULL,
  envir = parent.frame(),
  verbose = 1L,
  hook = NULL,
  cache = drake::drake_cache(),
  fetch_cache = NULL,
  parallelism = "loop",
  jobs = 1L,
  jobs_preprocess = 1L,
  packages = rev(.packages()),
  lib_loc = NULL,
  prework = character(0),
  prepend = NULL,
  command = NULL,
  args = NULL,
  recipe_command = NULL,
  log_progress = TRUE,
  skip_targets = FALSE,
  timeout = NULL,
  cpu = Inf,
  elapsed = Inf,
  retries = 0,
  force = FALSE,
  graph = NULL,
  trigger = drake::trigger(),
  skip_imports = FALSE,
  skip_safety_checks = FALSE,
  config = NULL,
  lazy_load = "eager",
  session_info = NULL,
  cache_log_file = NULL,
  seed = NULL,
  caching = "main",
  keep_going = FALSE,
  session = NULL,
  pruning_strategy = NULL,
  makefile_path = NULL,
  console_log_file = NULL,
  ensure_workers = NULL,
  garbage_collection = FALSE,
  template = list(),
  sleep = function(i) 0.01,
  hasty_build = NULL,
  memory_strategy = "speed",
  layout = NULL,
  spec = NULL,
  lock_envir = NULL,
  history = TRUE,
  recover = FALSE,
  recoverable = TRUE,
  curl_handles = list(),
  max_expand = NULL,
  log_build_times = TRUE,
  format = NULL,
  lock_cache = TRUE,
  log_make = NULL,
  log_worker = FALSE
)

Arguments

Value

nothing

Interactive mode

In interactive sessions, consider r_make(), r_outdated(), etc. rather than make(), outdated(), etc. The ⁠r_*()⁠ drake functions are more reproducible when the session is interactive. If you do run make() interactively, please restart your R session beforehand so your functions and global objects get loaded into a clean reproducible environment. This prevents targets from getting invalidated unexpectedly.

A serious drake workflow should be consistent and reliable, ideally with the help of a main R script. This script should begin in a fresh R session, load your packages and functions in a dependable manner, and then run make(). Example: ⁠https://github.com/wlandau/drake-examples/tree/main/gsp⁠. Batch mode, especially within a container, is particularly helpful.

Interactive R sessions are still useful, but they easily grow stale. Targets can falsely invalidate if you accidentally change a function or data object in your environment.

Self-invalidation

It is possible to construct a workflow that tries to invalidate itself. Example:

plan <- drake_plan(
  x = {
    data(mtcars)
    mtcars$mpg
  },
  y = mean(x)
)

Here, because data() loads mtcars into the global environment, the very act of building x changes the dependencies of x. In other words, without safeguards, x would not be up to date at the end of make(plan). Please try to avoid workflows that modify the global environment. Functions such as data() belong in your setup scripts prior to make(), not in any functions or commands that get called during make() itself.

For each target that is still problematic (e.g. ⁠https://github.com/rstudio/gt/issues/297⁠) you can safely run the command in its own special callr::r() process. Example: ⁠https://github.com/rstudio/gt/issues/297#issuecomment-497778735⁠. # nolint

Cache locking

When make() runs, it locks the cache so other processes cannot modify it. Same goes for outdated(), vis_drake_graph(), and similar functions when make_imports = TRUE. This is a safety measure to prevent simultaneous processes from corrupting the cache. If you get an error saying that the cache is locked, either set make_imports = FALSE or manually force unlock it with drake_cache()$unlock().

See Also

drake_plan(), drake_config(), vis_drake_graph(), outdated()

Examples

## Not run: 
isolate_example("Quarantine side effects.", {
if (suppressWarnings(require("knitr"))) {
load_mtcars_example() # Get the code with drake_example("mtcars").
config <- drake_config(my_plan)
outdated(my_plan) # Which targets need to be (re)built?
make(my_plan) # Build what needs to be built.
outdated(my_plan) # Everything is up to date.
# Change one of your imported function dependencies.
reg2 = function(d) {
  d$x3 = d$x^3
  lm(y ~ x3, data = d)
}
outdated(my_plan) # Some targets depend on reg2().
make(my_plan) # Rebuild just the outdated targets.
outdated(my_plan) # Everything is up to date again.
if (requireNamespace("visNetwork", quietly = TRUE)) {
vis_drake_graph(my_plan) # See how they fit in an interactive graph.
make(my_plan, cache_log_file = TRUE) # Write a CSV log file this time.
vis_drake_graph(my_plan) # The colors changed in the graph.
# Run targets in parallel:
# options(clustermq.scheduler = "multicore") # nolint
# make(my_plan, parallelism = "clustermq", jobs = 2) # nolint
}
clean() # Start from scratch next time around.
}
# Dynamic branching
# Get the mean mpg for each cyl in the mtcars dataset.
plan <- drake_plan(
  raw = mtcars,
  group_index = raw$cyl,
  munged = target(raw[, c("mpg", "cyl")], dynamic = map(raw)),
  mean_mpg_by_cyl = target(
    data.frame(mpg = mean(munged$mpg), cyl = munged$cyl[1]),
    dynamic = group(munged, .by = group_index)
  )
)
make(plan)
readd(mean_mpg_by_cyl)
})

## End(Not run)

Internal function with a drake_config() argument

Description

Not a user-side function.

Usage

make_impl(config)

Arguments

Just process the imports

Description

Deprecated on 2019-01-04

Usage

make_imports(config)

Arguments

Value

nothing

See Also

make(), drake_config()

Just make the targets

Description

Deprecated on 2019-01-04

Usage

make_targets(config)

Arguments

Value

nothing

See Also

make(), drake_config()

Apply make() with a pre-computed config object

Description

Deprecated on 2019-01-04

Usage

make_with_config(config)

Arguments

Value

nothing

See Also

make(), drake_config()

Manage the in-memory dependencies of a target.

Description

Load/unload a target's dependencies. Not a user-side function.

Usage

manage_memory(target, config, downstream = NULL, jobs = 1)

Arguments

Value

Nothing.

Create a plan that maps a function to a grid of arguments.

Description

Deprecated on 2019-05-16. Use drake_plan() transformations instead. See ⁠https://books.ropensci.org/drake/plans.html#large-plans⁠ for the details.

Usage

map_plan(args, fun, id = "id", character_only = FALSE, trace = FALSE)

Arguments

Details

map_plan() is like base::Map(): it takes a function name and a grid of arguments, and writes out all the commands calls to apply the function to each row of arguments.

Value

A workflow plan data frame.

See Also

drake_plan()

max_useful_jobs

Description

2019-05-16

Usage

max_useful_jobs(...)

Arguments

migrate_drake_project

Description

2019-05-16

Usage

migrate_drake_project(...)

Arguments

Report any import objects required by your drake_plan plan but missing from your workspace or file system.

Description

Checks your workspace/environment and file system.

Usage

missed(..., config = NULL)

Arguments

Value

Character vector of names of missing objects and files.

See Also

outdated()

Examples

## Not run: 
isolate_example("Quarantine side effects.", {
if (suppressWarnings(require("knitr"))) {
plan <- drake_plan(x = missing::fun(arg))
missed(plan)
}
})

## End(Not run)

Internal function with a drake_config() argument

Description

Not a user-side function.

Usage

missed_impl(config)

Arguments

Make a new drake cache.

Description

Uses the storr::storr_rds() function from the storr package.

Usage

new_cache(
  path = NULL,
  verbose = NULL,
  type = NULL,
  hash_algorithm = NULL,
  short_hash_algo = NULL,
  long_hash_algo = NULL,
  ...,
  console_log_file = NULL
)

Arguments

Value

A newly created drake cache as a storr object.

See Also

make()

Examples

## Not run: 
isolate_example("Quarantine new_cache() side effects.", {
clean(destroy = TRUE) # Should not be necessary.
unlink("not_hidden", recursive = TRUE) # Should not be necessary.
cache1 <- new_cache() # Creates a new hidden '.drake' folder.
cache2 <- new_cache(path = "not_hidden", hash_algorithm = "md5")
clean(destroy = TRUE, cache = cache2)
})

## End(Not run)

drake_deps constructor

Description

List of class drake_deps.

Usage

new_drake_deps(
  globals = character(0),
  namespaced = character(0),
  strings = character(0),
  loadd = character(0),
  readd = character(0),
  file_in = character(0),
  file_out = character(0),
  knitr_in = character(0)
)

Arguments