Package 'ssdsims'

Description

Activates the dqrng pcg64 RNG backend for the duration of the calling frame, then resets it when .local_envir exits. While active, base R's runif(), rnorm(), rbinom(), rexp(), rgamma(), rpois(), sample.int(), and sample() (and therefore dplyr::slice_sample() and ⁠ssdtools::ssd_r*()⁠) draw from dqrng's pcg64, seeded via dqrng::dqset.seed(). pcg64 is forced explicitly because it accepts the length-2 stream argument the per-task primer design relies on; dqrng's own default (⁠Xoroshiro128++⁠) does not.

Usage

local_dqrng_backend(.local_envir = parent.frame())

Arguments

Details

Registering the backend is a process-global side effect that also advances base R's .Random.seed. local_dqrng_backend() follows the withr convention (compare withr::local_seed()): it pairs activation with deferred reset so the backend is always restored, including on error.

The helper is reentrant. dqrng::register_methods() / dqrng::restore_methods() keep a single global save-slot, so a nested reset would tear the backend down for the still-open outer scope. To avoid this, a local_dqrng_backend() call made while the backend is already active is a no-op: it does not re-activate the backend and schedules no further reset. Only the outermost call activates the backend on entry and resets it on exit, so the RNG stream is identical whether or not a nested call occurs.

Value

Invisibly returns TRUE if this call activated the backend (the outermost scope) or FALSE if the backend was already active and the call was a no-op.

See Also

withr::local_seed(), dqrng::dqset.seed().

Examples

local_dqrng_backend()
dqrng::dqset.seed(42, stream = c(1L, 2L))
runif(3)

Description

local_dqrng_state() installs a per-task ⁠(seed, primer)⁠ starting point as the running dqrng RNG state via dqrng::dqset.seed(), restoring the previous state when .local_envir exits. with_dqrng_state() evaluates code with that state installed, then restores the previous state. The primer argument is the per-task primer (the value handed to dqrng's stream argument, per TARGETS-DESIGN.md §2 and the GLOSSARY); the ⁠_state⁠ suffix marks that the wrapper installs that primer as the running RNG state.

Usage

local_dqrng_state(seed, primer, .local_envir = parent.frame())

with_dqrng_state(seed, primer, code)

Arguments

Details

These are the dqrng-path analogues of local_lecuyer_cmrg_state() / with_lecuyer_cmrg_state(). Like those helpers they snapshot the RNG state on entry (via dqrng::dqrng_get_state()) and withr::defer() a restore (via dqrng::dqrng_set_state()), so a call leaves the surrounding RNG stream undisturbed, including on error.

Both require an active dqrng backend: they abort unless a local_dqrng_backend() scope is open. This fails fast rather than silently seeding base R's Mersenne-Twister.

Value

local_dqrng_state() invisibly returns primer; with_dqrng_state() returns the value of code.

See Also

withr::local_seed(), local_dqrng_backend(), local_lecuyer_cmrg_state().

Examples

local_dqrng_backend()
local_dqrng_state(42, c(1L, 2L))
runif(3)

with_dqrng_state(42, c(1L, 2L), runif(3))

Description

local_lecuyer_cmrg_seed() seeds the L'Ecuyer-CMRG RNG with a scalar integer via base::set.seed(), restoring the previous state when .local_envir exits. with_lecuyer_cmrg_seed() evaluates code with that seed in effect, then restores the previous state. For a .Random.seed-style state vector (e.g. from get_lecuyer_cmrg_stream_state() or parallel::nextRNGStream()) use local_lecuyer_cmrg_state() / with_lecuyer_cmrg_state().

Usage

local_lecuyer_cmrg_seed(seed, .local_envir = parent.frame())

with_lecuyer_cmrg_seed(seed, code)

Arguments

Value

with_lecuyer_cmrg_seed() returns the value of code.

See Also

withr::local_seed(), local_lecuyer_cmrg_state(), parallel::nextRNGStream().

Examples

local_lecuyer_cmrg_seed(42)
runif(3)

with_lecuyer_cmrg_seed(42, {
  runif(3)
})

Description

local_lecuyer_cmrg_state() sets the L'Ecuyer-CMRG RNG state to a .Random.seed-style integer vector (length 7) by assigning to .Random.seed directly, restoring the previous state when .local_envir exits. with_lecuyer_cmrg_state() evaluates code with that state in effect, then restores the previous state. A state is the full internal RNG state (as returned by parallel::nextRNGStream() or get_lecuyer_cmrg_stream_state()); contrast with base::set.seed() which takes a scalar seed (see local_lecuyer_cmrg_seed() / with_lecuyer_cmrg_seed()).

Usage

local_lecuyer_cmrg_state(state, .local_envir = parent.frame())

with_lecuyer_cmrg_state(state, code)

Arguments

Value

local_lecuyer_cmrg_state() invisibly returns state; with_lecuyer_cmrg_state() returns the value of code.

See Also

parallel::nextRNGStream(), local_lecuyer_cmrg_seed().

Examples

state <- with_lecuyer_cmrg_seed(42, parallel::nextRNGStream(.Random.seed))
local_lecuyer_cmrg_state(state)
runif(3)

with_lecuyer_cmrg_state(state, runif(3))

Description

Returns the validated, materialised dataset tibble stored on scenario under name. The dataset was validated (a numeric Conc column) and materialised at construction by ssd_define_scenario(), so this accessor performs no registration, persistence, or re-validation - it just isolates the value a shard body fits. Aborts with an informative error when name is not one of the scenario's datasets.

Usage

scenario_dataset(scenario, name)

Arguments

Details

Names - not values - drive task hashing (TARGETS-DESIGN.md section 1.1): the task path carries the dataset name and this accessor resolves it back to the tibble at run time, so the tibble never enters a task identity.

Value

The materialised dataset tibble stored under name.

See Also

scenario_min_pmix() for the min_pmix counterpart.

Examples

scenario <- ssd_define_scenario(ssddata::ccme_boron, nsim = 1L, seed = 42L)
scenario_dataset(scenario, "ccme_boron")

Description

Returns the single-argument min_pmix function materialised on scenario under name. ssd_define_scenario() resolves each min_pmix reference to a function once, at construction (so a cluster worker needs no shared interactive environment), and stores it keyed by name; this accessor isolates it. Aborts with an informative error when name is not one of the scenario's min_pmix names.

Usage

scenario_min_pmix(scenario, name)

Arguments

Details

Names - not function values - drive task hashing (TARGETS-DESIGN.md section 1.1): the fit-task path carries the min_pmix name, and this accessor resolves it back to the function at run time, so the function value never enters a task identity (no byte-stability concern from byte-compilation or captured environments).

Value

The single-argument min_pmix function stored under name.

See Also

scenario_dataset() for the dataset counterpart.

Examples

scenario <- ssd_define_scenario(ssddata::ccme_boron, nsim = 1L, seed = 42L)
scenario_min_pmix(scenario, "ssd_min_pmix")

Description

Returns ⁠<root>/layout=<hash>⁠, where the hash is derived from the scenario's partition_by. A step's Hive shard path depth and axes are a function of partition_by/bundle, so writing two different layouts into one root would leave shards of different granularity side by side - and the depth-agnostic glob the readers use (⁠<step>/**/part.parquet⁠) would then union stale and current shards, double-counting tasks. Keying the results root on the layout isolates each partition_by into its own subtree: re-running a scenario with a changed partition_by/bundle writes to a fresh root (never mixing granularities), while re-running the same layout reuses the root (idempotent and cache-friendly - the same shard paths are simply rewritten).

Usage

scenario_results_dir(scenario, root = "results")

Arguments

Details

The targets pipeline writes under this root (see the shipped ⁠_targets.R⁠ template). The single-core ssd_run_scenario_shards() takes the complementary approach: it owns and clears a fixed dir on each run.

Value

The layout-keyed path ⁠file.path(root, paste0("layout=", <hash>))⁠.

See Also

ssd_run_scenario_shards(), ssd_summarise().

Examples

scenario <- ssd_define_scenario(ssddata::ccme_boron, nsim = 1L, seed = 42L)
scenario_results_dir(scenario)

Description

Runs a small, fixed benchmark sweep on the current architecture - tiny nboot values, all ssdtools::ssd_ci_methods(), and a couple of nrow values - times each ssdtools::ssd_hc() call, fits the per-ci_method cost model time = (base + slope * max(nboot, n0)) * nrow_factor(nrow), and returns a versioned ssdsims_cost_calibration object carrying the fitted coefficients and provenance. Producing an architecture-specific estimator is nothing more than this single call plus passing the result to ssd_estimate_cost().

Usage

ssd_calibrate_cost(
  nboot = c(20L, 50L, 100L, 200L),
  nrow = c(5L, 10L, 20L, 50L),
  data = NULL,
  seed = 42L
)

Arguments

Details

The sweep is self-contained and dependency-light: it draws data by resampling a reference dataset (ssddata::ccme_boron by default) and times with base system.time(). It takes minutes - not the hours a real scenario costs - because the nboot values are tiny (the slope and floor are estimable from small bootstraps). The one-time research that discovered the model's form (which axes are free, the max(nboot, n0) shape, the non-monotonic nrow factor) is preserved under the change's ⁠exploration/⁠ directory; it is not rerun here.

Value

An ssdsims_cost_calibration object with per-ci_method base/slope/n0 coefficients, a bounded nrow_factor, a fixed_addend, and provenance (cpu, R version, ssdtools version, date, sweep grid).

See Also

ssd_cost_calibration() for the shipped default and ssd_estimate_cost() to apply the result.

Examples

## Not run: 
calibration <- ssd_calibrate_cost()
calibration

## End(Not run)

Description

Returns the ssdsims_cost_calibration object shipped with the package - the calibration fitted during development (see ssd_cost_calibration_default) and used by ssd_estimate_cost() when no calibration is supplied. Because the coefficients are architecture-specific, an estimate built on this default is a ballpark sized for the machine in its provenance; rerun ssd_calibrate_cost() on your own machine for a trustworthy estimate.

Usage

ssd_cost_calibration()

Value

The shipped ssdsims_cost_calibration object.

See Also

ssd_calibrate_cost() to re-fit on a target machine and ssd_estimate_cost() to apply a calibration to a scenario.

Examples

ssd_cost_calibration()

Description

The ssdsims_cost_calibration object shipped with the package and returned by ssd_cost_calibration(). It carries the per-ci_method cost-model coefficients (base, slope, n0), the bounded nrow_factor, a fixed_addend (sample + fit per-task overhead), and the provenance of the machine it was fitted on. ssd_estimate_cost() uses it when no calibration is supplied.

Usage

ssd_cost_calibration_default

Format

An ssdsims_cost_calibration object: a list with coefficients (a tibble of ci_method, base, slope, n0), nrow_factor (a tibble of nrow, factor), fixed_addend (a scalar), and provenance (cpu, R version, ssdtools version, date, sweep grid).

Details

Because the coefficients are architecture-specific, this default yields a ballpark estimate sized for the machine in its provenance. Re-fit on your own machine with ssd_calibrate_cost() for a trustworthy estimate. It was produced by data-raw/cost_calibration.R (which simply runs ssd_calibrate_cost()).

Source

Fitted by ssd_calibrate_cost() during package development (Intel Xeon @ 2.10 GHz, R 4.5.3, ssdtools 2.6.0.9002).

See Also

ssd_cost_calibration(), ssd_calibrate_cost(), ssd_estimate_cost().

Description

Collects one or more datasets into a validated, named collection - the single entry point through which ssd_define_scenario() takes dataset input. Each dataset must carry a numeric Conc column (the species sensitivity distribution convention); additional columns are preserved.

Usage

ssd_data(...)

Arguments

Details

Names are taken from the argument names where supplied, otherwise derived from the argument expression by symbol capture (e.g. ssddata::ccme_boron becomes "ccme_boron"). A literal with no derivable name (e.g. a bare data.frame(...) call) must be given an explicit name.

ssd_data() is intended to grow: the planned scenario-input-types change (see TARGETS-DESIGN.md section 12) will let each input also be one of the data generators ssd_run_scenario() accepts today - a fitdists or tmbfit object, a generator function, or a function-name string - with the data materialised by the dataset registry. For now each input must be a data frame.

Value

An ssdsims_data object: a named list of validated tibbles.

Examples

ssd_data(ssddata::ccme_boron)
ssd_data(boron = ssddata::ccme_boron, cadmium = ssddata::ccme_cadmium)

Description

Constructs a purely declarative ssdsims_scenario object: the root of the targets-based pipeline (see TARGETS-DESIGN.md section 1). The object stores only declarative fields - a scalar seed, the replicate count nsim, the sample sizes nrow, the dataset names, and the fit and hc argument grids. It performs no random-number generation, no task expansion, and has no dependency on targets.

Usage

ssd_define_scenario(
  data,
  nsim,
  seed,
  ...,
  name = NULL,
  nrow = 6L,
  replace = FALSE,
  rescale = FALSE,
  computable = FALSE,
  at_boundary_ok = TRUE,
  min_pmix = list(ssdtools::ssd_min_pmix),
  range_shape1 = list(c(0.05, 20)),
  range_shape2 = list(c(0.05, 20)),
  dists = ssdtools::ssd_dists_bcanz(),
  est_method = "multi",
  proportion = 0.05,
  ci = FALSE,
  nboot = 1000,
  ci_method = "weighted_samples",
  parametric = TRUE,
  samples = FALSE,
  partition_by = NULL,
  bundle = NULL
)

Arguments

Details

Input data is forwarded through ssd_data() for validation (a numeric Conc column is required) and retained on the scenario (as ⁠$data⁠) so a local run (ssd_run_scenario_baseline()) can sample it directly. The dataset names (⁠$datasets⁠) are what the targets/cluster path hashes; the validated tibbles ride on the scenario and are isolated by name via scenario_dataset(), so the hash need not carry the data frames.

Value

An S3 object of class ssdsims_scenario.

Dataset input

The preferred form is an ssd_data() collection, which owns validation and naming: ssd_define_scenario(ssd_data(boron = ccme_boron, cadmium = ccme_cadmium), ...). For convenience, bare data frame input is also accepted in four forms (routed through the same Conc validation):

1. A single data frame, name derived from the argument expression: ssd_define_scenario(ssddata::ccme_boron, ...) gives "ccme_boron".

2. A single data frame with an explicit ⁠name=⁠: ssd_define_scenario(ssddata::ccme_boron, name = "boron", ...).

3. A named list, names taken from the list: ssd_define_scenario(list(boron = ccme_boron, cadmium = ccme_cadmium), ...).

4. An unnamed list, names derived per element: ssd_define_scenario(list(ccme_boron, ccme_cadmium), ...).

Supplying both a named list and ⁠name=⁠ is an error.

ci

ci is a scalar flag (not a cross-join axis): the point estimate est is invariant to ci - it is computed analytically from the fit, independent of the bootstrap and RNG - so a single ci = TRUE run is a strict superset of ci = FALSE (same est, plus the se/lcl/ucl columns). The choice is scenario-wide either/or: ci = FALSE for cheap, bootstrap-free point estimates, or ci = TRUE for estimates plus confidence intervals. When ci = FALSE, the bootstrap-only knobs nboot, ci_method, and parametric are meaningless; passing any of them in that case is an error, so set ci = TRUE to enable bootstrap, or omit the knobs.

dists and est_method

dists and est_method are simulation settings, not cross-join axes - they are absent from task_axes("fit")/task_axes("hc"), so they never multiply tasks or enter the per-task RNG primer. dists is the fit-level setting: the whole character vector is handed to one ssd_fit_dists() call per fit task (fanning out per distribution would dissolve the model averaging that defines a fit). est_method is an hc-level setting: every requested method is summarised from each hc task's single bootstrap sample set rather than re-bootstrapping per method (the CI is est_method-invariant and the point est is analytical), so a vector est_method yields one row per method within a task without fanning out into separate tasks.

Examples

ssd_define_scenario(ssddata::ccme_boron, nsim = 100L, seed = 42L, nrow = c(5L, 10L))

Description

Predicts, before a scenario is launched, roughly how much compute it costs and how long its single longest task runs. It expands the scenario into its hc task table read-only (via ssd_scenario_hc_tasks(), without running any fit or bootstrap), applies the calibrated per-task cost model, and returns the ballpark serial total cost and the duration of the longest single task, plus a per-axis breakdown of which ci_method/nboot cells dominate.

Usage

ssd_estimate_cost(scenario, calibration = ssd_cost_calibration())

Arguments

Details

proportion and est_method are free axes: one bootstrap per ⁠nboot x ci_method x parametric⁠ cell serves every proportion/est_method, so adding values along those axes does not change the estimate. The estimator does not execute the scenario, draw random numbers, or alter any result.

The longest task is the irreducible wall-time floor under any amount of parallelism; wall-time under n workers is roughly max(longest_task, total / n), computed by the caller.

Value

An ssdsims_cost_estimate object: a list with total and longest (both difftime time quantities), a breakdown tibble grouped by ci_method x nboot, and the calibration's provenance.

See Also

ssd_calibrate_cost() and ssd_cost_calibration().

Examples

scenario <- ssd_define_scenario(
  ssddata::ccme_boron,
  nsim = 10L,
  seed = 42L,
  ci = TRUE,
  nboot = c(1000L, 5000L, 10000L, 50000L),
  nrow = c(5L, 10L, 20L, 50L)
)
ssd_estimate_cost(scenario)

Description

Fit SSD Distributions to Simulated Data

Usage

ssd_fit_dists_sims(
  x,
  dists = ssdtools::ssd_dists_bcanz(),
  ...,
  rescale = FALSE,
  computable = FALSE,
  at_boundary_ok = TRUE,
  min_pmix = list(ssdtools::ssd_min_pmix),
  range_shape1 = list(c(0.05, 20)),
  range_shape2 = range_shape1,
  seed = NULL,
  silent = TRUE,
  .progress = FALSE
)

Arguments

Value

The x tibble with a list column fits of fistdist objects.

Description

Estimate hazard concentrations for multiple simulations using bootstrapping

Usage

ssd_hc_sims(
  x,
  proportion = 0.05,
  ...,
  ci = FALSE,
  nboot = 1000,
  est_method = "multi",
  ci_method = "weighted_samples",
  parametric = TRUE,
  seed = NULL,
  save_to = NULL,
  .progress = FALSE
)

Arguments

Value

The x tibble with a list column hc of data frames produced by applying ssd_hc() to fits.

Description

A generic, dispatched on the upload object's class, that opens the uploaded results so a user can read them back and confirm they landed right after an upload (TARGETS-DESIGN.md section 6.1). For an Azure destination it returns a lazy duckplyr/DuckDB table over the Hive glob ⁠<container>[/<prefix>]/<step>/**/part.parquet⁠ (honouring the destination's optional prefix subdirectory), read in place via DuckDB's azure extension (predicate pushdown straight against blob storage - no download), composable with dplyr verbs so a one-line ssd_open_uploaded(upload, step) |> dplyr::count() is the immediate post-upload smoke test. It resolves the same front-end ⁠SSDSIMS_AZURE_*⁠ credentials as the write path and remaps them into a DuckDB azure secret for the backend read, aborting (naming the missing requirement) when the azure extension or a required credential is absent. For a dry-run destination it aborts: a dry run uploads nothing, so the local shards should be read directly.

Usage

ssd_open_uploaded(upload, step)

Arguments

Value

A lazy, dplyr-composable table over the uploaded results.

See Also

ssd_upload_shard(), ssd_test_upload().

Examples

## Not run: 
upload <- ssd_upload_azure("https://acct.blob.core.windows.net", "results")
ssd_open_uploaded(upload, "hc") |> dplyr::count()

## End(Not run)

Description

Run Scenario

Usage

ssd_run_scenario(x, ...)

## S3 method for class 'data.frame'
ssd_run_scenario(
  x,
  ...,
  nrow = 6L,
  replace = FALSE,
  dists = ssdtools::ssd_dists_bcanz(),
  rescale = FALSE,
  computable = FALSE,
  at_boundary_ok = TRUE,
  min_pmix = list(ssdtools::ssd_min_pmix),
  range_shape1 = list(c(0.05, 20)),
  range_shape2 = list(c(0.05, 20)),
  proportion = 0.05,
  ci = FALSE,
  nboot = 1000,
  est_method = "multi",
  ci_method = "weighted_samples",
  parametric = TRUE,
  seed = NULL,
  nsim = 100L,
  stream = getOption("ssdsims.stream", 1L),
  start_sim = 1L,
  .progress = FALSE
)

## S3 method for class 'fitdists'
ssd_run_scenario(
  x,
  ...,
  nrow = 6L,
  dist_sim = "top",
  dists = ssdtools::ssd_dists_bcanz(),
  rescale = FALSE,
  computable = FALSE,
  at_boundary_ok = TRUE,
  min_pmix = list(ssdtools::ssd_min_pmix),
  range_shape1 = list(c(0.05, 20)),
  range_shape2 = list(c(0.05, 20)),
  proportion = 0.05,
  ci = FALSE,
  nboot = 1000,
  est_method = "multi",
  ci_method = "weighted_samples",
  parametric = TRUE,
  seed = NULL,
  nsim = 100L,
  stream = getOption("ssdsims.stream", 1L),
  start_sim = 1L,
  .progress = FALSE
)

## S3 method for class 'tmbfit'
ssd_run_scenario(
  x,
  ...,
  nrow = 6L,
  dists = ssdtools::ssd_dists_bcanz(),
  rescale = FALSE,
  computable = FALSE,
  at_boundary_ok = TRUE,
  min_pmix = list(ssdtools::ssd_min_pmix),
  range_shape1 = list(c(0.05, 20)),
  range_shape2 = list(c(0.05, 20)),
  proportion = 0.05,
  ci = FALSE,
  nboot = 1000,
  est_method = "multi",
  ci_method = "weighted_samples",
  parametric = TRUE,
  seed = NULL,
  nsim = 100L,
  stream = getOption("ssdsims.stream", 1L),
  start_sim = 1L,
  .progress = FALSE
)

## S3 method for class 'character'
ssd_run_scenario(
  x,
  ...,
  nrow = 6L,
  args = list(),
  dists = ssdtools::ssd_dists_bcanz(),
  rescale = FALSE,
  computable = FALSE,
  at_boundary_ok = TRUE,
  min_pmix = list(ssdtools::ssd_min_pmix),
  range_shape1 = list(c(0.05, 20)),
  range_shape2 = list(c(0.05, 20)),
  proportion = 0.05,
  ci = FALSE,
  nboot = 1000,
  est_method = "multi",
  ci_method = "weighted_samples",
  parametric = TRUE,
  seed = NULL,
  nsim = 100L,
  stream = getOption("ssdsims.stream", 1L),
  start_sim = 1L,
  .progress = FALSE
)

## S3 method for class ''function''
ssd_run_scenario(
  x,
  ...,
  nrow = 6L,
  args = list(),
  dists = ssdtools::ssd_dists_bcanz(),
  rescale = FALSE,
  computable = FALSE,
  at_boundary_ok = TRUE,
  min_pmix = list(ssdtools::ssd_min_pmix),
  range_shape1 = list(c(0.05, 20)),
  range_shape2 = list(c(0.05, 20)),
  proportion = 0.05,
  ci = FALSE,
  nboot = 1000,
  est_method = "multi",
  ci_method = "weighted_samples",
  parametric = TRUE,
  seed = NULL,
  nsim = 100L,
  stream = getOption("ssdsims.stream", 1L),
  start_sim = 1L,
  .progress = FALSE
)

Arguments

Value

A tibble of nested data sets.

Methods (by class)

• ssd_run_scenario(data.frame): Run scenario using data.frame to sample data

• ssd_run_scenario(fitdists): Run scenario using fitdists object to generate data

• ssd_run_scenario(tmbfit): Run scenario using tmbfit object to generate data

• ssd_run_scenario(character): Run scenario using name of function to generate sequence of random numbers

• ssd_run_scenario(`function`): Run scenario data using function to generate sequence of random numbers

Examples

ssd_run_scenario(ssddata::ccme_boron, nsim = 2)

fit <- ssdtools::ssd_fit_dists(ssddata::ccme_boron)
ssd_run_scenario(fit, dist_sim = c("lnorm", "top"), nsim = 3)

fit <- ssdtools::ssd_fit_dists(ssddata::ccme_boron)
ssd_run_scenario(fit[[1]], nsim = 3)

ssd_run_scenario("rlnorm", nsim = 3)

ssd_run_scenario(ssdtools::ssd_rlnorm, nsim = 3)

Description

Executes the three task tables in dependency order - sample, fit, then hc - by looping over each table with purrr::pmap() and looking up each task's parent result by the parent's ⁠<step>_id⁠ foreign key. The fit step truncates its parent sample inline (head(sample, nrow)) before fitting. The runner does no task expansion of its own (it consumes ssd_scenario_tasks()); it just threads outputs forward and returns the collected per-step results.

Usage

ssd_run_scenario_baseline(scenario)

Arguments

Details

This is the no-frills baseline: it runs in-process, with no targets dependency, no shard grouping or partition_by, and no Parquet I/O.

It is reproducible without an external seed. The runner opens one local_dqrng_backend() scope and seeds each sample/fit/hc task exactly once through its ⁠*_data_task_primer()⁠ wrapper, with seed = scenario$seed and a per-task primer derived from the task's canonical identity (task_primer() over the task_axes(step) columns). Because each task's ⁠(seed, primer)⁠ pair fully determines its RNG starting point, two runs of a scenario with a fixed seed yield identical results, and a task's result is independent of the order in which tasks run. These same ⁠*_data_task_primer()⁠ wrappers are the per-task entry point a future targets shard body and the replay helper (TARGETS-DESIGN.md §7) reuse.

The scenario retains the data frames it was built from, so the runner reads them directly - no separate data argument. min_pmix names are resolved to their materialised functions off the scenario via scenario_min_pmix() (resolved once, at construction), not by a runtime ssdtools/global-env search.

Value

A named list with sample, fit, and hc elements: each the corresponding task table augmented with a list column of per-task results (sample draws, fits objects, and hc tibbles).

Examples

scenario <- ssd_define_scenario(
  ssddata::ccme_boron,
  nsim = 1L,
  nrow = 6L,
  seed = 42L,
  dists = "lnorm"
)
out <- ssd_run_scenario_baseline(scenario)
out$hc

Description

Executes a scenario's three task steps in dependency order - sample, then fit, then hc - materialising each step's results as one Parquet per partition_by path cell under a Hive-partitioned tree ⁠<dir>/<step>/<axis=value>/.../part.parquet⁠, and linking steps by reading the parent step's shards back via duckplyr (predicate pushdown), rather than threading results in memory. This is the single-core, targets-free sibling of ssd_run_scenario_baseline() and the first consumer of partition-by's path/inner split (scenario_dataset()'s sibling scenario_partition_axes()).

Usage

ssd_run_scenario_shards(scenario, dir = tempfile("ssdsims-shards-"))

Arguments

Details

It reuses the per-task seed-and-run wrappers, so for a fixed scenario$seed it is reproducible and order-independent, and its per-task results are byte-identical to ssd_run_scenario_baseline() - partition_by is a free re-layout that moves only file paths, never results. The m:n parent-shard dependency (a child shard reading several parent shards, or a parent shard feeding several children, per the section 5 coarsening defaults) is resolved at read time: each task opens the parent shard at its ⁠<parent>_id⁠ identity projected onto the parent's path axes and filters to the rows it needs.

No targets, crew, manifest, or cloud upload - this is the plain-R storage loop only, de-risking hive-partitioning/task-tables.

Value

An ssdsims_shard_run object: a list with dir and the written sample, fit, and hc shard Parquet paths (one per shard).

See Also

ssd_run_scenario_baseline() (the in-memory reference oracle), ssd_scenario_sample_shards(), ssd_run_sample_step().

Examples

scenario <- ssd_define_scenario(
  ssddata::ccme_boron,
  nsim = 1L,
  nrow = 6L,
  seed = 42L,
  dists = "lnorm"
)
run <- ssd_run_scenario_shards(scenario)
run$hc

Description

The per-shard step runners the targets pipeline (and the single-core ssd_run_scenario_shards()) call - one target per shard, one runner per step. Each takes a shard's tasks (the tasks list-column of a row of the matching ssd_scenario_sample_shards() family), runs the bundled tasks with the same per-task seed-and-run primitives the baseline runner uses (⁠*_data_task_primer()⁠) under one local_dqrng_backend() scope, reads any upstream shard back from Parquet by partition path, and writes one Parquet at the shard's Hive partition path - returning that path (the format = "file" contract). Because a task's result is fully determined by its ⁠(seed, primer)⁠ and is order-independent, the per-task results are byte-identical to ssd_run_scenario_baseline() regardless of how tasks bundle into shards.

Usage

ssd_run_sample_step(tasks, scenario, out_dir)

ssd_run_fit_step(tasks, scenario, sample_dir, out_dir)

ssd_run_hc_step(tasks, scenario, fit_dir, out_dir)

Arguments

Value

The shard's Parquet path (the format = "file" contract).

Functions

• ssd_run_sample_step(): Run the sample tasks: read each task's dataset off the scenario via scenario_dataset(), draw n_max rows through sample_data_task_primer(), and tag each draw with its sample_id and a .row order index so a downstream fit shard can isolate and re-order it.

• ssd_run_fit_step(): Run the fit tasks: read the distinct set of parent sample shards the shard's tasks reference (each once - they may span several sample shards), isolate each task's draw by sample_id (restoring row order), truncate it inline (head(sample, nrow), RNG-free, section 5), and fit with the per-task ⁠(seed, primer)⁠ through fit_data_task_primer() (resolving min_pmix off the scenario via scenario_min_pmix()). The fitted fitdists object is serialised into a fit_blob string column keyed by fit_id, and one Parquet is written at the shard's partition path.

• ssd_run_hc_step(): Run the hc tasks: read the distinct set of parent fit shards the shard's tasks reference (each once - an hc shard typically spans several fit shards), isolate each task's fit by fit_id, deserialise the fitdists object, and estimate the hazard concentration with the per-task ⁠(seed, primer)⁠ through hc_data_task_primer(). Each task's hc tibble (one or more rows - the proportion fan-out, with the scalar ci applied uniformly and bootstrap-only knobs NA when ci = FALSE) is tagged with its hc_id and parent fit_id, stacked, and written as one Parquet at the shard's partition path.

See Also

ssd_scenario_sample_shards() (the shard grouping these consume), ssd_run_scenario_shards(), ssd_run_scenario_baseline().

Examples

scenario <- ssd_define_scenario(ssddata::ccme_boron, nsim = 1L, seed = 42L)
shards <- ssd_scenario_sample_shards(scenario)
dir <- tempfile()
ssd_run_sample_step(shards$tasks[[1L]], scenario, file.path(dir, "sample"))

scenario <- ssd_define_scenario(
  ssddata::ccme_boron,
  nsim = 1L,
  nrow = 6L,
  seed = 42L,
  dists = "lnorm"
)
dir <- tempfile()
ssd_run_sample_step(
  ssd_scenario_sample_shards(scenario)$tasks[[1L]],
  scenario,
  file.path(dir, "sample")
)
ssd_run_fit_step(
  ssd_scenario_fit_shards(scenario)$tasks[[1L]],
  scenario,
  file.path(dir, "sample"),
  file.path(dir, "fit")
)


scenario <- ssd_define_scenario(
  ssddata::ccme_boron,
  nsim = 1L,
  nrow = 6L,
  seed = 42L,
  dists = "lnorm"
)
dir <- tempfile()
ssd_run_sample_step(
  ssd_scenario_sample_shards(scenario)$tasks[[1L]],
  scenario,
  file.path(dir, "sample")
)
ssd_run_fit_step(
  ssd_scenario_fit_shards(scenario)$tasks[[1L]],
  scenario,
  file.path(dir, "sample"),
  file.path(dir, "fit")
)
ssd_run_hc_step(
  ssd_scenario_hc_shards(scenario)$tasks[[1L]],
  scenario,
  file.path(dir, "fit"),
  file.path(dir, "hc")
)

Description

Group a step's per-task table into a per-shard table: one row per partition_by path cell, carrying the path-axis columns (the tar_map target-name suffix and Hive path) and a tasks list-column of that cell's task rows. Each task row is decorated with seed = scenario$seed and its per-task primer (task_primer() over the step's task_axes()); the decoration is RNG-free (a pure hash, not a draw), so the bare task tables (ssd_scenario_tasks()) keep their no-⁠(seed, primer)⁠ contract. The result is the values a tarchetypes::tar_map() consumes to mint one target per shard.

Usage

ssd_scenario_sample_shards(scenario)

ssd_scenario_fit_shards(scenario)

ssd_scenario_hc_shards(scenario)

Arguments

Details

For fit/hc each task row in tasks also carries its parent step's path-axis values and ⁠<parent>_id⁠, so the runner opens the matching parent shard by partition path.

Value

A tibble with one row per shard of the step: the path-axis columns and a tasks list-column. Suitable as tarchetypes::tar_map(values = ).

Functions

• ssd_scenario_sample_shards(): Group the sample tasks (ssd_scenario_sample_tasks()) by partition_by$sample.

• ssd_scenario_fit_shards(): Group the fit tasks (ssd_scenario_fit_tasks()) by partition_by$fit. Each task row in tasks carries its parent sample path-axis values and sample_id, so the runner opens the matching sample shard by partition path.

• ssd_scenario_hc_shards(): Group the hc tasks (ssd_scenario_hc_tasks()) by partition_by$hc. Each task row in tasks carries its parent fit path-axis values and fit_id, so the runner opens the matching fit shard by partition path.

See Also

ssd_run_sample_step() (the matching per-shard step runners).

Examples

scenario <- ssd_define_scenario(ssddata::ccme_boron, nsim = 2L, seed = 42L)
ssd_scenario_sample_shards(scenario)
scenario <- ssd_define_scenario(
  ssddata::ccme_boron,
  nsim = 2L,
  seed = 42L,
  rescale = c(FALSE, TRUE)
)
ssd_scenario_fit_shards(scenario)
scenario <- ssd_define_scenario(
  ssddata::ccme_boron,
  nsim = 2L,
  seed = 42L,
  ci = TRUE
)
ssd_scenario_hc_shards(scenario)

Description

A target factory: returns the list of targets objects that runs a scenario as a static-branching Hive-sharded pipeline (TARGETS-DESIGN.md section 6), so a whole ⁠_targets.R⁠ reduces to build a scenario and call this:

Usage

ssd_scenario_targets(
  scenario,
  ...,
  root = scenario_results_dir(scenario),
  upload = NULL,
  cue = NULL
)

Arguments

Details

library(targets)
library(tarchetypes)
library(ssdsims)
scenario <- ssd_define_scenario(ssddata::ccme_boron, nsim = 2L, seed = 42L)
ssd_scenario_targets(scenario)

The shard and summary targets carry error = "null" so a shard whose body fails entirely goes NULL (its error readable via tar_meta()) without aborting the run, and ssd_summarise() unions whatever landed (TARGETS-DESIGN.md section 6.2). The shipped ⁠_targets.R⁠ templates pair this with a pipeline-wide keep-going default (tar_option_set(error = "continue"), the make -k analogue) so an errored target skips only its dependents while the rest of the shards still build; fail-fast pre-flight checks (upload/cluster connectivity) belong in a separate script the user runs before tar_make(), not in this DAG.

For each step it tarchetypes::tar_map()s one named, format = "file", error = "null" target per partition_by path cell (the names are the step's path axes), and writes every shard and the summary under the per-layout scenario_results_dir() root (so a changed partition_by/bundle never mixes shard granularities). Each step's command depends only on the minimal scenario slice its runner consumes (scenario_step_slice()) rather than the bare scenario global, so editing a field a step does not read leaves the other steps' shards cached. The sample slice is built per shard, carrying only the dataset(s) that shard reads, so appending a dataset mints a new shard and leaves every existing shard cached.

Value

A list of targets target objects, for ⁠_targets.R⁠ to return.

Invalidation model

The shard targets use content-hash invalidation over their format = "file" Parquet outputs (TARGETS-DESIGN.md section 8), observable as cache-by-existence: a shard is up to date iff its Parquet exists and the inputs its body depends on - its task rows, the step's minimal scenario slice (scenario_step_slice()), and the parent shard target(s) it reads - are unchanged. A missing Parquet rebuilds; a recomputed shard whose bytes are byte-identical leaves its dependents skipped.

Instead of a coarse sample -> fit -> hc tar_combine() barrier (which marks the whole downstream step out of date when any one parent shard changes), each child shard target names only the specific parent shard target(s) its tasks read (the Option-3 per-child upstream edges of section 6), computed at sourcing time as unique(path_key(tasks, partition_by[[parent]])) - the same projection the runner uses to read them. So rewriting one parent shard re-runs only the child shards that read it. summary reads the whole hc directory, so it names every hc shard (it re-runs when any hc shard's bytes change, and unions the survivors of a partially-failed run).

Pinning trusted shards (cue)

Pass cue = targets::tar_cue(depend = FALSE) to pin the shard targets against upstream dependency/code changes (an edited per-task primitive, a bumped ssdtools), so trusted shards are not rebuilt by a code edit (TARGETS-DESIGN.md section 8.3). The carve-outs still hold: a shard rebuilds if its format = "file" Parquet is missing, if its task-table grouping changes (the grouping is part of the command, so path-axis and inner-axis growth still apply under the pin), or if it previously errored. Force a refresh of chosen shards with targets::tar_invalidate() (or by deleting their Parquet), overriding the pin (section 8.4). The default (NULL) is targets' standard cue.

The head(sample, nrow) truncation stays folded into the fit step (no materialised data shard): a fit shard is keyed by fit_id, which includes nrow, so extending nrow mints new fit shards and caches the rest, and a widened max(nrow) changes the sample shard's n_max task row, so its bytes change and the per-child edge propagates to exactly the fit shards that read the wider draw - no stale short draw is produced.

To parallelise the shards, set a controller (e.g. a mirai-backed crew::crew_controller_local()) with targets::tar_option_set() in ⁠_targets.R⁠ before calling this - the target set is unchanged.

Uploading shards to cloud storage (upload)

upload is the remote-destination sibling of root (default NULL). With upload = NULL the pipeline contains no ⁠upload_<step>⁠ targets - the clean default DAG for a non-uploader. With a non-NULL upload object the factory pairs each step shard with an ⁠upload_<step>⁠ target in the same tar_map (format = "file", error = "null"), so an unchanged shard is never re-uploaded (content-hash skip) and a per-shard upload failure isolates to its own branch. Pass ssd_upload_dryrun() for no-op upload targets that reach no network (exercising the DAG shape offline / in CI) or ssd_upload_azure() to ship to Azure. The factory performs no network I/O and never runs the ssd_test_upload() probe: it only assembles the target list, so sourcing ⁠_targets.R⁠ (which targets does on every tar_make(), tar_manifest(), tar_visnetwork(), and on each worker) stays side-effect free. Run ssd_test_upload(upload) yourself as a one-line preflight before tar_make() to confirm credentials and connectivity up front; a missing credential still fails loud per-shard at upload time as a backstop. The per-task results are byte-identical across all three upload modes; only the presence and behaviour of the ⁠upload_<step>⁠ targets differ.

See Also

scenario_results_dir(), ssd_run_scenario_shards() (the single-core, targets-free equivalent), ssd_upload_azure().

Examples

## Not run: 
# _targets.R
library(targets)
library(tarchetypes)
library(ssdsims)
scenario <- ssd_define_scenario(ssddata::ccme_boron, nsim = 2L, seed = 42L)
ssd_scenario_targets(scenario)

# Pair each shard with a (no-op) upload target, exercised offline:
ssd_scenario_targets(scenario, upload = ssd_upload_dryrun())

## End(Not run)

Description

The canonical expansion entry point (TARGETS-DESIGN.md section 1/section 2): ssd_scenario_tasks() derives the sample, fit, and hc task tables from a scenario in one call and bundles them into an ssdsims_task_set. The per-step derivations (ssd_scenario_sample_tasks(), ssd_scenario_fit_tasks(), ssd_scenario_hc_tasks()) remain available for callers that need a single table; each is equivalent to ssd_scenario_tasks(scenario, step) for the matching step.

All derivations are RNG-free: they perform no random-number generation and add no seed/primer/stream columns (those arrive in later roadmap steps; see TARGETS-DESIGN.md section 2). Each row carries a path-style ⁠<step>_id⁠ primary key (the Hive partition path) and, for non-root steps, its parent step's ⁠<parent>_id⁠ as a joinable foreign key, so a child task references its parent by a single column.

Usage

ssd_scenario_tasks(scenario, step = NULL)

ssd_scenario_sample_tasks(scenario)

ssd_scenario_fit_tasks(scenario)

ssd_scenario_hc_tasks(scenario)

Arguments

Value

An ssdsims_task_set object (a list with sample, fit, and hc elements, each an ssdsims_tasks table), or - when step is supplied - the single ssdsims_tasks table for that step. Each ssdsims_tasks table is a classed tibble recording one step, with one row per cell of that step's cross-join.

Functions

• ssd_scenario_sample_tasks(): Derive just the sample task table: one row per cell of the cross-join of the scenario's dataset names, replicate index (1:nsim), and replace values, keyed by sample_id. Each row is the single random draw of n_max = max(nrow) rows that every nrow value sub-truncates (TARGETS-DESIGN.md section 5), so nrow is not a sample axis - the draw is shared - and n_max is carried as an ordinary integer column.

• ssd_scenario_fit_tasks(): Derive just the fit task table: cross each sample-task identity (dataset, sim, replace) with the scenario's nrow values and each row of the scenario's fit argument grid (rescale, computable, at_boundary_ok, min_pmix name, range_shape1, range_shape2). nrow is a genuine fit cross-join axis: the fit step truncates its parent sample inline (head(sample, nrow), RNG-free) before fitting, so the shared draw is sub-truncated without a separate data step (TARGETS-DESIGN.md section 5). min_pmix is referenced by name, not by function value (TARGETS-DESIGN.md section 1.1). Each row carries a fit_id primary key and a sample_id foreign key referencing its parent sample task.

• ssd_scenario_hc_tasks(): Derive just the hc task table: cross each fit-task identity with each row of the scenario's hc argument grid (nboot, ci_method, parametric). The scenario's scalar ci flag and the est_method setting are applied uniformly to every hc row - neither is a cross-join axis; ci rides as a carried column and every requested est_method is summarised within each task from its single bootstrap sample set. When ci = FALSE the bootstrap-only knobs (nboot, ci_method, parametric) are canonically NA and there is no fan-out axis, so the grid is exactly one hc row per fit task; when ci = TRUE the grid fans out across ⁠nboot x ci_method x parametric⁠. Each row carries an hc_id primary key and a fit_id foreign key referencing its parent fit task.

Examples

scenario <- ssd_define_scenario(ssddata::ccme_boron, nsim = 3L, seed = 42L)
tasks <- ssd_scenario_tasks(scenario)
tasks
tasks$hc
ssd_scenario_tasks(scenario, "hc")
ssd_scenario_sample_tasks(scenario)
scenario <- ssd_define_scenario(
  ssddata::ccme_boron,
  nsim = 3L,
  seed = 42L,
  rescale = c(FALSE, TRUE)
)
ssd_scenario_fit_tasks(scenario)
scenario <- ssd_define_scenario(
  ssddata::ccme_boron,
  nsim = 2L,
  seed = 42L,
  ci = TRUE,
  nboot = c(10L, 100L)
)
ssd_scenario_hc_tasks(scenario)

Description

A family of functions to generate a tibble of nested data sets.

Usage

ssd_sim_data(x, ...)

## S3 method for class 'data.frame'
ssd_sim_data(
  x,
  ...,
  nrow = 6L,
  replace = FALSE,
  seed = NULL,
  nsim = 100L,
  stream = getOption("ssdsims.stream", 1L),
  start_sim = 1L,
  .progress = FALSE
)

## S3 method for class 'fitdists'
ssd_sim_data(
  x,
  ...,
  nrow = 6L,
  dist_sim = "top",
  seed = NULL,
  nsim = 100L,
  stream = getOption("ssdsims.stream", 1L),
  start_sim = 1L,
  .progress = FALSE
)

## S3 method for class 'tmbfit'
ssd_sim_data(
  x,
  ...,
  nrow = 6L,
  seed = NULL,
  nsim = 100L,
  stream = getOption("ssdsims.stream", 1L),
  start_sim = 1L,
  .progress = FALSE
)

## S3 method for class 'character'
ssd_sim_data(
  x,
  ...,
  nrow = 6L,
  args = list(),
  seed = NULL,
  nsim = 100L,
  stream = getOption("ssdsims.stream", 1L),
  start_sim = 1L,
  .progress = FALSE
)

## S3 method for class ''function''
ssd_sim_data(
  x,
  ...,
  nrow = 6L,
  args = list(),
  seed = NULL,
  nsim = 100L,
  stream = getOption("ssdsims.stream", 1L),
  start_sim = 1L,
  .progress = FALSE
)

Arguments

Value

A tibble of nested data sets.

Methods (by class)

• ssd_sim_data(data.frame): Generate data by sampling from data.frame

• ssd_sim_data(fitdists): Generate data from fitdists object

• ssd_sim_data(tmbfit): Generate data from tmbfit object

• ssd_sim_data(character): Generate data using name of function

• ssd_sim_data(`function`): Generate data using function to generate sequence of random numbers

Examples

ssd_sim_data(ssddata::ccme_boron, nrow = 5, nsim = 3)

fit <- ssdtools::ssd_fit_dists(ssddata::ccme_boron)
ssd_sim_data(fit, nrow = 5, nsim = 3)

fit <- ssdtools::ssd_fit_dists(ssddata::ccme_boron)
ssd_sim_data(fit[[1]], nrow = 5, nsim = 3)

ssd_sim_data("rnorm", nrow = 5, nsim = 3)

ssd_sim_data(ssdtools::ssd_rlnorm, nrow = 5, nsim = 3)

Description

Fans in the run's results without pulling shard target values back into R or recomputing anything: reads every hc shard Parquet under dir_hc (a Hive glob) with duckplyr - the analysis-ready per-task hazard-concentration estimates - unions them, and writes path. Because it reads the result directory (not the shard targets), it sees whatever shards landed, so it unions the survivors of a partially-failed run (error = "null", section 6.2). dir_sample and dir_fit are accepted for signature symmetry with the three result layers; the sample draws and serialised fit objects are not summary material, so the combined summary is the hc layer.

Usage

ssd_summarise(
  dir_sample,
  dir_fit,
  dir_hc,
  path,
  path_with_samples = NULL,
  samples_row_group_bytes = "100MB"
)

Arguments

Details

The compact summary at path projects the dists/samples list-columns out at the DuckDB level, so the potentially-large retained bootstrap draws are never pulled into R. Supply path_with_samples to also write a full summary that retains those list-columns: that write reuses the same lazy DuckDB read, so the draws never materialise in R there either. The draws are populated only when the scenario set samples = TRUE, so the full summary is the analysis-ready estimates plus the per-row draws.

In a targets pipeline a directory read carries no dependency edge, so ssd_scenario_targets() orders summary after the shards by naming every hc shard target in its command (it re-runs when any hc shard's bytes change). Reading the directory - rather than the shard target values - is what lets it union whatever shards landed (the survivors of a partially-failed run, section 6.2).

Value

The summary Parquet path(s) (the format = "file" contract): path when path_with_samples is NULL, otherwise c(path, path_with_samples).

Memory and the full summary's row groups

The full summary is written in byte-budgeted Parquet row groups (samples_row_group_bytes, default "100MB"), so its memory requirement follows the per-group budget - about five times the budget - rather than the union's total row count, and the row-group row count adapts to the samples cell size (large groups for small draws, small groups for large ones). The engine accepts the byte budget because the pipeline configuration scope holds preserve_insertion_order = false (restored when ssd_summarise() returns); it is refused while preserving order, and only the global setting counts - the per-copy PRESERVE_ORDER option cannot substitute. The trade: the full summary's row order is not contractual - re-summarising the same shards yields the same rows (address them by hc_id/fit_id), but their order and the file's bytes may differ. Under the default single thread, writes were observed in input order and byte-identical across runs regardless. Evidence: the duckplyr-config change's exploration/experiment-summary-union.R, exploration/experiment-rgbytes.R, and exploration/experiment-preserve-order-copy-option.R.

Examples

scenario <- ssd_define_scenario(
  ssddata::ccme_boron,
  nsim = 1L,
  nrow = 6L,
  seed = 42L,
  dists = "lnorm"
)
# Materialise the shards single-core, then fan in the hc layer.
run <- ssd_run_scenario_shards(scenario)
ssd_summarise(
  file.path(run$dir, "sample"),
  file.path(run$dir, "fit"),
  file.path(run$dir, "hc"),
  file.path(run$dir, "summary.parquet")
)

Description

The cloud counterpart of ssd_summarise(): a generic, dispatched on the upload object's class, that fans a step's uploaded shards into a single lazy duckplyr table read in place (no download). For an Azure destination it reads the ⁠<container>[/<prefix>]/<step>/**/part.parquet⁠ Hive glob via DuckDB's azure extension - resolving the same front-end secret as the write path and remapping it (with the account derived from url) into a DuckDB azure secret - and returns the union as a lazy duckplyr tibble (not collected, so the read and projection stay in DuckDB). By default it projects away the heavy dists/samples list-columns (the analysis-ready summary, mirroring ssd_summarise()); pass drop_samples = FALSE to keep them when the in-flight bootstrap samples are needed. The default method (an unknown destination) and the dry-run method both abort.

Usage

ssd_summarise_uploaded(upload, step = "hc", drop_samples = TRUE)

Arguments

Value

A lazy duckplyr/DuckDB tibble over the unioned, uploaded step layer (not collected), composable with dplyr verbs - dplyr::collect() it (or write it with duckplyr::compute_parquet()) when you need the rows in R.

See Also

ssd_open_uploaded(), ssd_summarise(), ssd_upload_shard().

Examples

## Not run: 
upload <- ssd_upload_azure("https://acct.blob.core.windows.net", "results")
ssd_summarise_uploaded(upload, "hc")
ssd_summarise_uploaded(upload, "hc", drop_samples = FALSE) # keep samples

## End(Not run)

Description

The front-door check, dispatched on the upload object's class, that confirms before any compute whether the destination is reachable and the credentials are in the right place (TARGETS-DESIGN.md section 6.1). Run it as a one-liner at the prompt before tar_make() - it is the user's explicit preflight. ssd_scenario_targets() deliberately does not call it (the factory does no network I/O, so sourcing ⁠_targets.R⁠ stays side-effect free); a missing credential still fails loud per-shard at ssd_upload_shard() time as a backstop.

Usage

ssd_test_upload(upload)

Arguments

Details

For an Azure destination it resolves the credentials from the environment and, when a required variable is absent, aborts with a loud error naming the missing variable (rather than failing later on a worker); when they are present it lists the container and writes then deletes a small marker blob, returning invisibly on success and aborting with the backend's diagnostic on failure. For a dry-run destination it succeeds trivially without resolving credentials or reaching any network.

Value

NULL, invisibly (called for its side effect: the probe).

See Also

ssd_upload_shard(), ssd_open_uploaded(), ssd_upload_azure().

Examples

ssd_test_upload(ssd_upload_dryrun())

Description

Typed, self-validating destination objects for the targets pipeline's per-shard upload (TARGETS-DESIGN.md section 6.1). Pass one to ssd_scenario_targets()'s upload argument (the remote-destination sibling of root) to pair each step shard with an ⁠upload_<step>⁠ target.

Usage

ssd_upload_azure(
  url,
  container,
  ...,
  prefix = NULL,
  domain = "blob.core.windows.net"
)

ssd_upload_dryrun()

Arguments

Details

ssd_upload_azure() describes an Azure Blob Storage container; ssd_upload_dryrun() is a no-op destination that reaches no network, so the upload DAG shape can be exercised offline and in CI without credentials. Both return a plain, serialisable S3 object of class c("ssdsims_upload_<backend>", "ssdsims_upload") that carries only the destination - never credentials, open connections, or environments - so it travels unchanged to crew workers and through targets.

Credentials stay external to the object: the Azure methods (ssd_test_upload(), ssd_upload_shard(), ssd_open_uploaded(), ssd_summarise_uploaded()) resolve the secret from the environment at call time - one of SSDSIMS_AZURE_STORAGE_KEY, SSDSIMS_AZURE_STORAGE_SAS, or the service-principal trio SSDSIMS_AZURE_TENANT_ID/SSDSIMS_AZURE_CLIENT_ID/SSDSIMS_AZURE_CLIENT_SECRET

• and abort with a loud error naming the missing variable when none is present. The storage account name is derived from url (see domain), so it is not an environment variable.

Value

An S3 object of class c("ssdsims_upload_azure_blob", "ssdsims_upload") (for ssd_upload_azure()) or c("ssdsims_upload_dryrun", "ssdsims_upload") (for ssd_upload_dryrun()).

See Also

ssd_test_upload(), ssd_upload_shard(), ssd_open_uploaded(), ssd_scenario_targets().

Examples

ssd_upload_azure("https://acct.blob.core.windows.net", "ssdsims-results")
ssd_upload_azure(
  "https://acct.blob.core.windows.net",
  "ssdsims-results",
  prefix = "study-2026/run-3"
)
ssd_upload_dryrun()

Description

A generic, dispatched on the upload object's class, that ships one shard Parquet to the destination and returns the local path (so the paired ⁠upload_<step>⁠ target stays format = "file"). For an Azure destination it uploads the file at path to ⁠<url>/<container>[/<prefix>]/<step>/<partition-path>/part.parquet⁠ (the optional prefix subdirectory from ssd_upload_azure()); when the required credentials are absent it aborts with a loud error - never a silent no-op - so intent to skip the network is only ever expressed by passing ssd_upload_dryrun(). For a dry-run destination it performs no network I/O, records a skip, and returns the local path.

Usage

ssd_upload_shard(path, upload)

Arguments

Value

The local path (a string), so the ⁠upload_<step>⁠ target stays format = "file".

Adding a backend

The destination set is open and extended by a constructor-plus-methods contract - no edit to the existing methods. To add S3, GCS, or another backend:

1. Write a constructor returning an object of class c("ssdsims_upload_<backend>", "ssdsims_upload") that validates its destination at construction (as ssd_upload_azure() validates its url and container) and carries no credentials.

2. Implement the generic methods for that class: ssd_upload_shard() (ship one shard, return the local path), ssd_test_upload() (the credentials/connectivity probe, failing loud on a missing credential), ssd_open_uploaded() (read the uploaded results back in place), and ssd_summarise_uploaded() (the in-place fan-in summary).

The package ships only the Azure and dry-run backends; no speculative backends are added.

See Also

ssd_test_upload(), ssd_open_uploaded(), ssd_scenario_targets().

Examples

path <- tempfile(fileext = ".parquet")
file.create(path)
ssd_upload_shard(path, ssd_upload_dryrun())

Description

Derives the per-task primer – a length-2 integer vector – from rlang::hash(params), suitable for the stream argument of dqrng::dqset.seed(). Together with the scenario seed, the primer fully specifies a task's RNG starting point: dqrng::dqset.seed(seed, stream = task_primer(params)). It pairs with local_dqrng_state(), which installs the ⁠(seed, primer)⁠ pair under an active local_dqrng_backend() scope.

Usage

task_primer(params)

Arguments

Details

The primer packs 64 bits of the rlang::hash() digest (xxhash128) as c(hi32, lo32). Each 32-bit half is encoded as a signed int32, with the reserved bit pattern 0x80000000 (INT_MIN, which R cannot represent as a non-NA integer) mapped to NA_integer_; dqrng accepts NA_integer_ in stream and treats it as INT_MIN, so the encoding recovers the full 64 bits of stream entropy.

params may be a plain named list or a single-row data frame (one row of a ⁠{sample,fit,hc}_tasks⁠ table). A data-frame row is normalised to a canonical plain list – the inverse of tibble::tibble_row() – by dropping all attributes, unwrapping length-1 list-style columns to their element, and leaving df-style (nested data-frame) columns as data frames, before hashing. The primer is therefore identical whether derived from the row or from the equivalent plain list. Note that rlang::hash() is order-sensitive, so the plain list must use the same name order as the task-table columns to reproduce the row's primer (assembling params in a canonical column order is part of the task-tables caller contract below).

task_primer() normalises structure, not meaning: it hashes whatever params it is given. The canonical, name-keyed representation is a caller contract assembled where params is built (task-tables, over the task-lists tables). Per the three-step model the RNG-consuming steps each take a primer over their task identity:

• sample – keyed ⁠(dataset, sim, replace)⁠ only. nrow is deliberately absent: every nrow shares one n_max-row draw that the fit step truncates inline (head(sample, nrow), RNG-free, no separate primer), so excluding nrow is load-bearing for the sub-truncation property (TARGETS-DESIGN.md §5).

• fit – the parent sample identity plus nrow and the fit-grid row (rescale, computable, at_boundary_ok, min_pmix name, range_shape1, range_shape2). nrow IS part of the fit primer: a fit on a different truncation is a genuinely different computation.

• hc – the parent fit identity plus the hc-grid row (nboot, est_method, ci_method, parametric). ci is a scalar hc flag applied uniformly, not part of the task identity, so it is a carried column rather than a primer field.

Function-valued parameters (e.g. min_pmix) MUST be referenced by name, not by function value, so a recompile or JIT does not move a task's primer.

Value

An integer vector of length 2 – the per-task primer – to pass as the stream argument of dqrng::dqset.seed() (via local_dqrng_state()).

See Also

local_dqrng_state(), local_dqrng_backend().

Examples

task_primer(list(dataset = "boron", sim = 1L, replace = FALSE))