Manual Execution

Run any DAG immediately from the command line, regardless of trigger configuration.

Basic usage

daggle run <dag-name>

Parameter overrides

daggle run etl-pipeline -p department=marketing -p date=2026-01-15

The -p / --param flag is repeatable. Parameters override the defaults defined in the DAG YAML.

What happens

  1. daggle parses the DAG YAML and validates it
  2. Template variables are expanded ({ .Today }, { .Params.x }, etc.)
  3. base.yaml defaults are merged (if present)
  4. Secrets are resolved (${env:}, ${file:}, ${vault:}) – fails fast if any are missing
  5. R version is checked (if r_version: is set)
  6. renv is auto-detected and R_LIBS_USER is set
  7. Steps are topologically sorted into parallel tiers
  8. Tiers execute sequentially; steps within a tier run in parallel
  9. Lifecycle hooks fire on completion/failure
  10. meta.json is written with reproducibility metadata

Signal handling

  • Ctrl+C (SIGINT) or SIGTERM sends SIGTERM to the entire process group of all running steps
  • After a 5-second grace period, SIGKILL is sent
  • No orphaned R processes

Directory overrides

daggle run my-dag --dags-dir /path/to/dags --data-dir /path/to/data

These override the default DAG discovery and data storage locations.

Authoring loop: daggle watch

While iterating on a DAG, run it once and re-run automatically on save:

daggle watch my-dag

Watches the DAG YAML plus every referenced script (script, validate, quarto, rmd, query_file, body_file, email attachments, etc.). Steps marked cache: true are skipped when their inputs haven’t changed, so only the affected steps re-execute. --debounce tunes how long to wait for the editor to finish writing (default 500ms). Press Ctrl+C to stop.

Pre-flight: daggle lint

Beyond daggle validate (schema/cycle checks), daggle lint runs semantic checks useful in CI or an editor problem matcher:

daggle lint my-dag --format gnu        # path:line:col: severity: msg [code]
daggle lint my-dag --format json       # for tooling
daggle lint my-dag --check-packages    # verify required R packages are installed

It flags missing referenced scripts, unresolvable ${env:} / ${file:} secrets, and unknown notify: / email.channel references. Exits 1 if any error-severity diagnostic is produced.