Secrets & Security

daggle resolves secrets at the start of a DAG run and injects them as environment variables into R steps. If any secret cannot be resolved, the DAG fails before any step executes.

Secret sources

Environment variables

Reference a process environment variable with ${env:VAR}:

steps:
  - id: fetch
    script: fetch.R
    env:
      API_KEY: "${env:MY_API_KEY}"

File contents

Read a secret from a file with ${file:/path}. The file contents are trimmed of leading/trailing whitespace.

steps:
  - id: fetch
    script: fetch.R
    env:
      API_KEY: "${file:/run/secrets/api_key}"

HashiCorp Vault

Read from Vault KV v2 with ${vault:path#field}:

steps:
  - id: fetch
    script: fetch.R
    env:
      DB_PASSWORD: "${vault:secret/data/myapp#db_password}"
      API_KEY: "${vault:secret/data/myapp#api_key}"

Fail-fast behavior

All secrets are resolved before the first step runs. If any source is unavailable – a missing environment variable, an unreadable file, or a Vault path that does not exist – the DAG fails immediately with a clear error message identifying the unresolved secret.

Vault integration

daggle authenticates to Vault using:

  1. VAULT_ADDR environment variable (required)
  2. VAULT_TOKEN environment variable, or if not set, the contents of ~/.vault-token

The path format follows Vault KV v2 conventions: secret/data/<path>#<field>.

Redaction

Secrets from ${vault:} and ${file:} sources are automatically redacted in:

  • events.jsonl lifecycle events
  • daggle status output
  • API responses

Environment variable sources (${env:}) are not redacted by default, since the value may not actually be sensitive. To opt in to redaction, add secret: true:

steps:
  - id: fetch
    script: fetch.R
    env:
      PUBLIC_URL: "${env:SERVICE_URL}"
      API_KEY:
        value: "${env:MY_API_KEY}"
        secret: true

With secret: true, the value is treated identically to vault and file sources for redaction purposes.