Inter-Step Data Passing

Steps pass data to downstream steps using stdout markers. daggle parses these markers, strips them from terminal output, and passes the values as environment variables.

Emitting and reading outputs

In an R step, use the daggleR helpers — daggle_output() to emit and daggle_get_output() to read:

- id: extract
  r_expr: |
    daggleR::daggle_output("row_count", nrow(data))

- id: report
  r_expr: |
    rows <- as.integer(daggleR::daggle_get_output("extract", "row_count"))
    cat(sprintf("Processing %d rows\n", rows))
  depends: [extract]

daggle_output() validates the key format (must match [a-zA-Z_][a-zA-Z0-9_]*), coerces the value to character, and prints a single line. Each emitted value is single-line — for complex data, write to a file and pass the path.

Wire format and raw protocol

daggleR is a thin convenience layer over a stdout protocol any language can emit. Shell steps and non-R steps emit it directly:

# Emit
daggleR::daggle_output("row_count", nrow(data))

# Read
n <- as.integer(daggleR::daggle_get_output("extract", "row_count"))

# Emit (R)
cat(sprintf("::daggle-output name=row_count::%d\n", nrow(data)))

# Read (R)
n <- as.integer(Sys.getenv("DAGGLE_OUTPUT_EXTRACT_ROW_COUNT"))

# Emit (shell)
echo "::daggle-output name=row_count::$count"

Format: ::daggle-output name=<key>::<value>. Downstream steps receive outputs as environment variables DAGGLE_OUTPUT_<STEP_ID>_<KEY>, with the step ID uppercased and hyphens replaced by underscores (e.g. step fit-lda emitting accuracy becomes DAGGLE_OUTPUT_FIT_LDA_ACCURACY). Full spec: Output Marker Protocol.

Log handling

Output marker lines are stripped from terminal output (you don’t see them while watching the run)
They are kept in log files (<step>.stdout.log) for debugging
Multiple outputs per step are supported – emit as many markers as needed