Pipeline file reference

Pipeline file reference#

An HTTomo pipeline file is a YAML sequence of method entries. HTTomo executes the entries from top to bottom, passing the main dataset from one method to the next. The first entry must be a loader.

YAML formatting#

YAML uses indentation to define structure. Use spaces rather than tabs, keep indentation consistent and include a space after each colon. Comments begin with #.

Common values include:

null: No value. This is converted to Python None.
true and false: Boolean values all in small letters. These are converted to Python True and False.
[value1, value2]: A list written on one line. Lists may also be written across multiple lines.

Strings normally do not need quotation marks. Quote a string when it contains YAML punctuation or might otherwise be interpreted as a number, Boolean or null value.

Method entries#

Each list entry configures one loader or processing method:

- method: median_filter3d
  module_path: tomopy.misc.corr
  parameters:
    size: 3

The supported fields are:

method: Required. The name of the Python function to run.
module_path: Required. The import path containing the function.
parameters: Required. A mapping of parameter names to values. Use parameters: {} when the method has no configurable parameters.
id: Optional. A unique name used when another method references this entry’s side output.
side_outputs: Optional. Maps an output name supplied by the method to a pipeline name. See Side outputs.
save_result: Optional Boolean. Controls whether the main dataset is saved after this method. See Save intermediate datasets.

Use Available methods for the supported method names, module paths, parameters and defaults.

Side-output references#

A later method can use a named side output with the following syntax:

${{method_id.side_outputs.output_name}}

The producing method must appear earlier in the pipeline, and every explicit id must be unique. See Side outputs for a complete example.

Parameter sweeps#

Use !Sweep for an explicit list of parameter values and !SweepRange for a start, stop and step. A pipeline can contain only one sweep. See Parameter Sweeping for syntax and output behaviour.

Minimal pipeline#

This example loads an NXtomo file, normalises the projections and applies the negative logarithm:

- method: standard_tomo
  module_path: httomo.data.hdf.loaders
  parameters:
    data_path: auto
    image_key_path: auto
    rotation_angles: auto

- method: normalize
  module_path: tomopy.prep.normalize
  parameters:
    cutoff: null
    averaging: mean

- method: minus_log
  module_path: tomopy.prep.normalize
  parameters: {}

Before running a pipeline, follow Validate the pipeline to validate its structure, methods, parameters and input-data paths.