.. _explanation_pipelines_templates:

Pipelines and templates
=======================

An HTTomo *pipeline* is an ordered sequence of data-loading and processing
operations described using :ref:`YAML <explanation_yaml>` syntax. Older
material may call a pipeline a *process list*.

The pipeline is assembled from reusable *YAML templates*. Each template
configures one loader or a processing method.

.. _explanation_templates:

YAML templates
--------------

A method template contains:

- ``method``: the function to execute
- ``module_path``: the Python module containing the function
- ``parameters``: the values passed to the function

For example:

.. code-block:: yaml

   - method: median_filter3d
     module_path: tomopy.misc.corr
     parameters:
       size: 3

This entry tells HTTomo to import ``median_filter3d`` from ``tomopy.misc.corr``
and run it with ``size=3``.

Data parameters such as the input array are not included because HTTomo manages
data flow through its :ref:`wrapper layer <info_wrappers>`.

Ready-to-use templates are available in :ref:`reference_templates` and are
generated by the `HTTomo-backends YAML generator
<https://diamondlightsource.github.io/httomo-backends/utilities/yaml_generator.html>`_.
See :ref:`developers_httomo_backends` for the developer workflow.

.. _explanation_process_list:

Pipelines
---------

A pipeline combines templates in execution order. It must begin with a loader;
the remaining methods receive the output of the preceding operation.

For example:

.. code-block:: yaml

   - method: standard_tomo
     module_path: httomo.data.hdf.loaders
     parameters:
       data_path: entry1/tomo_entry/data/data
       image_key_path: entry1/tomo_entry/instrument/detector/image_key
       rotation_angles:
         data_path: entry1/tomo_entry/data/rotation_angle

   - method: normalize
     module_path: tomopy.prep.normalize
     parameters:
       cutoff: null
       averaging: mean

   - method: minus_log
     module_path: tomopy.prep.normalize
     parameters: {}

HTTomo reads this pipeline from top to bottom: it loads the data, normalises
it and then applies the negative logarithm.

Next steps
----------

- See :ref:`howto_process_list` to build and configure a pipeline.
- Browse :ref:`reference_templates` for supported method templates.
- See :ref:`explanation_yaml` for YAML syntax.
- Use the :ref:`YAML checker <utilities_yamlchecker>` before running a pipeline.
- Browse :ref:`tutorials_pl_templates` for complete example pipelines.