Frequently asked questions

Frequently asked questions#

YAML and pipelines#

What is YAML and how does HTTomo use it?

YAML is a human-readable format for structured configuration files. HTTomo uses YAML to describe a pipeline containing the loader and processing methods.

See the Pipeline file reference for YAML formatting, supported method fields, side-output references and parameter-sweep syntax.

What is a YAML template?

A YAML template configures one loader or processing method. It identifies the method, its Python module and its configurable parameters.

Templates can be copied from Available methods and adapted for a particular dataset or processing task.

What is a pipeline?

A pipeline is a YAML file containing an ordered sequence of method templates. Older documentation may call it a process list.

The first entry must be a loader. Subsequent methods are executed in order from top to bottom, with each method receiving the data produced by the preceding operation.

See Pipelines and templates for an introduction to pipelines and templates.

How do I build a pipeline?

Start with a compatible loader template, then add processing templates in execution order. Edit the parameters for your data and processing requirements.

The recommended workflow is:

  1. Select methods from Available methods.

  2. Copy their templates into one YAML file.

  3. Place the loader first.

  4. Configure the method parameters.

  5. Validate the completed pipeline.

  6. Run the pipeline.

See Configure a pipeline for detailed instructions and Ready-to-use pipelines for complete examples.

How do I validate a pipeline?

Use the HTTomo YAML checker before running the pipeline:

python -m httomo check pipeline.yaml

You can optionally provide the input data file so that HTTomo also validates its dataset paths:

python -m httomo check pipeline.yaml input.nxs

The checker detects malformed YAML, unknown methods, invalid parameters and incompatible dataset paths. See Validate the pipeline for details.

Can I create a method template?

Yes. A template can be written manually or generated from a supported backend method. It must provide the correct method, module_path and parameters fields.

For reusable integration, add the method and its metadata to httomo-backends. See Adding a processing method.

Using and extending HTTomo#

How do I run HTTomo?

First install or load HTTomo, prepare a validated pipeline and select the input data.

See How to run HTTomo or Running HTTomo at Diamond when working at Diamond Light Source.

Can HTTomo run my own Python method?

Usually, provided that:

  • the method is importable in the HTTomo environment

  • its execution properties are described in httomo-backends

  • a YAML template is available

  • its inputs and outputs are compatible with an existing wrapper

A new wrapper may be required for methods with unusual inputs or outputs. See Adding a processing method for the integration procedure.

How can I contribute?

Contributions can add backend methods, improve documentation or modify the HTTomo source code.

See How to contribute for development guidance.

Where should I start when a run fails?

Read user.log and then debug.log in the run directory. Validate the pipeline against the input file with python -m httomo check and consult Troubleshooting for data, MPI, HDF5, CUDA and memory problems.