Optimise pipeline performance

Pipeline performance is influenced mainly by method order, CPU/GPU data transfers and intermediate file output. Understanding Sections and Re-slicing can help when applying the guidance below.

Group methods by data pattern

HTTomo runs pipeline methods sequentially from top to bottom. Each method has one of three data patterns:

projection

Data is sliced by projection.

sinogram

Data is sliced by sinogram.

all

The method inherits the pattern of the preceding method.

Changing between projection and sinogram requires a potentially costly re-slice. Where processing requirements allow, group methods with the same pattern to reduce the number of re-slices.

HTTomo loaders use the projection pattern, so start with projection-based methods where possible. Centre-finding methods should normally be placed near the start of the pipeline.

Method metadata

HTTomo obtains each method’s pattern, implementation type and memory requirements from httomo-backends. See the httomo-backends method metadata documentation for details.

Group GPU methods

Supported methods use one of three implementation types:

cpu

Runs on the CPU.

gpu

Runs on a GPU but receives its input as a NumPy array in CPU memory.

gpu_cupy

Runs on a GPU using CuPy arrays. Data remains in GPU memory between consecutive gpu_cupy methods.

When a GPU is available, prefer GPU implementations where appropriate and keep gpu_cupy methods together to reduce transfers between CPU and GPU memory. Available implementations are listed in Processing libraries.

Minimise writing to disk

Writing intermediate datasets can significantly slow a pipeline and consume substantial disk space. Use save_result or --save-all only when those intermediate results are needed. Use --save-snapshots to capture lightweight diagnostic snapshots instead. See Save intermediate datasets for details.