Side outputs

Side outputs#

There are cases where the output of a method is needed as the value of a parameter for a method further down the pipeline. For example, the output of a method that calculates the Centre of Rotation (CoR), that is required for a reconstruction method.

HTTomo provides a special syntax (loosely based on the syntax for references in GitHub Actions) for how such an output of a method needs to be defined and how to refer to that special output later.

Specifying the side output#

The output of some methods isn’t processed data, but rather supplementary information to be used later in the pipeline. The given term for that supplementary data is “side outputs”, and is what the side_outputs parameter is for. As an example, let us consider the following centering algorithm:

- method: find_center_vo
  module_path: httomolibgpu.recon.rotation
  parameters:
    ind: null
    smin: -50
    smax: 50
    srad: 6.0
    step: 0.25
    ratio: 0.5
    drop: 20
  id: centering
  side_outputs:
    cor: centre_of_rotation

One can see that side_outputs here includes a single value cor with the centre_of_rotation reference. The id parameter here is needed to refer to the method later.

Referring to the side output#

The purpose of side_outputs is to refer to it later, when some method(s) require the contained information in the reference. Consider this example where the reconstruction method refers to the centering method’s side outputs. The required information of Centre of Rotation (CoR) is stored in the reference ${{centering.side_outputs.centre_of_rotation}}.

- method: FBP3d_tomobar
  module_path: httomolibgpu.recon.algorithm
  parameters:
    center: ${{centering.side_outputs.centre_of_rotation}}
    filter_freq_cutoff: 1.1
    recon_size: null
    recon_mask_radius: null

There could be various configurations when this reference is required from other methods as well. We present more verbose Example of side outputs below.

Note

Side outputs and references to them are generated automatically with the YAML generator. Usually there is no need to modify them when editing a process list.

Example of side outputs#

Pipeline overview#

This pipeline is for reconstructing DFoV data which needs to be stitched into the traditional 180 degrees data. It demonstrates three cases where a method produces one or more side outputs, and a method later in the pipeline references them.

A rough outline of the three side outputs being used is:

  1. the 360 centering method produces an “overlap” value as a side output, for the stitching method to use

  2. the 360 centering method also produces a CoR value as a side output, for the recon method to use

  3. the stats calculator method produces a global stats value as a side output, for the image saver method to use

Detailed look at side outputs usage#

Parameters for stitching are generated by find_center_360, stored in side outputs, and then used later in the sino_360_to_180 method. The reconstruction method FBP3d_tomobar then refers to the found Centre of Rotation (CoR) for the stitched dataset produced by find_center_360. Finally, we also need to extract the global statistics for normalisation of the data using the calculate_stats method when saving into images with the save_to_images method.

Listing 1 Pipeline for 360 degrees scan, a double field of view (DFoV) data.#
# This pipeline should be supported by the latest developments of HTTomo. Use module load httomo/latest module at Diamond.
# --- Standard tomography loader for NeXus files. ---
- method: standard_tomo
  module_path: httomo.data.hdf.loaders
  parameters:
    data_path: auto
    image_key_path: auto
    rotation_angles: auto
    preview:
      detector_x:   # horizontal data previewing/cropping.
        # when null, the full data dimension is used, i.e., no previewing
        start: null
        stop: null
      detector_y:   # vertical data previewing/cropping.
        # when null, the full data dimension is used, i.e., no previewing
        start: null
        stop: null
    darks: null
    flats: null
# --- Center of Rotation auto-finding. Required for reconstruction bellow. ---
- method: find_center_360
  module_path: httomolibgpu.recon.rotation
  parameters:
    ind: null  # A vertical slice (sinogram) index to calculate CoR, 'mid' can be used for middle
    win_width: 10
    side: null # 'None' corresponds to fully automated determination, 'left' to the left side, 'right' to the right side.
    denoise: true
    norm: false
    use_overlap: false
  id: centering
  side_outputs:
    cor: centre_of_rotation  # An estimated CoR value provided as a side output
    overlap: overlap # An overlap to use for converting 360 degrees scan to 180 degrees scan.
    side: side
    overlap_position: overlap_position
# --- Normalisation of projection data using collected flats/darks images. --- 
- method: normalize
  module_path: httomolibgpu.prep.normalize
  parameters:
    flats_multiplier: 1.0
    darks_multiplier: 1.0
    cutoff: 10.0
    minus_log: true  # If Paganin method is used bellow, set it to 'false'.
    nonnegativity: false
    remove_nans: false
# --- Applying optical distortion correction to projections. --- 
- method: distortion_correction_proj_discorpy
  module_path: httomolibgpu.prep.alignment
  parameters:
    metadata_path: REQUIRED  # Provide an absolute path to the text file with distortion coefficients.
    order: 3
    mode: constant
# --- Using the overlap and side provided, converting 360 degrees scan to 180 degrees scan. --- 
- method: sino_360_to_180
  module_path: httomolibgpu.misc.morph
  parameters:
    overlap: ${{centering.side_outputs.overlap}}
    side: ${{centering.side_outputs.side}}
# --- Method to remove stripe artefacts in the data that lead to ring artefacts in the reconstruction. --- 
- method: remove_stripe_based_sorting
  module_path: httomolibgpu.prep.stripe
  parameters:
    size: 11
    dim: 1
# --- Reconstruction method. ---
- method: FBP3d_tomobar
  module_path: httomolibgpu.recon.algorithm
  parameters:
    center: ${{centering.side_outputs.centre_of_rotation}}  # Reference to center of rotation side output above OR a float number.
    detector_pad: false # Horizontal detector padding to minimise circle/arc-type artifacts in the reconstruction. Set to 'true' to enable automatic padding or an integer
    filter_freq_cutoff: 0.35
    recon_size: null
    recon_mask_radius: 0.95 # Zero pixels outside the mask-circle radius. Make radius equal to 2.0 to remove the mask effect.
    neglog: false # Perform negative log here if it was previously switched off.
# --- Calculate global statistics on the reconstructed volume, required for data rescaling. ---
- method: calculate_stats
  module_path: httomo.methods
  parameters: {}
  id: statistics
  side_outputs:
    glob_stats: glob_stats
# --- Rescaling the data using min/max obtained from `calculate_stats`. ---
- method: rescale_to_int
  module_path: httomolib.misc.rescale
  parameters:
    perc_range_min: 0.0
    perc_range_max: 100.0
    bits: 8
    glob_stats: ${{statistics.side_outputs.glob_stats}}
# --- Saving data into images. ---
- method: save_to_images
  module_path: httomolib.misc.images
  parameters:
    subfolder_name: images
    axis: auto
    file_format: tif  # `tif` or `jpeg` can be used.
    asynchronous: true

Please see below for a concise description of how each of the individual side outputs of methods are connected to the method using them:

  1. find_center_360 produces side outputs called overlap and side that the sino_360_to_180 method uses.

  2. find_center_360 also produces a side output called centre_of_rotation that the FBP3d_tomobar method uses.

  3. calculate_stats produces a side output called glob_stats that the rescale_to_int method uses.

Contents