Full YAML pipelines#
This is a collection of ready to be used pipeline templates aka process lists. See more on What is a process list? and how to Configure efficient pipelines.
CPU Pipeline templates#
Basic TomoPy’s (CPU-only) pipeline for the classical 180-degrees scan
- method: standard_tomo
module_path: httomo.data.hdf.loaders
parameters:
data_path: entry1/tomo_entry/data/data
image_key_path: entry1/tomo_entry/instrument/detector/image_key
rotation_angles:
data_path: /entry1/tomo_entry/data/rotation_angle
- method: normalize
module_path: tomopy.prep.normalize
parameters:
cutoff: null
averaging: mean
- method: minus_log
module_path: tomopy.prep.normalize
parameters: {}
- method: find_center_vo
module_path: tomopy.recon.rotation
parameters:
ind: null
smin: -50
smax: 50
srad: 6
step: 0.25
ratio: 0.5
drop: 20
id: centering
side_outputs:
cor : centre_of_rotation
- method: recon
module_path: tomopy.recon.algorithm
parameters:
center: ${{centering.side_outputs.centre_of_rotation}}
sinogram_order: false
algorithm: 'gridrec'
init_recon: null
- method: rescale_to_int
module_path: httomolibgpu.misc.rescale
parameters:
perc_range_min: 0.0
perc_range_max: 100.0
bits: 8
- method: save_to_images
module_path: httomolib.misc.images
parameters:
subfolder_name: images
axis: auto
file_format: tif
jpeg_quality: 95
TomoPy’s pipeline where Previewing is demonstrated
- method: standard_tomo
module_path: httomo.data.hdf.loaders
parameters:
data_path: entry1/tomo_entry/data/data
image_key_path: entry1/tomo_entry/instrument/detector/image_key
rotation_angles:
data_path: /entry1/tomo_entry/data/rotation_angle
preview:
detector_y:
start: 30
stop: 60
- method: find_center_vo
module_path: tomopy.recon.rotation
parameters:
ind: mid
smin: -50
smax: 50
srad: 6
step: 0.25
ratio: 0.5
drop: 20
id: centering
side_outputs:
cor: centre_of_rotation
- method: remove_outlier
module_path: tomopy.misc.corr
parameters:
dif: 0.1
size: 3
axis: auto
- method: normalize
module_path: tomopy.prep.normalize
parameters:
cutoff: null
averaging: mean
- method: minus_log
module_path: tomopy.prep.normalize
parameters: {}
- method: remove_stripe_fw
module_path: tomopy.prep.stripe
parameters:
level: null
wname: db5
sigma: 2
pad: true
- method: recon
module_path: tomopy.recon.algorithm
parameters:
center: ${{centering.side_outputs.centre_of_rotation}}
sinogram_order: false
algorithm: gridrec
init_recon: null
#additional parameters: AVAILABLE
save_result: true
- method: median_filter
module_path: tomopy.misc.corr
parameters:
size: 3
axis: auto
- method: rescale_to_int
module_path: httomolibgpu.misc.rescale
parameters:
perc_range_min: 0.0
perc_range_max: 100.0
bits: 8
- method: save_to_images
module_path: httomolib.misc.images
parameters:
subfolder_name: images
axis: auto
file_format: tif
jpeg_quality: 95
This pipeline shows how “calculate_stats” module extracts global statistics in order to be passed to “save_to_images” function which uses it to rescale data for saving images
- method: standard_tomo
module_path: httomo.data.hdf.loaders
parameters:
data_path: entry1/tomo_entry/data/data
image_key_path: entry1/tomo_entry/instrument/detector/image_key
rotation_angles:
data_path: /entry1/tomo_entry/data/rotation_angle
- method: find_center_vo
module_path: tomopy.recon.rotation
parameters:
ind: mid
smin: -50
smax: 50
srad: 6
step: 0.25
ratio: 0.5
drop: 20
id: centering
side_outputs:
cor: centre_of_rotation
- method: remove_outlier
module_path: tomopy.misc.corr
parameters:
dif: 0.1
size: 3
axis: auto
- method: normalize
module_path: tomopy.prep.normalize
parameters:
cutoff: null
averaging: mean
- method: minus_log
module_path: tomopy.prep.normalize
parameters: {}
- method: recon
module_path: tomopy.recon.algorithm
parameters:
center: ${{centering.side_outputs.centre_of_rotation}}
sinogram_order: false
algorithm: gridrec
init_recon: null
#additional parameters: AVAILABLE
- method: calculate_stats
module_path: httomo.methods
parameters: {}
id: statistics
side_outputs:
glob_stats: glob_stats
- method: rescale_to_int
module_path: httomolibgpu.misc.rescale
parameters:
perc_range_min: 0.0
perc_range_max: 100.0
bits: 8
glob_stats: ${{statistics.side_outputs.glob_stats}}
- method: save_to_images
module_path: httomolib.misc.images
parameters:
subfolder_name: images
axis: auto
file_format: tif
jpeg_quality: 95
GPU Pipeline templates#
Basic GPU pipeline which uses functions from the httomolibgpu library.
- method: standard_tomo
module_path: httomo.data.hdf.loaders
parameters:
data_path: entry1/tomo_entry/data/data
image_key_path: entry1/tomo_entry/instrument/detector/image_key
rotation_angles:
data_path: /entry1/tomo_entry/data/rotation_angle
- method: find_center_vo
module_path: httomolibgpu.recon.rotation
parameters:
ind: mid
smin: -50
smax: 50
srad: 6.0
step: 0.25
ratio: 0.5
drop: 20
id: centering
side_outputs:
cor: centre_of_rotation
- method: remove_outlier
module_path: httomolibgpu.misc.corr
parameters:
dif: 0.1
kernel_size: 3
- method: normalize
module_path: httomolibgpu.prep.normalize
parameters:
cutoff: 10.0
minus_log: true
nonnegativity: false
remove_nans: false
- method: remove_stripe_based_sorting
module_path: httomolibgpu.prep.stripe
parameters:
size: 11
dim: 1
- method: FBP
module_path: httomolibgpu.recon.algorithm
parameters:
center: ${{centering.side_outputs.centre_of_rotation}}
filter_freq_cutoff: 0.6
recon_size: null
recon_mask_radius: null
save_result: true
- method: rescale_to_int
module_path: httomolibgpu.misc.rescale
parameters:
perc_range_min: 0.0
perc_range_max: 100.0
bits: 8
- method: save_to_images
module_path: httomolib.misc.images
parameters:
subfolder_name: images
axis: auto
file_format: tif
jpeg_quality: 95
DLS Specific templates#
GPU-based pipeline using httomolibgpu methods for DIAD (k11) data. Global statistics and referencing is used.
- method: standard_tomo
module_path: httomo.data.hdf.loaders
parameters:
data_path: /entry/imaging/data
image_key_path: /entry/instrument/imaging/image_key
rotation_angles:
data_path: /entry/imaging_sum/gts_theta_value
preview:
detector_y:
start: 8
stop: 15
- method: find_center_vo
module_path: httomolibgpu.recon.rotation
parameters:
ind: mid
smin: -50
smax: 50
srad: 6
step: 0.25
ratio: 0.5
drop: 20
id: centering
side_outputs:
cor: centre_of_rotation
- method: remove_outlier
module_path: httomolibgpu.misc.corr
parameters:
dif: 0.1
kernel_size: 3
- method: normalize
module_path: httomolibgpu.prep.normalize
parameters:
cutoff: 10.0
minus_log: true
nonnegativity: false
remove_nans: false
- method: remove_stripe_based_sorting
module_path: httomolibgpu.prep.stripe
parameters:
size: 11
dim: 1
- method: FBP
module_path: httomolibgpu.recon.algorithm
parameters:
center: ${{centering.side_outputs.centre_of_rotation}}
filter_freq_cutoff: 0.6
recon_size: null
recon_mask_radius: null
save_result: true
- method: calculate_stats
module_path: httomo.methods
parameters: {}
id: statistics
side_outputs:
glob_stats: glob_stats
- method: rescale_to_int
module_path: httomolibgpu.misc.rescale
parameters:
perc_range_min: 0.0
perc_range_max: 100.0
bits: 8
glob_stats: ${{statistics.side_outputs.glob_stats}}
- method: save_to_images
module_path: httomolib.misc.images
parameters:
subfolder_name: images
axis: auto
file_format: tif
jpeg_quality: 95
GPU-driven pipeline for the 360-degrees data which estimates the CoR value and the overlap. The 180-degrees sinogram is obtained by stitching using the overlap value. The pipeline shows the extensive use of side_outputs and refrencing.
- method: standard_tomo
module_path: httomo.data.hdf.loaders
parameters:
data_path: entry1/tomo_entry/data/data
image_key_path: entry1/tomo_entry/instrument/detector/image_key
rotation_angles:
data_path: /entry1/tomo_entry/data/rotation_angle
- method: find_center_360
module_path: httomolibgpu.recon.rotation
parameters:
ind: mid
win_width: 10
side: null
denoise: true
norm: false
use_overlap: false
id: centering
side_outputs:
cor: centre_of_rotation
overlap: overlap
side: side
overlap_position: overlap_position
- method: remove_outlier
module_path: httomolibgpu.misc.corr
parameters:
dif: 0.1
kernel_size: 3
- method: normalize
module_path: httomolibgpu.prep.normalize
parameters:
cutoff: 10.0
minus_log: true
nonnegativity: false
remove_nans: false
- method: remove_stripe_based_sorting
module_path: httomolibgpu.prep.stripe
parameters:
size: 11
dim: 1
- method: sino_360_to_180
module_path: httomolibgpu.misc.morph
parameters:
overlap: ${{centering.side_outputs.overlap}}
rotation: right
- method: FBP
module_path: httomolibgpu.recon.algorithm
parameters:
center: ${{centering.side_outputs.centre_of_rotation}}
filter_freq_cutoff: 0.6
recon_size: null
recon_mask_radius: null
save_result: true
- method: data_resampler
module_path: httomolibgpu.misc.morph
parameters:
newshape: [500, 500]
axis: auto
interpolation: linear
- method: calculate_stats
module_path: httomo.methods
parameters: {}
id: statistics
side_outputs:
glob_stats: glob_stats
- method: rescale_to_int
module_path: httomolibgpu.misc.rescale
parameters:
perc_range_min: 0.0
perc_range_max: 100.0
bits: 8
glob_stats: ${{statistics.side_outputs.glob_stats}}
- method: save_to_images
module_path: httomolib.misc.images
parameters:
subfolder_name: images
axis: auto
file_format: tif
jpeg_quality: 95
More advanced GPU pipeline for the 360-degrees data. Here we preview the section and then reconstruct it iteratively, the result then downsampled before saving smaller images.
- method: standard_tomo
module_path: httomo.data.hdf.loaders
parameters:
data_path: entry1/tomo_entry/data/data
image_key_path: entry1/tomo_entry/instrument/detector/image_key
rotation_angles:
data_path: /entry1/tomo_entry/data/rotation_angle
preview:
detector_y:
start: 1000
stop: 1005
- method: find_center_360
module_path: httomolibgpu.recon.rotation
parameters:
ind: mid
win_width: 10
side: null
denoise: true
norm: false
use_overlap: false
id: centering
side_outputs:
cor: centre_of_rotation
overlap: overlap
side: side
overlap_position: overlap_position
- method: normalize
module_path: httomolibgpu.prep.normalize
parameters:
cutoff: 10.0
minus_log: true
nonnegativity: false
remove_nans: false
- method: sino_360_to_180
module_path: httomolibgpu.misc.morph
parameters:
overlap: ${{centering.side_outputs.overlap}}
rotation: right
- method: remove_all_stripe
module_path: httomolibgpu.prep.stripe
parameters:
snr: 3.0
la_size: 61
sm_size: 21
dim: 1
- method: CGLS
module_path: httomolibgpu.recon.algorithm
parameters:
center: ${{centering.side_outputs.centre_of_rotation}}
recon_size: null
iterations: 3
nonnegativity: true
save_result: true
- method: data_resampler
module_path: httomolibgpu.misc.morph
parameters:
newshape: [256, 256]
axis: auto
interpolation: nearest
- method: calculate_stats
module_path: httomo.methods
parameters: {}
id: statistics
side_outputs:
glob_stats: glob_stats
- method: rescale_to_int
module_path: httomolibgpu.misc.rescale
parameters:
perc_range_min: 0.0
perc_range_max: 100.0
bits: 8
glob_stats: ${{statistics.side_outputs.glob_stats}}
- method: save_to_images
module_path: httomolib.misc.images
parameters:
subfolder_name: images
axis: auto
file_format: tif
jpeg_quality: 95
Parameter Sweeps templates#
These templates demonstrate how to perform a sweep across multiple values of a single parameter (see Parameter Sweeping for more details).
Parameter sweep over 6 CoR values (center param) in recon method, and saving the result as tiffs. Note that there is need to add image saving plugin in this case. It is also preferable to keep preview small.
- method: standard_tomo
module_path: httomo.data.hdf.loaders
parameters:
data_path: entry1/tomo_entry/data/data
image_key_path: entry1/tomo_entry/instrument/detector/image_key
rotation_angles:
data_path: /entry1/tomo_entry/data/rotation_angle
preview:
detector_y:
start: 60
stop: 67
- method: normalize
module_path: httomolibgpu.prep.normalize
parameters:
cutoff: 10.0
minus_log: false
nonnegativity: false
remove_nans: true
- method: paganin_filter_tomopy
module_path: httomolibgpu.prep.phase
parameters:
pixel_size: 0.0001
dist: 50.0
energy: 53.0
alpha: 0.001
- method: FBP
module_path: httomolibgpu.recon.algorithm
save_result: False
parameters:
center: !SweepRange
start: 60
stop: 120
step: 10
filter_freq_cutoff: 1.1
recon_size: null
recon_mask_radius: null
Parameter sweep over 50 (alpha param) values of Paganin filter method, and saving the result as tiffs for both Paganin filter and the reconstruction module.
- method: standard_tomo
module_path: httomo.data.hdf.loaders
parameters:
data_path: entry1/tomo_entry/data/data
image_key_path: entry1/tomo_entry/instrument/detector/image_key
rotation_angles:
data_path: /entry1/tomo_entry/data/rotation_angle
preview:
detector_y:
start: 60
stop: 67
- method: normalize
module_path: httomolibgpu.prep.normalize
parameters:
cutoff: 10.0
minus_log: false
nonnegativity: false
remove_nans: true
- method: paganin_filter_tomopy
module_path: httomolibgpu.prep.phase
parameters:
pixel_size: 0.0004
dist: 50.0
energy: 53.0
alpha: !SweepRange
start: 0.001
stop: 0.5
step: 0.01
- method: FBP
module_path: httomolibgpu.recon.algorithm
parameters:
center: 80
filter_freq_cutoff: 1.1
recon_size: null
recon_mask_radius: null