Full YAML pipelines#
This is a collection of ready to be used pipeline templates aka process lists. See more on What is a process list? and how to Configure efficient pipelines.
CPU Pipeline templates#
Basic TomoPy’s (CPU-only) pipeline for the classical 180-degrees scan
- method: standard_tomo
module_path: httomo.data.hdf.loaders
parameters:
name: tomo
data_path: entry1/tomo_entry/data/data
image_key_path: entry1/tomo_entry/instrument/detector/image_key
rotation_angles:
data_path: /entry1/tomo_entry/data/rotation_angle
- method: normalize
module_path: tomopy.prep.normalize
parameters:
cutoff: null
averaging: mean
- method: minus_log
module_path: tomopy.prep.normalize
parameters: {}
- method: find_center_vo
module_path: tomopy.recon.rotation
parameters:
ind: null
smin: -50
smax: 50
srad: 6
step: 0.25
ratio: 0.5
drop: 20
id: centering
side_outputs:
cor : centre_of_rotation
- method: recon
module_path: tomopy.recon.algorithm
parameters:
center: ${{centering.side_outputs.centre_of_rotation}}
sinogram_order: false
algorithm: 'gridrec'
init_recon: null
- method: save_to_images
module_path: httomolib.misc.images
parameters:
subfolder_name: images
axis: auto
file_format: tif
bits: 8
perc_range_min: 0.0
perc_range_max: 100.0
jpeg_quality: 95
TomoPy’s pipeline where Previewing is demonstrated
- method: standard_tomo
module_path: httomo.data.hdf.loaders
parameters:
name: tomo
data_path: entry1/tomo_entry/data/data
image_key_path: entry1/tomo_entry/instrument/detector/image_key
rotation_angles:
data_path: /entry1/tomo_entry/data/rotation_angle
preview:
detector_y:
start: 30
stop: 60
- method: find_center_vo
module_path: tomopy.recon.rotation
parameters:
ind: mid
smin: -50
smax: 50
srad: 6
step: 0.25
ratio: 0.5
drop: 20
id: centering
side_outputs:
cor: centre_of_rotation
- method: remove_outlier
module_path: tomopy.misc.corr
parameters:
dif: 0.1
size: 3
axis: auto
- method: normalize
module_path: tomopy.prep.normalize
parameters:
cutoff: null
averaging: mean
- method: minus_log
module_path: tomopy.prep.normalize
parameters: {}
- method: remove_stripe_fw
module_path: tomopy.prep.stripe
parameters:
level: null
wname: db5
sigma: 2
pad: true
- method: recon
module_path: tomopy.recon.algorithm
parameters:
center: ${{centering.side_outputs.centre_of_rotation}}
sinogram_order: false
algorithm: gridrec
init_recon: null
#additional parameters: AVAILABLE
save_result: true
- method: median_filter
module_path: tomopy.misc.corr
parameters:
size: 3
axis: auto
- method: save_to_images
module_path: httomolib.misc.images
parameters:
subfolder_name: images
axis: auto
file_format: tif
bits: 8
perc_range_min: 0.0
perc_range_max: 100.0
jpeg_quality: 95
This pipeline shows how “calculate_stats” module extracts global statistics in order to be passed to “save_to_images” function which uses it to rescale data for saving images
- method: standard_tomo
module_path: httomo.data.hdf.loaders
parameters:
name: tomo
data_path: entry1/tomo_entry/data/data
image_key_path: entry1/tomo_entry/instrument/detector/image_key
rotation_angles:
data_path: /entry1/tomo_entry/data/rotation_angle
preview:
detector_y:
start: 30
stop: 60
- method: find_center_vo
module_path: tomopy.recon.rotation
parameters:
ind: mid
smin: -50
smax: 50
srad: 6
step: 0.25
ratio: 0.5
drop: 20
id: centering
side_outputs:
cor: centre_of_rotation
- method: remove_outlier
module_path: tomopy.misc.corr
parameters:
dif: 0.1
size: 3
axis: auto
- method: normalize
module_path: tomopy.prep.normalize
parameters:
cutoff: null
averaging: mean
- method: minus_log
module_path: tomopy.prep.normalize
parameters: {}
- method: remove_stripe_fw
module_path: tomopy.prep.stripe
parameters:
level: null
wname: db5
sigma: 2
pad: true
- method: recon
module_path: tomopy.recon.algorithm
parameters:
center: ${{centering.side_outputs.centre_of_rotation}}
sinogram_order: false
algorithm: gridrec
init_recon: null
#additional parameters: AVAILABLE
save_result: true
- method: median_filter
module_path: tomopy.misc.corr
parameters:
size: 3
axis: auto
- method: save_to_images
module_path: httomolib.misc.images
parameters:
subfolder_name: images
axis: auto
file_format: tif
bits: 8
perc_range_min: 0.0
perc_range_max: 100.0
jpeg_quality: 95
GPU Pipeline templates#
Basic GPU pipeline which uses functions from the httomolibgpu library.
- method: standard_tomo
module_path: httomo.data.hdf.loaders
parameters:
name: tomo
data_path: entry1/tomo_entry/data/data
image_key_path: entry1/tomo_entry/instrument/detector/image_key
rotation_angles:
data_path: /entry1/tomo_entry/data/rotation_angle
dimension: 1
pad: 0
- method: find_center_vo
module_path: httomolibgpu.recon.rotation
parameters:
ind: mid
smin: -50
smax: 50
srad: 6.0
step: 0.25
ratio: 0.5
drop: 20
id: centering
side_outputs:
cor: centre_of_rotation
- method: remove_outlier
module_path: httomolibgpu.misc.corr
parameters:
dif: 0.1
kernel_size: 3
axis: auto
- method: normalize
module_path: httomolibgpu.prep.normalize
parameters:
cutoff: 10.0
minus_log: true
nonnegativity: false
remove_nans: false
- method: remove_stripe_based_sorting
module_path: httomolibgpu.prep.stripe
parameters:
size: 11
dim: 1
- method: FBP
module_path: httomolibgpu.recon.algorithm
parameters:
center: ${{centering.side_outputs.centre_of_rotation}}
filter_freq_cutoff: 0.6
recon_size: null
recon_mask_radius: null
save_result: true
- method: save_to_images
module_path: httomolib.misc.images
parameters:
subfolder_name: images
axis: auto
file_format: tif
bits: 8
perc_range_min: 0.0
perc_range_max: 100.0
jpeg_quality: 95
DLS Specific templates#
GPU-based pipeline using httomolibgpu methods for DIAD (k11) data. Global statistics and referencing is used.
- method: standard_tomo
module_path: httomo.data.hdf.loaders
parameters:
name: tomo
data_path: /entry/imaging/data
image_key_path: /entry/instrument/imaging/image_key
rotation_angles:
data_path: /entry/imaging_sum/gts_theta_value
dimension: 1
preview:
detector_y:
start: 8
stop: 15
pad: 0
- method: find_center_vo
module_path: httomolibgpu.recon.rotation
parameters:
ind: mid
smin: -50
smax: 50
srad: 6
step: 0.25
ratio: 0.5
drop: 20
id: centering
side_outputs:
cor: centre_of_rotation
- method: remove_outlier
module_path: httomolibgpu.misc.corr
parameters:
dif: 0.1
kernel_size: 3
axis: auto
- method: normalize
module_path: httomolibgpu.prep.normalize
parameters:
cutoff: 10.0
minus_log: true
nonnegativity: false
remove_nans: false
- method: remove_stripe_based_sorting
module_path: httomolibgpu.prep.stripe
parameters:
size: 11
dim: 1
- method: FBP
module_path: httomolibgpu.recon.algorithm
parameters:
center: ${{centering.side_outputs.centre_of_rotation}}
filter_freq_cutoff: 0.6
recon_size: null
recon_mask_radius: null
save_result: true
- method: calculate_stats
module_path: httomo.methods
parameters: {}
id: statistics
side_outputs:
glob_stats: glob_stats
- method: save_to_images
module_path: httomolib.misc.images
parameters:
subfolder_name: images
axis: auto
file_format: tif
bits: 8
perc_range_min: 0.0
perc_range_max: 95.0
jpeg_quality: 95
glob_stats: ${{statistics.side_outputs.glob_stats}}
GPU-driven pipeline for the 360-degrees data which estimates the CoR value and the overlap. The 180-degrees sinogram is obtained by stitching using the overlap value. The pipeline shows the extensive use of side_outputs and refrencing.
- method: standard_tomo
module_path: httomo.data.hdf.loaders
parameters:
name: tomo
data_path: entry1/tomo_entry/data/data
image_key_path: entry1/tomo_entry/instrument/detector/image_key
rotation_angles:
data_path: /entry1/tomo_entry/data/rotation_angle
- method: find_center_360
module_path: httomolibgpu.recon.rotation
parameters:
ind: mid
win_width: 10
side: null
denoise: true
norm: false
use_overlap: false
id: centering
side_outputs:
cor: centre_of_rotation
overlap: overlap
side: side
overlap_position: overlap_position
- method: remove_outlier
module_path: httomolibgpu.misc.corr
parameters:
dif: 0.1
kernel_size: 3
axis: auto
- method: normalize
module_path: httomolibgpu.prep.normalize
parameters:
cutoff: 10.0
minus_log: true
nonnegativity: false
remove_nans: false
- method: remove_stripe_based_sorting
module_path: httomolibgpu.prep.stripe
parameters:
size: 11
dim: 1
- method: sino_360_to_180
module_path: httomolibgpu.misc.morph
parameters:
overlap: ${{centering.side_outputs.overlap}}
rotation: right
- method: FBP
module_path: httomolibgpu.recon.algorithm
parameters:
center: ${{centering.side_outputs.centre_of_rotation}}
filter_freq_cutoff: 0.6
recon_size: null
recon_mask_radius: null
save_result: true
- method: calculate_stats
module_path: httomo.methods
parameters: {}
id: statistics
side_outputs:
glob_stats: glob_stats
- method: save_to_images
module_path: httomolib.misc.images
parameters:
subfolder_name: images
axis: auto
file_format: tif
bits: 8
perc_range_min: 0.0
perc_range_max: 95.0
jpeg_quality: 95
glob_stats: ${{statistics.side_outputs.glob_stats}}
More advanced GPU pipeline for the 360-degrees data. Here we preview the section and then reconstruct it iteratively, the result then downsampled before saving smaller images.
- method: standard_tomo
module_path: httomo.data.hdf.loaders
parameters:
name: tomo
data_path: entry1/tomo_entry/data/data
image_key_path: entry1/tomo_entry/instrument/detector/image_key
rotation_angles:
data_path: /entry1/tomo_entry/data/rotation_angle
dimension: 1
preview:
-
- start: 1000
stop: 1005
-
pad: 0
- method: find_center_360
module_path: httomolibgpu.recon.rotation
parameters:
ind: mid
win_width: 10
side: null
denoise: true
norm: false
use_overlap: false
id: centering
side_outputs:
cor: centre_of_rotation
overlap: overlap
side: side
overlap_position: overlap_position
- method: normalize
module_path: httomolibgpu.prep.normalize
parameters:
cutoff: 10.0
minus_log: true
nonnegativity: false
remove_nans: false
- method: sino_360_to_180
module_path: httomolibgpu.misc.morph
parameters:
overlap: ${{centering.side_outputs.overlap}}
rotation: right
- method: remove_all_stripe
module_path: httomolibgpu.prep.stripe
parameters:
snr: 3.0
la_size: 61
sm_size: 21
dim: 1
- method: CGLS
module_path: httomolibgpu.recon.algorithm
parameters:
center: ${{centering.side_outputs.centre_of_rotation}}
recon_size: null
iterations: 3
nonnegativity: true
save_result: true
- method: data_resampler
module_path: httomolibgpu.misc.morph
parameters:
newshape: [256, 256]
axis: auto
interpolation: nearest
- method: calculate_stats
module_path: httomo.methods
parameters: {}
id: statistics
side_outputs:
glob_stats: glob_stats
- method: save_to_images
module_path: httomolib.misc.images
parameters:
subfolder_name: images
axis: auto
file_format: tif
bits: 8
perc_range_min: 0.0
perc_range_max: 90.0
jpeg_quality: 95
glob_stats: ${{statistics.side_outputs.glob_stats}}
Parameter Sweeps templates#
Those templates demonstrate how to perform sweeps across multiple values a single parameter. See more on Parameter Tuning.
To be added in the forthcoming releases.