AreaDetector Tutorial

You should already know how to create a Block in the Device Layer that looks like a detector, and how to integrate it into a Scan Layer Block, using a DetectorChildPart. Now let’s build a Detector Block to control an EPICS areaDetector simDetector and its plugin chain, and integrate it into a scan.

While we use simDetector for this tutorial, support for other detectors can be found in the ./malcolm/modules directory.

Acquisition Strategy

The application we have in mind is a multi-dimensional continuous scan, so we want to be able to take a number of frames with the detector driver, calculate some statistics on them, and write them in the same dimensionality as the scan suggests into a NeXus formatted HDF5 file. The driver and each plugin in the chain will be represented by a Block in the Hardware Layer, and they will all be controlled detector Block in the Device Layer. This is best viewed as a diagram:

digraph simDetector_child_connections { bgcolor=transparent compound=true node [fontname=Arial fontsize=10 shape=rect style=filled fillcolor="#8BC4E9"] graph [fontname=Arial fontsize=10] edge [fontname=Arial fontsize=10 arrowhead=vee] subgraph cluster_device { label="Device Layer" style=filled color=lightgrey subgraph cluster_detector { label="DETECTOR" ranksep=0.1 color=white detector_c [label="RunnableController"] DRV [label=<SimDetectorDriverPart<BR/>name: 'DRV'>] POS [label=<PositionLabellerPart<BR/>name: 'POS'>] STAT [label=<StatsPluginPart<BR/>name: 'STAT'>] HDF [label=<HDFWriterPart<BR/>name: 'HDF'>] DSET [label=<DatasetTablePart<BR/>name: 'DSET'>] detector_c -> DRV [style=invis] detector_c -> HDF [style=invis] DRV -> DSET [style=invis] {rank=same; DRV -> POS -> STAT -> HDF} } } subgraph cluster_hardware { label="Hardware Layer" style=filled color=lightgrey subgraph cluster_drv { label="DETECTOR:DRV" color=white drv_c [label="StatefulController"] drv_p [label="CAParts"] drv_c -> drv_p [style=invis] } subgraph cluster_pos { label="DETECTOR:POS" color=white pos_c [label="StatefulController"] pos_p [label="CAParts"] pos_c -> pos_p [style=invis] } subgraph cluster_stat { label="DETECTOR:STAT" color=white stat_c [label="StatefulController"] stat_p [label="CAParts"] stat_c -> stat_p [style=invis] } subgraph cluster_hdf { label="DETECTOR:HDF" color=white hdf_c [label="StatefulController"] hdf_p [label="CAParts"] hdf_c -> hdf_p [style=invis] } } DRV -> drv_c [lhead=cluster_drv minlen=3 style=dashed] POS -> pos_c [lhead=cluster_pos minlen=3 style=dashed] STAT -> stat_c [lhead=cluster_stat minlen=3 style=dashed] HDF -> hdf_c [lhead=cluster_hdf minlen=3 style=dashed] }

Note

There is a separation and hence an interface between Part and child Block. The interface goes in the child Block, and the logic goes in the controlling Part. This is desirable because we could potentially have many possible logic Parts that could control the same kind of child Block, and making this split keeps the Parts small and more readable.

Each Hardware Block is responsible for controlling a group of PVs that make up a single plugin or driver:

  • The DRV Block corresponds to the simDetector driver, which is responsible for producing the right number of NDArrays, each tagged with a unique ID.

  • The POS Block corresponds to the NDPosPlugin plugin which tags each NDArray with a number of attributes that can be used to determine its position within the dataset dimensions.

  • The STAT Block corresponds to the NDPluginStats plugin which tags each NDArray with a number of statistics that can be calculated from the data.

  • The HDF Block corresponds to the NDFileHDF5 plugin which writes NDArrays into an HDF file, getting the position within the dataset dimensions from an attribute attached to the NDArray.

The detector Device Block contains 4 Parts, one for each Hardware Block, that are responsible for setting Attributes on the relevant child Block in the right order. The Controller is responsible for calling each of its Parts Hooked methods in the right order.

Note

Malcolm’s role in this application is purely supervisory, it just sets up the underlying plugins and presses Acquire. EPICS is responsible for writing data

Creating the Blocks

So let’s start with the Process Definition ./malcolm/modules/demo/DEMO-AREADETECTOR.yaml:

# To start the IOC, run Launcher -> Utilities -> GDA AreaDetector Simulation
- builtin.defines.cmd_string:
    name: hostname
    cmd: hostname -s

- builtin.defines.export_env_string:
    name: EPICS_CA_SERVER_PORT
    value: 6064

- builtin.defines.export_env_string:
    name: EPICS_CA_REPEATER_PORT
    value: 6065

# Define a directory to store config in
- builtin.defines.tmp_dir:
    name: config_dir

# Create some Blocks
- demo.blocks.motion_block:
    mri: $(hostname)-ML-MOT-01
    config_dir: $(config_dir)

- demo.blocks.detector_block:
    mri: $(hostname)-ML-DET-01
    config_dir: $(config_dir)
    label: Interference detector

- ADSimDetector.blocks.sim_detector_runnable_block:
    mri_prefix: $(hostname)-ML-DET-02
    config_dir: $(config_dir)
    pv_prefix: $(hostname)-AD-SIM-01
    label: Ramp detector
    drv_suffix: CAM

- demo.blocks.scan_2det_block:
    mri: $(hostname)-ML-SCAN-01
    config_dir: $(config_dir)
    initial_design: template_both_detectors

- system.blocks.system_block:
    mri_prefix: $(hostname)-ML-MALC-01
    iocs: $(hostname)-EA-IOC-01
    pv_prefix: $(hostname)-ML-MALC-01
    subnet_validation: 0
    config_dir: $(config_dir)

We have a couple more items to explain than in previous examples:

  • The builtin.defines.cmd_string entry runs the shell command hostname -s and makes it available inside this YAML file as $(hostname). This is needed because we are interfacing to an IOC that calculates the PV prefix based on the machine we are currently running on.

  • The builtin.defines.export_env_string entries are so that we can export the EPICS server and repeater ports, again required as the IOC runs on these ports.

The other items are Blocks just like we encountered in previous tutorials.

Device Block

The top level Device Block is a sim_detector_runnable_block. Let’s take a look at ./malcolm/modules/ADSimDetector/blocks/sim_detector_runnable_block.yaml to see what one of those looks like:

- builtin.parameters.string:
    name: mri_prefix
    description: Malcolm resource id of the Block and prefix for children

- builtin.parameters.string:
    name: pv_prefix
    description: PV prefix for driver and all plugins

- builtin.parameters.string:
    name: config_dir
    description: Where to store saved configs

- builtin.parameters.string:
    name: label
    description: Beamline specific label for the detector
    default: SimDetector

- builtin.parameters.string:
    name: drv_suffix
    description: PV suffix for detector driver
    default: DET

- builtin.defines.docstring:
    value: |
      Device Block corresponding to SimDetector + stat + pos + hdf writer.

      - Detector driver should have pv prefix $(pv_prefix):$(drv_suffix)
      - Pos should have pv prefix $(pv_prefix):POS
      - Stat should have pv prefix $(pv_prefix):STAT
      - HDF should have pv prefix $(pv_prefix):HDF5

- scanning.controllers.RunnableController:
    mri: $(mri_prefix)
    config_dir: $(config_dir)
    template_designs: $(yamldir)/$(yamlname)_designs
    description: |
      SimDetector produces a simulated detector image, with either a Linear
      ramp, array of peaks, or sine wave function used to make the 2D image

- builtin.parts.LabelPart:
    value: $(label)

- ADSimDetector.blocks.sim_detector_driver_block:
    mri: $(mri_prefix):DRV
    prefix: $(pv_prefix):$(drv_suffix)

- ADCore.parts.DetectorDriverPart:
    name: DRV
    mri: $(mri_prefix):DRV
    soft_trigger_modes:
        - Internal

- scanning.parts.ExposureDeadtimePart:
    name: DEADTIME

- ADCore.blocks.stats_plugin_block:
    mri: $(mri_prefix):STAT
    prefix: $(pv_prefix):STAT

- ADCore.parts.StatsPluginPart:
    name: STAT
    mri: $(mri_prefix):STAT

- ADCore.includes.filewriting_collection:
    pv_prefix: $(pv_prefix)
    mri_prefix: $(mri_prefix)

The top of the file tells us what parameters should be passed, and defines a docstring for the Block. After that we instantiate the RunnableController, sim_detector_driver_block and its corresponding DetectorDriverPart, and then a stats_plugin_block with is corresponding StatsPluginPart.

The entry after this is an Include. It lets us take some commonly used Blocks and Parts and instantiate them at the level of the currently defined Block. If we look at ./malcolm/modules/ADCore/includes/filewriting_collection.yaml we’ll see how it does this:

- builtin.parameters.string:
    name: pv_prefix
    description: PV prefix for all the other plugins

- builtin.parameters.string:
    name: mri_prefix
    description: Malcolm resource id prefix for all created blocks

- builtin.parameters.string:
    name: runs_on_windows
    description: Translate directory paths if IOC runs on Windows
    default: False

- scanning.parts.DatasetTablePart:
    name: DSET

- ADCore.blocks.position_labeller_block:
    mri: $(mri_prefix):POS
    prefix: $(pv_prefix):POS

- ADCore.parts.PositionLabellerPart:
    name: POS
    mri: $(mri_prefix):POS

- ADCore.blocks.hdf_writer_block:
    mri: $(mri_prefix):HDF5
    prefix: $(pv_prefix):HDF5

- ADCore.parts.HDFWriterPart:
    name: HDF5
    mri: $(mri_prefix):HDF5
    runs_on_windows: $(runs_on_windows)
    required_version: 3.12

This will also instantiate the DatasetTablePart, position_labeller_block and it corresponding PositionLabellerPart, and then hdf_writer_block with its corresponding HDFWriterPart.

The reason we use an include file is so that other detectors can use this same filewriting collection without having to copy and paste into the top level object. There is some duplication in the parameter descriptions, but it ensures that each YAML file is a self contained description of this level downwards.

Hardware Blocks

If we look at the next level down at something like ./malcolm/modules/ADSimDetector/blocks/sim_detector_driver_block.yaml we will see our PV interface:

- builtin.parameters.string:
    name: mri
    description: Malcolm resource id of the Block

- builtin.parameters.string:
    name: prefix
    description: The root PV for the all records

- builtin.defines.docstring:
    value: |
      Hardware Block corresponding to PVs used for SimDetector detector driver

      - simDetector.template should have pv prefix $(prefix)

- builtin.controllers.StatefulController:
    mri: $(mri)
    description: $(docstring)

- ADCore.includes.adbase_parts:
    prefix: $(prefix)

- ca.parts.CADoublePart:
    name: gainX
    description: Gain in the X direction for generating image
    pv: $(prefix):GainX
    rbv_suffix: _RBV

- ca.parts.CADoublePart:
    name: gainY
    description: Gain in the Y direction for generating image
    pv: $(prefix):GainY
    rbv_suffix: _RBV

After the parameters, defines and StatefulController definition, most of our CAPart objects are instantiated in ./malcolm/modules/ADCore/includes/adbase_parts.yaml. Let’s look at the start of that file:

- builtin.parameters.string:
    name: prefix
    description: The root PV for the all records

- builtin.parameters.string:
    name: post_acquire_status
    description: The value of ADStatus when acquire returns at the end of an acquisition
    default: Idle

- builtin.parameters.string:
    name: num_images_pv_suffix
    description: The PV suffix for number of images
    default: NumImages

- ADCore.includes.ndarraybase_parts:
    prefix: $(prefix)

- ca.parts.CAChoicePart:
    name: imageMode
    description: Whether to take 1, many, or unlimited images at start
    pv: $(prefix):ImageMode
    rbv_suffix: _RBV

- ca.parts.CALongPart:
    name: numImages
    description: Number of images to take if imageMode=Multiple
    pv: $(prefix):$(num_images_pv_suffix)
    rbv_suffix: _RBV

- ca.parts.CAActionPart:
    name: start
    description: Demand for starting acquisition
    pv: $(prefix):Acquire
    status_pv: $(prefix):DetectorState_RBV
    good_status: $(post_acquire_status)

- ca.parts.CAActionPart:
    name: stop
    description: Stop acquisition
    pv: $(prefix):Acquire
    value: 0
    wait: False

This include structure mirrors that of the underlying templates, and allows us to maintain a one to one mapping of YAML file to template file. If you look at all of these CAParts you will see that they wrap up small numbers of PVs into recognisable Attributes and Methods.

For instance:

- ca.parts.CAChoicePart:
    name: imageMode
    description: Whether to take 1, many, or unlimited images at start
    pv: $(prefix):ImageMode
    rbv_suffix: _RBV

This corresponds to an Attribute that caputs to the ImageMode pv with callback when set, and uses ImageMode_RBV as the current value.

Alternatively:

- ca.parts.CAActionPart:
    name: start
    description: Demand for starting acquisition
    pv: $(prefix):Acquire
    status_pv: $(prefix):DetectorState_RBV
    good_status: $(post_acquire_status)

This corresponds to a Method that caputs to the Acquire pv with callback, and when it completes checks DetectorState_RBV to see if the detector completed successfully or with an error.

Template Designs

One of the benefits of splitting the Hardware Layer from the Device Layer is that we now get a useful interface that tells us what to load and save. We tag all writeable CAParts as config Attributes by default, which will mean that when we save() the Device Block, it will write the current value of all these Attributes of all its child Hardware Blocks to a Design file.

We learned in the Motion Tutorial that Designs are JSON formatted files stored in the config_dir on save(), and that they can be loaded by setting the design Attribute at runtime. We now introduce the concept of a Template Design. This is a read-only Design that demonstrates how a Block might be used to implement a particular use case. It always starts with the text template_.

In our demo, we want our simDetector wired up in such a way that we can implement the Acquisition Strategy set out earlier. The ADSimDetector module provides a design template_software_triggered that will do this for us. We would discover this by running up Malcolm, and seeing the possible values in the design drop-down list. If you are interested you can click below to expand the text of blocks/sim_detector_runnable_block_designs/template_software_triggered.json in ./malcolm/modules/ADSimDetector/ to see what it will load:

Template Design JSON: template_software_triggered

{
  "attributes": {
    "layout": {
      "DRV": {
        "x": -321.0, 
        "y": 4.024997711181641, 
        "visible": true
      }, 
      "STAT": {
        "x": 107.0, 
        "y": -4.024997711181641, 
        "visible": true
      }, 
      "POS": {
        "x": -107.0, 
        "y": 4.024997711181641, 
        "visible": true
      }, 
      "HDF5": {
        "x": 321.0, 
        "y": -4.024997711181641, 
        "visible": true
      }
    }, 
    "exports": {}, 
    "attributesToCapture": {
      "typeid": "malcolm:core/Table:1.0", 
      "name": [], 
      "sourceId": [], 
      "description": [], 
      "sourceType": [], 
      "dataType": [], 
      "datasetType": []
    }, 
    "label": "Ramping SimDetector",
    "exposure": 0.0, 
    "writeAllNdAttributes": true
  }, 
  "children": {
    "DRV": {
      "attributesFile": "", 
      "triggerMode": "Internal", 
      "gainX": 1.0, 
      "gainY": 1.0
    }, 
    "STAT": {
      "arrayCallbacks": true, 
      "input": "ADSIM.POS"
    }, 
    "POS": {
      "arrayCallbacks": true, 
      "attributesFile": "",
      "input": "ADSIM.CAM"
    }, 
    "HDF5": {
      "arrayCallbacks": false, 
      "attributesFile": "", 
      "input": "ADSIM.stat", 
      "cacheFramesPerChunk": 0, 
      "xmlLayout": ""
    }
  }
}

This Design will setup the plugin chain correctly for areaDetector to work the way that Malcolm expects. In particular it makes sure that the plugins are in the correct way that the HDF writer gets the tags it expects on each NDArray that it receives. There may be many template designs associated with a particular type of Block to support different use cases.

Note

areaDetector plugin chain wiring is done in the Design file rather than in each plugin Part. This means that the chain can be rewired for different scan use cases without having to change the code contained plugin Part.

Scan Block Design

Scan Blocks can have saved Design files just like Device Blocks. The difference is that they have far fewer entries as their children typically save their config in their own Design files. If we look at ./malcolm/modules/demo/blocks/scan_2det_block_designs/template_both_detectors.json we will see just how few entries there are:

{
  "attributes": {
    "layout": {
      "INTERFERENCE": {
        "x": 0.0, 
        "y": -139.60000610351562, 
        "visible": true
      }, 
      "RAMP": {
        "x": 0.0, 
        "y": 0.0, 
        "visible": true
      }, 
      "MOT": {
        "x": 0.0, 
        "y": 139.60000610351562, 
        "visible": true
      }
    }, 
    "exports": {}, 
    "simultaneousAxes": [
      "x", 
      "y"
    ], 
    "label": "Mapping x, y with Interference and Ramp detectors"
  }, 
  "children": {
    "INTERFERENCE": {
      "design": ""
    }, 
    "RAMP": {
      "design": "template_software_triggered"
    }, 
    "MOT": {
      "design": ""
    }
  }
}

There are a couple of scan Block Attributes that are saved here:

  • simultaneousAxes: The superset of axes that are allowed in axesToMove. This is used to specify the set of axes that are currently movable in a single run()

  • label: A short human readable label that identifies to a user what the scan will do if run

The reason that these are saved rather than specified in the scan Block definition or Process Definition is that multiple instances of this scan can be created with different values for these Attributes.

We imagine that each Device Block will have a number of designs for hardware or software triggering or different motor setups, and the Scan Block will say “I need DET with the hardware_trigger design and MOTORS with hkl_geometry”.

The Scan Block will not load its children’s designs at init, but will set them before every configure() call, ensuring the Device Blocks are all setup correctly at the beginning of every scan.

Now we know what we need to load, we need to work out when to load it. There is an initial_design parameter that we pass to any ManagerController or RunnableController that will tell it what design to load when Malcolm starts up, and we have two layers that are able to load an Initial Design:

  1. In the detector (Device Layer). In this case, the design will be loaded as soon as Malcolm starts, but if there is not a clear single design that all scans use then it is not clear what to set it to.

  2. In the scan (Scan Layer). As child designs are loaded on configure(), the initial_design loading will be deferred until the first time the scan is run. This means that different scan Blocks can use different initial designs for their children. For instance one scan Block could require a Detector to be in software triggered mode, and another scan Block could require the Detector to be in hardware triggered mode.

In a production system we will generally set the initial_design of our scan Blocks (case 2), but we may additionally set the initial_design of our child Blocks (case 1) if we want to ensure a particular configuration on Malcolm startup.

The detector setting at startup is not relevant to us here, so we will set the initial_design only on the scan Block.

Caution

If you set initial_design on a Block in the device_layer, including detector Blocks, then PVs will change when you restart Malcolm. This may or may not be what you want.

It is worth pointing out that we are only likely to set initial_design once a scan is working. Once a design is set, it will be restored every configure(), so a save() or unsetting the design is required to keep any manual changes to child Blocks.

Running a Scan

First you need an areaDetector IOC. From the Diamond launcher, select Utilities -> GDA AreaDetector Simulation, then click the Start IOC button.

Let’s start up the example and see it in action:

[me@mypc pymalcolm]$ pipenv run imalcolm malcolm/modules/demo/DEMO-AREADETECTOR.yaml
Loading malcolm...
Python 3.7.2 (default, Jan 20 2020, 11:03:41)
Type 'copyright', 'credits' or 'license' for more information
IPython 7.19.0 -- An enhanced Interactive Python. Type '?' for help.


Welcome to iMalcolm.

self.mri_list:
    ['mypc-ML-MOT-01:COUNTERX', 'mypc-ML-MOT-01:COUNTERY', 'mypc-ML-MOT-01', 'mypc-ML-DET-01', 'mypc-ML-DET-02:DRV', 'mypc-ML-DET-02:STAT', 'mypc-ML-DET-02:POS', 'mypc-ML-DET-02:HDF5', 'mypc-ML-DET-02', 'mypc-ML-SCAN-01', 'mypc:WEB', 'mypc:PVA']

# To create a view of an existing Block
block = self.block_view("<mri>")

# To create a proxy of a Block in another Malcolm
self.make_proxy("<client_comms_mri>", "<mri>")
block = self.block_view("<mri>")

# To view state of Blocks in a GUI
!firefox localhost:8008

In [1]:

This time we will configure from the commandline. You may have some of these lines in your history from earlier tutorials. Note you will need to replace ‘mypc’ with the name of your pc:

In [1]: from scanpointgenerator import LineGenerator, CompoundGenerator

In [2]: scan = self.block_view("mypc-ML-SCAN-01")

In [3]: yline = LineGenerator("y", "mm", -1, 0, 6)

In [4]: xline = LineGenerator("x", "mm", 4, 5, 5, alternate=True)

In [5]: generator = CompoundGenerator([yline, xline], [], [], duration=0.5)

In [6]: scan.configure(generator, "/tmp")

After configure, the detector will also report the datasets that it is about to write in the datasets Attribute:

In [7]: from annotypes import json_encode

In [8]: print(json_encode(scan.datasets.value, indent=4))
{
    "typeid": "malcolm:core/Table:1.0",
    "name": [
        "INTERFERENCE.data",
        "INTERFERENCE.sum",
        "y.value_set",
        "x.value_set",
        "RAMP.data",
        "RAMP.sum",
        "y.value_set",
        "x.value_set"
    ],
    "filename": [
        "INTERFERENCE.h5",
        "INTERFERENCE.h5",
        "INTERFERENCE.h5",
        "INTERFERENCE.h5",
        "RAMP.h5",
        "RAMP.h5",
        "RAMP.h5",
        "RAMP.h5"
    ],
    "type": [
        "primary",
        "secondary",
        "position_set",
        "position_set",
        "primary",
        "secondary",
        "position_set",
        "position_set"
    ],
    "rank": [
        4,
        4,
        1,
        1,
        4,
        4,
        1,
        1
    ],
    "path": [
        "/entry/data",
        "/entry/sum",
        "/entry/y_set",
        "/entry/x_set",
        "/entry/detector/detector",
        "/entry/sum/sum",
        "/entry/detector/y_set",
        "/entry/detector/x_set"
    ],
    "uniqueid": [
        "/entry/uid",
        "/entry/uid",
        "",
        "",
        "/entry/NDAttributes/NDArrayUniqueId",
        "/entry/NDAttributes/NDArrayUniqueId",
        "",
        ""
    ]
}

This is very similar to the Scanning Tutorial, but now datasets are reported from both detectors. Their setpoints are also reported for every scannable in every file. This is to allow a triggering scheme where a detector produces multiple frames for each scan point (explained in a future tutorial).

Now that you have the files open, you can use the h5watch command to monitor the dataset and see it grow:

[me@mypc pymalcolm]$ h5watch /tmp/INTERFERENCE.h5/entry/uid
Opened "/tmp/INTERFERENCE.h5" with sec2 driver.
Monitoring dataset /entry/uid...

You will be able to run a the same h5watch command on /tmp/RAMP.h5/entry/NDAttributes/NDArrayUniqueId to see the areaDetector dataset grow, but only when the scan has started as the HDF writer can’t write the datasets until it knows the size of the first detector frame.

You can open the web GUI again to inspect the state of the various objects, and you will see that both the RAMP and INTERFERENCE detector objects are in state Armed, as is the SCAN. You can then run a scan, either from the web GUI or the commandline. Other than h5watch, the commandline tools are not SWMR aware, so a reset is required in order to read the updated files:

In [9]: scan.run()

In [10]: scan.reset()

This will write 30 frames of data to /tmp/INTERFERENCE.h5 directly, and supervise the writing of 30 frames of data to /tmp/RAMP.h5 via areaDetector. You can take a look at the HDF5 files to see what has been written:

[me@mypc pymalcolm]$ module load hdf5/1-10-4
[me@mypc pymalcolm]$ h5dump -n /tmp/RAMP.h5
HDF5 "/tmp/RAMP.h5" {
FILE_CONTENTS {
 group      /
 group      /entry
 group      /entry/NDAttributes
 dataset    /entry/NDAttributes/ColorMode
 dataset    /entry/NDAttributes/NDArrayEpicsTSSec
 dataset    /entry/NDAttributes/NDArrayEpicsTSnSec
 dataset    /entry/NDAttributes/NDArrayTimeStamp
 dataset    /entry/NDAttributes/NDArrayUniqueId
 dataset    /entry/NDAttributes/d0
 dataset    /entry/NDAttributes/d1
 dataset    /entry/NDAttributes/timestamp
 group      /entry/detector
 dataset    /entry/detector/detector
 dataset    /entry/detector/x_set
 dataset    /entry/detector/y_set
 group      /entry/sum
 dataset    /entry/sum/sum
 dataset    /entry/sum/x_set -> /entry/detector/x_set
 dataset    /entry/sum/y_set -> /entry/detector/y_set
 }
}

This corresponds to the dataset table that the Block reported before run() was called. You can examine the uniqueid dataset to see the order that the frames were written:

[me@mypc pymalcolm]$ h5dump -d /entry/NDAttributes/NDArrayUniqueId /tmp/RAMP.h5
HDF5 "/tmp/RAMP.h5" {
DATASET "/entry/NDAttributes/NDArrayUniqueId" {
   DATATYPE  H5T_STD_I32LE
   DATASPACE  SIMPLE { ( 6, 5, 1, 1 ) / ( H5S_UNLIMITED, H5S_UNLIMITED, 1, 1 ) }
   DATA {
   (0,0,0,0): 1,
   (0,1,0,0): 2,
   (0,2,0,0): 3,
   (0,3,0,0): 4,
   (0,4,0,0): 5,
   (1,0,0,0): 10,
   (1,1,0,0): 9,
   (1,2,0,0): 8,
   (1,3,0,0): 7,
   (1,4,0,0): 6,
   (2,0,0,0): 11,
   (2,1,0,0): 12,
   (2,2,0,0): 13,
   (2,3,0,0): 14,
   (2,4,0,0): 15,
   (3,0,0,0): 20,
   (3,1,0,0): 19,
   (3,2,0,0): 18,
   (3,3,0,0): 17,
   (3,4,0,0): 16,
   (4,0,0,0): 21,
   (4,1,0,0): 22,
   (4,2,0,0): 23,
   (4,3,0,0): 24,
   (4,4,0,0): 25,
   (5,0,0,0): 30,
   (5,1,0,0): 29,
   (5,2,0,0): 28,
   (5,3,0,0): 27,
   (5,4,0,0): 26
   }
   ...

This tells us that it was written in a snake fashion, with the first row written 1-5 left-to-right, the second row 6-10 right-to-left, etc.

The detector will always increment the uniqueid number when it writes a new frame, so if you try pausing and setting the Completed Steps Attribute, you will see the uniqueID number jump where you overwrite existing frames with new frames with a greater uniqueID. The two detectors will end up with matching uniqueID datasets.

Conclusion

This tutorial has given us an understanding of how areaDetector plugin chains can be controlled in Malcolm, and how multiple detectors interface into a scan and can be paused and rewound together. The next tutorial will focus on using real hardware to perform a continuous scan.