Hello World Tutorial

If you have followed the Installation Guide using pipenv you will have a checked out a copy of pymalcolm, which includes some example code, and a virtual environment ready for running Malcolm.

So now would be a good time for a “Hello World” tutorial.

Let’s start with some terminology. A Malcolm application consists of a Process which hosts a number of Blocks. Each Block has a number of Attributes and Methods that can be used to interact with it. The Process may also contain ServerComms that allow it to expose its Blocks to the outside world, and it may also contain ClientComms that link it to another Malcolm Process and allow access to its Blocks.

Launching a Malcolm Process

So how do we launch a Malcolm process?

The simplest way is to use the imalcolm application. It will be installed on the system as imalcolm, but you can use it from your checked out copy of pymalcolm by running pipenv run imalcolm. You also need to tell imalcolm what Blocks it should instantiate and what Comms modules it should use by writing a YAML Process Definition file.

Let’s look at ./malcolm/modules/demo/DEMO-HELLO.yaml now:

# Create some Blocks
- demo.blocks.hello_block:
    mri: HELLO

- demo.blocks.hello_block:
    mri: HELLO2

- demo.blocks.counter_block:
    mri: COUNTER

# Add a webserver
- web.blocks.web_server_block:
    mri: WEB

You will see 4 entries in the file. The first 3 entries are instantiating Blocks that have already been defined. These Blocks each take a single mri (Malcolm Resource Identifier) argument which tells the Process how clients will address that Block. The last entry creates a ServerComms Block which starts an HTTP server on port 8008 and listen for websocket connections from another Malcolm process or a web GUI.

Let’s run it now:

[me@mypc pymalcolm]$ pipenv run imalcolm malcolm/modules/demo/DEMO-HELLO.yaml
Loading malcolm...
Python 3.7.2 (default, Jan 20 2020, 11:03:41)
Type 'copyright', 'credits' or 'license' for more information
IPython 7.19.0 -- An enhanced Interactive Python. Type '?' for help.


Welcome to iMalcolm.

self.mri_list:
    ['HELLO', 'HELLO2', 'COUNTER', 'WEB']

# To create a view of an existing Block
block = self.block_view("<mri>")

# To create a proxy of a Block in another Malcolm
self.make_proxy("<client_comms_mri>", "<mri>")
block = self.block_view("<mri>")

# To view state of Blocks in a GUI
!firefox localhost:8008

In [1]:

We are presented with an IPython interactive console with a Context object as self. This is a utility object that makes us a Block view so we can interact with it. Let’s try to get a view of one of the Blocks we created and call a Method on it:

In [1]: hello = self.block_view("HELLO")

In [2]: hello.greet("me")
Manufacturing greeting...
Out[2]: 'Hello me'

In [3]:

So what happened there?

Well we called a Method on a Block, which printed “Manufacturing greeting…” to stdout, then returned the promised greeting. You can also specify an optional argument “sleep” to make it sleep for a bit before returning the greeting:

In [3]: hello.greet("me again", sleep=2)
Manufacturing greeting...
Out[3]: 'Hello me again'

In [4]:

Connecting a second Malcolm Process

So how about accessing this object from outside the Process we just ran?

Well if we start a second imalcolm session we can tell it to connect to the first session, get the HELLO block from the first Process, and run a Method on it:

[me@mypc pymalcolm]$ pipenv run imalcolm -c ws://localhost:8008
Loading malcolm...
Python 3.7.2 (default, Jan 20 2020, 11:03:41)
Type 'copyright', 'credits' or 'license' for more information
IPython 7.19.0 -- An enhanced Interactive Python. Type '?' for help.


Welcome to iMalcolm.

self.mri_list:
    ['localhost:8008']

# To create a view of an existing Block
block = self.block_view("<mri>")

# To create a proxy of a Block in another Malcolm
self.make_proxy("<client_comms_mri>", "<mri>")
block = self.block_view("<mri>")

# To view state of Blocks in a GUI
!firefox localhost:8008

In [1]: self.make_proxy("localhost:8008", "HELLO")

In [2]: self.block_view("HELLO").greet("me")
Out[2]: u'Hello me'

In [3]:

So how do we know it actually worked?

Well if you look closely, you’ll see that the printed statement Manufacturing greeting... came out on the console of the first session rather than the second session (you can get your prompt back on the first session by pressing return). This means that the Block in the first session was doing the actual “work”, while the Block in the second session was just firing off a request and waiting for the response as shown in the diagram below.

digraph distributed_object_usage { bgcolor=transparent node [fontname=Arial fontsize=10 shape=box style=filled fillcolor="#8BC4E9"] graph [fontname=Arial fontsize=11 fillcolor="#DDDDDD"] edge [fontname=Arial fontsize=10 arrowhead=vee] subgraph cluster_p2 { label="Second Process" subgraph cluster_h2 { label="Hello Block" style=filled g2 [label="greet()"] e2 [label="error()"] } } subgraph cluster_p1 { label="First Process" subgraph cluster_c1 { label="Counter Block" style=filled counter "zero()" "increment()" } subgraph cluster_h1 { label="Hello Block" style=filled g1 [label="greet()"] e1 [label="error()"] } } g2 -> g1 [style=dashed label="Post\n{name:'me'}"] g1 -> g2 [style=dashed label="Return\n'Hello me'"] }

You can quit those imalcolm sessions now by pressing CTRL-D or typing exit.

Defining a Block

We have already seen that a Block is made up of Methods and Attributes, but how do we define one? Well, although Methods and Attributes make a good interface to the outside world, they aren’t the right size unit to divide our Block into re-usable chunks of code. What we actually need is something to co-ordinate our Block and provide a framework for the logic we will write, and plugins that can extend and customize this logic. The object that plays a co-ordinating role is called a Controller and each plugin is called a Part. This is how they fit together:

digraph controllers_and_parts { newrank=true; // Sensible ranking of clusters bgcolor=transparent node [fontname=Arial fontsize=10 shape=rect style=filled fillcolor="#8BC4E9"] graph [fontname=Arial fontsize=11] edge [fontname=Arial fontsize=10 arrowhead=none] subgraph cluster_control { label="Control" labelloc="b" Controller -> Parts } subgraph cluster_view { label="View" labelloc="b" Block -> Methods Block -> Attributes } {rank=same;Controller Block} Process -> Controller Controller -> Block [arrowhead=vee dir=from style=dashed label=produces] }

The Controller is responsible for making a Block View on request that we can interact with. It populates it with Methods and Attributes that it has created as well as those created by Parts attached to it. Parts are also called at specific times during Controller Methods to allow them to contribute logic.

Lets take a look at how the Hello Block of the last example is created. It is defined in the ./malcolm/modules/demo/blocks/hello_block.yaml file:

# The mri parameters should be passed when instantiating this Block. It is
# available for use in this file as $(mri)
- builtin.parameters.string:
    name: mri
    description: Malcolm resource id of the Block

# Define the docstring that appears in the docs for this Block, and put it in
# the $(docstring) variable for use in this file
- builtin.defines.docstring:
    value: Hardware Block with a greet() Method

# The Controller will create the Block for us
- builtin.controllers.BasicController:
    mri: $(mri)
    description: $(docstring)

# The Part will add a Method to the Block
- demo.parts.HelloPart:
    name: hello

The first item in the YAML file is a builtin.parameters.string. This defines a parameter that must be defined when instantiating the Block. It’s value is then available throughout the YAML file by using the $(<name>) syntax.

The second item is a BasicController that just acts as a container for Parts. It only contributes a health Attribute to the Block.

The third item is a HelloPart. It contributes the greet() and error() Methods to the Block.

Here’s a diagram showing who created those Methods and Attributes:

digraph hello_controllers_and_parts { newrank=true; // Sensible ranking of clusters bgcolor=transparent node [fontname=Arial fontsize=10 shape=rect style=filled fillcolor="#8BC4E9"] graph [fontname=Arial fontsize=11] edge [fontname=Arial fontsize=10 arrowhead=none] subgraph cluster_control { label="Control" controller [label=<BasicController<BR/>mri: 'HELLO'>] hello [label=<HelloPart<BR/>name: 'hello'>] controller -> hello } subgraph cluster_view { label="View" block [label=<Block<BR/>mri: 'HELLO'>] greet [label=<Method<BR/>name: 'greet'>] error [label=<Method<BR/>name: 'error'>] health [label=<Attribute<BR/>name: 'health'>] block -> greet block -> error block -> health } {rank=same;controller block} {rank=same;hello greet error health} controller -> health [style=dashed] hello -> greet [style=dashed] hello -> error [style=dashed] controller -> block [arrowhead=vee dir=from style=dashed label=produces] }

The outside world only sees the View side, but whenever a Method is called or an Attribute set, something on the Control side is responsible for actioning the request.

Defining a Part

We’ve seen that we don’t write any code to define a Block, we compose it from a Controller and the Parts that contribute Methods and Attributes to it. We will normally use one of the builtin Controllers, so the only place we write code is when we define a Part. Let’s take a look at our ./malcolm/modules/demo/parts/hellopart.py now:

from annotypes import Anno, add_call_types

from malcolm.core import Part, PartRegistrar
from malcolm.core import sleep as sleep_for

with Anno("The name of the person to greet"):
    AName = str
with Anno("Time to wait before returning"):
    ASleep = float
with Anno("The manufactured greeting"):
    AGreeting = str


class HelloPart(Part):
    """Defines greet and error `Method` objects on a `Block`"""

    def setup(self, registrar: PartRegistrar) -> None:
        super().setup(registrar)
        registrar.add_method_model(self.greet)
        registrar.add_method_model(self.error)

    @add_call_types
    def greet(self, name: AName, sleep: ASleep = 0) -> AGreeting:
        """Optionally sleep <sleep> seconds, then return a greeting to <name>"""
        print("Manufacturing greeting...")
        sleep_for(sleep)
        greeting = f"Hello {name}"
        return greeting

    def error(self):
        """Raise an error"""
        raise RuntimeError("You called method error()")

After the imports, you will see three with Anno() statements. Each of these defines a named type variable that can be used by the annotypes library to infer runtime types of various parameters. The first argument to Anno() gives a description that can be used for documentation, and the body of the with statement defines a single variable (starting with A by convention) that will be used to give a type to some code below. These annotypes can be imported and used between files to make sure that the description only has to be defined once.

The class we define is called HelloPart and it subclasses from Part. It implements Part.setup so that it can register two methods with the PartRegistrar object passed to it by it’s Controller.

It has a a method called greet that has a decorator on it and contains the actual business logic. In Python, decorators can be stacked many deep and can modify the function or class they are attached to. It also has a special type comment that tells some IDEs like PyCharm what type the arguments and return value are.

The decorator and type comment work together to annotate the function at runtime with a special call_types variable that Malcolm uses to validate and provide introspection information about the Method.

Inside the actual function, we print a message just so we can see what is happening, then sleep for a bit to simulate doing some work, then return the greeting.

There is also a second method called error that just raises an error. This doesn’t need a decorator as it doesn’t take any arguments or return anything (although adding one would be harmless).

Conclusion

This first tutorial has taken us through running up a Process with some Blocks and shown us how those Blocks are specified by instantiating Parts and placing them within a Controller. The HelloPart we have seen encapsulates the functionality required to add a greet() function to a Block. It means that we could now add “greeting” functionality to another Block just by adding it to the instantiated parts. In the next tutorial we will read more about adding functionality using Parts.