Engine Architecture Overview

The Engine is designed to be used as a fullscreen display user interface.  This can be an LCD display where the application owns the screen and manages the displayed content.  Developing an Engine based application is quite straightforward, but relies on understanding a few concepts and terms and how they relate to one another within the Engine framework.

Graphical Composition Elements

An application can be composed of multiple screens however only a single screen can be visible at a time.  Each screen is composed of layers and the screen controls how multiple layers have their content composited together to form final display output.  If possible, layers may be mapped directly to graphics hardware layers.  Layers can be referenced by multiple screens and in which case their content is shared. When an instance of a layer is used on a screen it is given a position, size and z-order.  When a screen is painted the z-order is used to determine which areas of a layer are visible to the user.

Layers contain controls.   Controls are grouping containers for renderable content.  A control is sized and positioned relative to a layer and contains the instructions on how to render various content “within” the control area.

Render Extensions

A render extension is the most basic piece of the user interface. It is responsible for rendering a specific type of content.  Render extensions must be bound to a specific control and are not permitted to draw beyond the boundaries of the control to which they are bound.  It is possible to include multiple render extensions on a control and the order in which they are declared in the deployment bundle will define the order in which they perform their rendering.  Examples of render extensions would be:

  • Render an Image (PNG, GIF, JPEG)

  • Render a text string

  • Render a filled rectangle

When designing a user interface, it is often possible to achieve the same look and feel by using multiple render extensions on a control or using multiple controls with overlapping behavior.  The general rule of thumb is that render extensions should be used when there is a common event to action binding that would occur for all of the visual components being displayed.  


A user interface is made up of many controls.  A control is a rectangular area of the screen which can render content and react to events.  Each control can be made up of many render extensions and react to any number of events.  Controls can be shown or hidden and active/inactive as the system requires.  A special type of control called a table is available in order to create a row/column based layout of information.  Each column in a table can have a distinct template for rendering.  This allows the efficient and easy creation of list style information.


Controls are contained in layers.  A user interface can have 1 or more layers which in turn are composed of 1 or more controls.  Layers are logical groupings of controls and can also react to events and execute actions.  As with controls, layers can be shown or hidden as required.  A single layer can be reused across multiple screens to allow common controls to be displayed.


A screen represents the users view at any given moment.  A screen is composed of one or more layers.  These layers are composited to the screen in a z-order (back to front) in order to combine to form the desired output.  Each screen in the system must have a unique name and by the use of actions and events a user can traverse the user interfaces many screens.

Here is an example of a screen containing three layers containing controls with images and text that combine to form the final screen:

Events and Actions


Actions are most frequently bound to controls in a specific context, but they can also be bound to the application, screen or layer.  When an event is received it is matched first against the controls actions, then the layer actions followed by the screen and application actions.  This cascade of action handling provides an opportunity for application behavior to be driven in several different contexts, whichever is most appropriate to the situation.  

Actions always are invoked in response to an event to perform some sort of activity and are always executed in the context of the main execution thread.  The event can be a Storyboard user interface event or any other event injected into the system, potentially via the GREIO library or from a scripting interface.  

More than one action can be invoked when an event is received.  In this case the order in which the actions are declared within the deployment bundle is the sequence in which they will be invoked when a matching event is injected into the system.  This sequencing can be used to perform simple logic operations such as changing a state variable (using the data change action) and then invoking a different action whose behavior is governed by the value of that state variable (such as a timer action).

Here are some action examples tied to input events showing a data change (that could trigger changes to the screen), a Lua script callout and a screen transition:

Actions are one of the more extensible pieces of the Storyboard application framework and are often implemented using a Storyboard plugin.  This provides them with the ability to hook into internal behavior or bridge between Storyboard operations and information available from the outside system.

A complete list of standard actions can be found in the actions definition section of this document.

Action Naming Conventions

Actions must follow the same naming convention as data variables, described in the “Data Variables” section, with the following addition.

  • The namespace “gra.” is reserved for Storyboard internal actions.


Events are the basic communication mechanism for passing data in the Storyboard framework.  Each event is named using a string identifier and contains a data format string as well as a data payload.  The data format definition is used to extract data from the event for abstract processing, primarily by actions and action definitions, but may also be used by other clients that are receiving Storyboard events.

When an event is transmitted to the Engine, regardless of the mechanism that is used (Scripting API, SDK API, GREIO API) once the transmission function call returns, the event data can be de-allocated.  Depending on the interface used, the event data will have been copied or the call will have blocked until the event data is no longer being referenced.

A complete list of standard events can be found in the events definition section of this document.

Event Naming Conventions

New events can readily be defined and are not required to contain a data payload, in which case their format string and data payload will be NULL values.  When creating new events, it is appropriate to namespace the event definitions so that the names of events do not collide.  For example the Storyboard framework reserves the name prefix of ‘gre’. for user interface events and the timer functions all generate events that are prefixed with timer.

The use of events is closely coupled with the declaration and operation of actions.  An action can only be invoked when an event matching the action definition is received.  This results in a common design pattern where an action will perform sophisticated logic in an external script or program and then signal a completion action to run once the script work is complete.

Event >Action (script) >Work >Trigger Event >Action (completion)

Event Format String

The format string on the event is used to assist with the decoding of the event data.  This decoding is used in the action bindings, to allow a minimal amount of additional logic to be placed into the event matching code and also by certain advanced actions, such as Lua scripts, that can symbolically access the event data in the context of the script.

The format string is a description of the data structure passed in the event, it uses entries that are formatted with [numbytes][signed/unsigned][numelements][ ][name]. For the standard C data types the number formatting would look like:

1s0     --> Special null terminated string
1s1|1u1 --> 1 byte integer (int8_t uint8_t)
2s1|2u1 --> 2 byte integer (int16_t uint16_t)
4s1|4u1 --> 4 byte integer (int32_t uint32_t)
8s1|8u1 --> 8 byte integer (int64_t uint64_t)        

So if you were transmitting a structure such as:

struct {
  int32_t    a;
  uint16_t    b;

You would create a format string containing “4s1 a 2u1 b”.  The ‘a’ and ‘b’ fields are optional, but they are used to provide the symbolic description of the data to other applications that may want to access it.  This is particularly important if the application incorporates scripting to manage additional complex operations.

Maintaining State and Reacting to Changes

In addition to providing a framework for defining the visual display of an application, the Engine framework contains a data manager that is responsible for maintaining the state information that controls the behavior of the application, such as which screens to display and what content should be rendered inside of a control.   The Data Manager contains variables that are user defined (string keys) and typed (string, integer etc) and can be readily modified.

Most often the Data Manager variables are modified by the application itself as it responds to events in the system.   Events are asynchronous triggers, originating internally or externally to the Engine, that are in turn mapped to actions that perform work in the system.  

Actions are where all of the real ‘execution’ is performed within an Engine.  Actions are managed as extensions to the Engine and are flexible.  Some of the built-in actions include changing content in the data manager, manipulating timers, logging messages or triggering additional events both inside and outside of the Storyboard framework.

Execution Pipeline

Below is the execution pipeline:

Trigger Event

Events are named data payloads.  While the Engine defines a number of standard user interface types of events, there is no limit to the number of new events that can be created to control custom logic in an application.  There can be multiple event input event sources active concurrently, though event delivery and execution is serialized into the context of the main application thread.  This is discussed further in the discussion of the Execution Environment.

Action Execution

The application changes its internal state in response to events through action handlers.  These changes are achieved by mapping specific events (i.e. press, release, gesture) to actions (i.e. set data, call script, log) in the deployment bundle.  

As an event is processed, it is matched against what events the action handlers available in a particular screen context are expecting.  When there is a match between the received event and the expected event, then the action handler is invoked.  The action handler is invoked with arguments set in the deployment bundle as well as the context of the invocation.  The context includes the current screen, the current and focused controls as well as the event and its payload.

Multiple actions can be bound to a single event.  These actions will be serially executed based on the order in which they were declared; first declared is first executed.  

Actions can be declared on controls, layers, screens and applications. The event to action matching is performed first for controls based on screen display order (where applicable) and moves sequentially up the stack to process layer, screen and the application actions.  The execution of actions does not stop when one action handler is executed, with the exception that if a control is marked as opaque, then no additional controls will have their action handlers invoked .  If an event is directed (an event which has a position) the only layer which this control is part of will have action handlers invoked, otherwise the actions for all layers will be invoked.  Once control and layer actions have been invoked the following continues to the active screen and application context.  The following diagram illustrates the affects of control flags on action handlers for a directed event (an event which has a position):

The following diagram illustrates the flow for an event which does not have positional information.  These events may be targeted at a specific control or the focused control.


When an event is received the event is delivered to the currently focused control and this controls layer, and application.  This is true for all events except for directed events such as mouse/touchscreen events which contain positional data.  Focus for a screen is determined using the focus index for a control.  A focus index is an integer value which places the control in the screen focus queue.  A focus index must be unique across a screen.  Focus can be navigated via the focus actions which allow for setting focus and moving through the focus queue (next/prev).  These actions are listed in the API appendix.

Data Change

The data change is a meta-stage in the execution pipeline.  It is one of many potential outcomes of executing an action handler in response to an event.  However, since most actions cause a change of state, and most state information and variables are maintained in the data manager, this topic deserves special discussion.

The data manager provides clients with the ability to be notified of data content changes through data change listener callback functions.  Since it is possible for multiple action handlers to be invoked as a result of a single event, these callback functions should be restricted to a bare minimum of functionality otherwise they may introduce undesired delay to the graphical rendering operations.

Data change listener notification/processing occurs after all of the actions have been processed for a particular event, allowing data changes resulting from multiple actions to be more efficiently processed.

Display Render

The display render action is also a meta-stage in the execution pipeline.  As actions are executed, if any of these actions cause changes to data or state which is relevant to the rendering of content on the current screen, then the display will be refreshed with the updated content.  If there is no change in data content that would affect the current screen, then there is no display update required.

Execution Environment

A Storyboard application may be multi-threaded, however the execution pipeline is single threaded.  This serialization is provided by the servicing of the event queue as each event is processed through the execution pipeline in sequence.

Internally, within the Engine framework, multiple threads are used to simplify control logic for things such as supporting multiple input sources, or timed event activities.  

The threads that are run as part of the STORYBOARD framework are generally not signal handling, and mask off all signals.  In certain situations there may be a requirement to handle specific signals, but in those situations the signal handling behavior will be documented as part of the component API.

Data Variables

All user accessible data in the Storyboard is maintained by the data manager database.  This data manager provides centralized notification of data changes to various interested parties within the Storyboard framework.  

The data manager stores data as key/value pairs where the key is a string identifier and the value is an arbitrary blob of data and its format string.  The data manager can be used to store state information for the application and that information can be bound to certain contexts through the use of variables. These variables can then be used within the deployment bundle to facilitate general design and development.

The format of a data manager variable within the deployment bundle is:


For example to refer to the application data heading which contains a global font string that many controls should use, you could use the ${app:heading} variable.  

This example seeds the initial value of the application variable heading with the string fonts/courier.ttf and then later uses that value to control the font that is used in a control’s text render extension.  The render extension also references the data manager user variable text that could be modified by other actions and scripts and set to contain the text that will be displayed by the control.

The data manager variables that are currently available are:


Refers to the data associated with the application keyname


Refers to the data associated with the screen keyname


Refers to the data associated with the layer keyname


Refers to the data associated with the control keyname


Refers to the data associated with a general keyname

Naming conventions

Valid data variables and control/layer/screen names must follow the following rules:

  • A valid name matches the following [a-zA-Z]+[a-zA-Z0-9_]*

    • Starts with a character a-z/A-Z not a digit or special character

    • Only '_' is supported as a special character, no spaces in names

  • Screen, Layer, Controls must all be uniquely named

  • User variables cannot collide with a Screen/Layer/Control name

  • Model names and user variables cannot start with [grd]

  • The period (.) character is reserved as a namespace separator

Control and Layer data variables

Controls and Layers can be manipulated at runtime by modifying their internal data variables.  These variables are created for each control and layer in the system and are namespaced as “grd_”.  The following variables are available

Control variables

The following values can be queried and changed through the normal data management channels.   The position variables are relative to the layer on which the control is rendered.

control:grd_x (format = 4s1)

The controls x position relative to its layer

control:grd_y (format = 4s1)

The controls y position relative to its layer

control:grd_width (format = 4s1)

The controls width

control:grd_height (format = 4s1)

The controls height

control:grd_hidden (format  = 1u1)

A value of 0 states that the control is shown and 1 for a hidden control

control:grd_active (format  = 1u1)

A value of 0 states that the control is active (can receive and react to events) and 1 for an inactive control (cannot receive of react to events)

control:grd_opaque (format  = 1u1)

Indicates if the control is opaque to events, if opaque (1) then the control will block events from being handled by other controls.  If the value is 0 then the events flow through the control to ones behind it.

Layer variables

The following values can be queried and changed through the normal data management channels.   The position variables are relative to the screen.

layer:grd_x (format = 4s1)

The controls x position relative to the screen

layer:grd_y (format = 4s1)

The controls y position relative to the screen

layer:grd_alpha (format = 1u1)

The layers opacity value.  The values range from 255 (opaque) to 0 (transparent)

layer:grd_hidden ( format  = 1u1)

A value of 0 states that the layer and all of its controls are shown and 1 to hide the layer and all of its controls

Table variables

A table contains all of the control variables and also a set of table specific variables.  These table specific variables can be queried but not dynamically changed.  In order to change these values in a table actions are provided: gra.table.resize, gra.table.scroll.  The variables are as follows.

control:grd_rows (format = 4s1)

The number of rows in the table

control:grd_cols (format = 4s1)

The number of columns in the table

control:grd_visible_rows (format = 4s1)

The number of visible rows in the table

control:grd_visible_cols (format = 4s1)

The number of visible columns in the table

control:grd_active_row (format = 4s1)

The row index of the currently active cell

control:grd_active_col (format = 4s1)

The column index of the currently active cell

control:grd_row (format = 4s1)

The table’s current top left row

control:grd_col (format = 4s1)

The table’s current top left column


Screen Transitions

The most frequently used animation are those that occur during screen transitions.  A screen transition is a way to move from the visible screen to a new screen which may or may not have common layers.  By default screen transitions can be invoked by using one of the following actions:

  • gra.screen

    • Transition to a new screen immediately

  • gra.screen.fade

    • Fade the new screen into the current screen over time

  • gra.screen.path

    • Slide the new screen in and the old screen out over time, this happens from one of the following directions

      • Left

      • Right

      • Top

      • Bottom

  • gra.screen.scale

    • Grow the new screen over the current screen

All transition actions which are time based take similar arguments that control the duration of the transition, the rate at which the transitions will occur, the orientation of the transition and the number of frames that should be used.   Using these arguments, the designer can both control the user experience (i.e. duration and effects) as well as the overhead incurred on the system (frequency of frame updates).  

During a screen transition 4 events will be generated to notify the system of the current state.  These events are:


This event is generated for the new screen being shown.  The event will be generated before the transition starts. This event gives the user a chance to change data via the gra.datachange action or Lua before ther transition content is updated.


This event is generated for the previous screen being hidden.  The event will be generated before the transition starts.


This event is generated for the new screen being shown.  The event will be generated after the transition has completed


This event is generated for the previous screen being hidden.  The event will be generated after the transition has completed.

The following illustrates the sequence of events:

The transitions are written such that if graphics hardware layer support is are available, then these layers, assuming they are available for use, will be leveraged to lower the processing overhead for the system during the transition period.  Experience has demonstrated that it is possible to achieve smooth transitions at almost no CPU cost when the hardware capabilities can be properly leveraged.


Animation can be achieved through the use of timers and data variables.  A control can be moved by changing its x coordinate (control:grd_x) over time.

Animation action

The Engine internally supports animations through the use of the animation action “gra.animate”.  This action invokes an animation context which is a list of variables to modify over time at each step of the animation.  As with other animations the rate of change of the variables can also be interpolated over the duration of the animation.


The basic Engine operation behavior based on events and actions is relatively limited with respect to advanced logic and computations that can be performed.  In order to have more advanced functionality, a script plug-in such as lua can be used.

There is no limit to which scripting languages can be provided as action handlers, but the default scripting language provided with the Storyboard framework is Lua.  Lua(www.lua.org) was selected for its simplicity, footprint and high performance and is ideal for use in embedded systems where resources are at a premium.

To find out more about lua scripting please see the Scripting with Lua section.

Performance Considerations

All actions are executed within the context of an event delivery and as such their execution will have an impact on the overall throughput and responsiveness of the system.  In particular with Lua scripts, it is important to limit the length of time that functions take to perform their work or to separate lengthy operations into separate tasks, threads or processes depending on the operating environment being used.

The screen manager listens for data changes and checks the state of controls to determine when the display needs to be refreshed.  If the data for controls is changing rapidly this may cause thrashing of the display and possible flicker if not using double buffering.   When changing data values, moving controls or generating events which would cause the display to be updated it is advisable to hold the screen manager updates until all changes have been made.  Once modifications are complete the screen manager can be released and the display updated is needed.  The actions are as follows:

  1. gra.screen.hold

  2. gra.screen.release

External Communication (Storyboard IO)

Communication with external processes is accomplished thought the use of the Storyboard IO plugin and API (known as GREIO).  When the GREIO plugin is loaded a channel is created in order for processes to inject events into the system.  A single event queue is used to serialize the events and therefore any events sent via GREIO will be placed in the queue with standard Storyboard system events.  If the external application wishes to receive events it can create its own GREIO channel which can have events sent through.  Applications can have multiple receive channels and the Engine has a single input channel.  The following diagram illustrates an application which can send events to the Engine and review events on a named channel.

More in-depth information can found in the Storyboard IO API section of this document.