A screen represents the user's view at any given moment. A screen is composed of one or more layers. These layers are composited to the screen in a z-order (back to front) in order to combine them to form the desired output. Each screen has a name and by using screen transition actions a user can traverse the user interface's many screens.

Here is an example of a screen containing three layers containing controls with images and text that combine to form the final screen:

Layers are the containers for controls and groups. A screen in the user interface is generally composed of multiple layers. A single layer can be reused and shared across multiple screens to allow common content to be displayed and manipulated. If the control or group content of a layer is changed, that change will propagate to all screens where that layer is being displayed. Certain layer attributes, such as a layer's position or its visibility are maintained on a per-screen instance basis. This allows designers to hide, show and move content on a screen by screen basis without affecting other screens where the layer may be used.

Layers can receive events and respond with actions as well as contain variables that can be referenced by other model elements. These events and variables are common to all places the layer is used.

Groups are containers for controls. Groups provide a mechanism to organize content and to place common UI elements together along with variables that may be shared among the controls. Groups have a position within a layer and a size that is dictated by the groups children. All controls within the group are laid out relative to the group's origin.

Groups can receive events and respond with actions as well as contain variables that can be referenced by other model elements, in particular the child controls of the group.

A user interface is made up of many controls. A control is a rectangular area of the screen that can render content and react to events. Each control can be made up of many render extensions and react to any number of events. Controls can be shown or hidden and active/inactive as the system requires. A special type of control, called a table, is available to create row/column-based layout of information. Each column in a table can have a distinct template for rendering. This allows for efficient and easy creation of list-style information.

A render extension is the most basic piece of the user interface. It is responsible for rendering a specific type of content. Render extensions must be bound to a specific control and are not permitted to draw beyond the boundaries of the control to which they are bound. It is possible to include multiple render extensions on a control, and the order in which they are declared in the deployment bundle will define the order in which they perform their rendering.

Some render extensions included with the standard Storyboard release include:

When designing a user interface, it is often possible to achieve the same look and feel by using multiple render extensions on a control or by using multiple controls with overlapping behaviour. The general rule of thumb is that multiple render extensions on a single control should be used when there is a common event for action binding that would occur for all of the visual components being displayed.  

Events are the basic communication mechanism for triggering activity and passing data in a Storyboard application.

Events contain a name and a binary data payload. The content of the data payload is described using a format string that allows clients to generically decode the data payload. For example, the Lua action performs this decoding to allow access to the event data using string keys.

Events may be received from multiple input sources, but are processed serially by the Storyboard Engine. Clients can quickly and easily define their own custom events to enhance and drive their application, and to use these in conjunction with the standard event definitions provided by Storyboard.

A complete list of standard events can be found in the events definition section of this document.

1s0     --> Special null terminated string
1s1|1u1 --> 1 byte integer (int8_t uint8_t)
2s1|2u1 --> 2 byte integer (int16_t uint16_t)
4s1|4u1 --> 4 byte integer (int32_t uint32_t)
4f1     --> 4 byte floating point (IEEE754 float)
8s1|8u1 --> 8 byte integer (int64_t uint64_t)       

struct {
 int32_t    a;
 uint16_t   b;
};

Events trigger actions. Actions perform tasks; manipulate data, interact with the system, log messages, generate more events, etc.

Actions can be associated with any model object, but they are frequently associated with controls. When an event is received it is matched first against the actions associated with the currently focused control. After the control's actions, the processing passes to the actions associated with the visible layers on the screen, followed by the screen and application actions. This cascade of action handling provides an opportunity for the action execution to take place in the model context (application, screen, layer, control) that is most appropriate.

Actions are always invoked in response to an event to perform some sort of activity, and they are always executed in the context of the main execution thread. The trigger event might be a Storyboard standard event or a user event, and it might come from within the application, from a Lua script action, or be generated from an external program and injected using the Storyboard IO library.

More than one action might be invoked when an event is received. In this case the order in which the actions are declared within the design is the sequence that they will be invoked when an input event is matched. This action ordering can be used to ensure an order of operations execution. For example, changing the value of a data variable has to happen before executing a script that references that data variable.

The following demonstrates how the a gre.press event triggers a gra.datachange action:

For more information on the Storyboard Engine execution pipeline, refer to the Execution Pipeline section of this document.

Actions are extensible components in the Storyboard framework and are usually implemented as plugins to be included in the runtime environment. Plugins have the ability to hook into the Storyboard Engine managers and can provide a bridge between a Storyboard operation and the embedded system outside of the application.

A complete list of standard actions can be found in the Action Definitions section of this document.

Actions follow the same naming conventions as data variables, which are described in the Data Variables section of this document, with the following addition:

Valid data variables and control/layer/screen names must follow the following rules:

Layers, groups and controls can be manipulated at runtime by modifying their internal data variables. These variables are created automatically for each layer, control and group in the model and do not have to be created by users. These variables use the reserved grd_ variable namespace.

These variables are generally accessed using ${model_object:varname}, for example ${control:grd_x} to indicate the x position of the current control

While the Engine defines a number of standard user interface types of events, there is no limit to the number of new events that can be created to control custom logic in an application.  There can be multiple event input event sources concurrently generating events, however the events are queued. The delivery of the events, and potential execution of actions as a result, is serialized by the main application thread. This serialization is discussed further in the Execution Environment section of this document.

The application changes its internal state in response to events via actions bound to particular Storyboard model objects.

As an event is processed, it is matched against what events the action handlers available in a particular screen context are expecting.  When there is a match between the received event and the expected event, the action handler is invoked.  The action handler is invoked with arguments set in the deployment bundle, as well as the context of the invocation. The context includes the current screen, the current and focused controls as well as the event and its payload.

Multiple actions can be bound to a single event. These actions will be serially executed based on the order in which they were declared in the Design environment. The first declared action is the first action executed.  

Actions can be declared on controls, layers, screens and applications. The event to action matching is performed first for controls based on screen display order (where applicable) and moves sequentially up the stack to process layer, screen, and the application actions.  The execution of actions does not stop when one action handler is executed, with the exception that if a control is marked as opaque, then no additional controls will have their action handlers invoked .  If an event is directed (an event which has a position) the only layer which this control is part of will have action handlers invoked, otherwise the actions for all layers will be invoked.  Once control and layer actions have been invoked the following continues to the active screen and application context.  The following diagram illustrates the affects of control flags on action handlers for a directed event (an event which has a position):

The following diagram illustrates the flow for an event which does not have positional information. These events may be targeted at a specific control or the focused control.

When an event is received the event is delivered to the currently focused control and this control's layer, and application. This is true for all events except for directed events such as mouse/touchscreen events which contain positional data.

The focus control for a screen is determined using the focus index value assigned to a control. A focus index is an integer value which places the control in the screen focus queue. A focus index must be unique across a screen. Focus can be navigated via the focus actions which allow for setting focus and moving through the focus queue (next/prev). The focus actions are described in more detail in the Action Definitions section of this document.

The data change is a meta-stage in the execution pipeline. It is one of many potential outcomes of executing an action handler in response to an event.  However, since most actions cause a change of state, and most state information and variables are maintained in the data manager, this topic deserves special discussion.

The data manager provides clients with the ability to be notified of data content changes through data change listener callback functions. Since it is possible for multiple action handlers to be invoked as a result of a single event, these callback functions should be restricted to a bare minimum of functionality, otherwise they may introduce undesired delay to the graphical rendering operations.

Data change listener notification/processing occurs after all of the actions have been processed for a particular event, allowing data changes resulting from multiple actions to be more efficiently processed.

The display render action is also a meta-stage in the execution pipeline. As actions are executed, if any of these actions cause changes to data or state which is relevant to the rendering of content on the current screen, then the display will be refreshed with the updated content.  If there is no change in data content that would affect the current screen, then there is no display update required.

The Engine supports user defined animations using the animation action, gra.animate. This action starts executing an animation immediately, monitors the animation, and applies the specified changes as they have been defined by the user in Designer.

An animation is a named block of operations that will perform changes on Storyboard data values at a pre-determined frame rate. The individual data changes that occur within an animation are referred to as animation steps.

An animation step contains the following information:

Animation steps are all synchronized within an animation block so that their data changes will occur in a synchronized manner. While it is possible to specify arbitrary time offsets and durations, these values will be mapped onto the nearest synchronized frame slot. The frame slots are dictated by the frame rate of the animation block.

Animation instances can be labelled with a string id which is an identifier used to provide exclusive excution. It is possible for many animations to run concurrently, however if two animations have the same id value, then only the last one invoked will actually run. For example, if you have an animation to shrink and grow a control, then you only want one of either the shrink or grow operations to occur at one time. This can be achieved by having the shrink and grow animation actions share the same identifier.

Animations may be stopped at any time by using the animation stop action, gra.animate.stop.

Animations can also be created in the more traditional method of setting a timer and operating on data on every timer firing. By manipulating data in the timer callback, clients can cause any number of custom behaviours to occur, as the data changes on variables will automatically be reflected through to the user interface as would be done at any other time.

The timer callback allows non-traditional data change rates to be applied, as well as flip-book style animations where a sequence of images is pre-defined and changed on each timer iteration. This could be achieved through a series of 0 duration animations as of Storyboard 3.0.

Animations are frequently used during screen transitions. A screen transition is a way to move from the visible screen to a new screen which may or may not have common layers.  By default, screen transitions can be invoked by using one of the following actions:

All transition actions, which are time-based, take similar arguments that control the duration of the transition, the rate at which the transitions will occur, the orientation of the transition, and the number of frames that should be used. Using these arguments, the designer can control the user experience (i.e. duration and effects) as well as the overhead incurred on the system (frequency of frame updates).  

During a screen transition, four events will be generated to notify the system of the current state. These events are:

The following illustrates the sequence of events:

The transitions are written such that if graphics hardware layer support is are available, then these layers, assuming they are available for use, will be leveraged to lower the processing overhead for the system during the transition period. Experience has demonstrated that it is possible to achieve smooth transitions at almost no CPU cost when the hardware capabilities can be properly leveraged.