A screen represents the user's view at any given moment.  A screen is composed of one or more layers.  These layers are composited to the screen in a z-order (back to front) in order to combine to form the desired output.  Each screen in the system must have a unique name and by the use of actions and events a user can traverse the user interfaces many screens.

Here is an example of a screen containing three layers containing controls with images and text that combine to form the final screen:

Controls are contained in layers.  A user interface can have 1 or more layers which in turn are composed of 1 or more controls.  Layers are logical groupings of controls and can also react to events and execute actions.  As with controls, layers can be shown or hidden as required.  A single layer can be reused across multiple screens to allow common controls to be displayed.

A user interface is made up of many controls.  A control is a rectangular area of the screen which can render content and react to events.  Each control can be made up of many render extensions and react to any number of events.  Controls can be shown or hidden and active/inactive as the system requires.  A special type of control called a table is available in order to create a row/column based layout of information.  Each column in a table can have a distinct template for rendering.  This allows the efficient and easy creation of list style information.

A render extension is the most basic piece of the user interface. It is responsible for rendering a specific type of content.  Render extensions must be bound to a specific control and are not permitted to draw beyond the boundaries of the control to which they are bound.  It is possible to include multiple render extensions on a control and the order in which they are declared in the deployment bundle will define the order in which they perform their rendering.

Some render extensions included with the standard Storyboard release include:

When designing a user interface, it is often possible to achieve the same look and feel by using multiple render extensions on a control or by using multiple controls with overlapping behavior.  The general rule of thumb is that multiple render extensions on a single control should be used when there is a common event to action binding that would occur for all of the visual components being displayed.  

Events are the basic communication mechanism for triggering activity and passing data in a Storyboard application.

Events contain a name and a binary data payload. The content of the data payload is described using a format string that allows clients to generically decode the data payload. For example the Lua action performs this decoding to allow access to the event data using string keys.

Events may be received from multiple input sources, but are processed serially by the Storyboard Engine. Clients can quickly and easily define their own custom events to enhance to drive their application and use these in conjunction with the standard events event definitions provided by Storyboard.

A complete list of standard events can be found in the events definition section of this document.

1s0     --> Special null terminated string
1s1|1u1 --> 1 byte integer (int8_t uint8_t)
2s1|2u1 --> 2 byte integer (int16_t uint16_t)
4s1|4u1 --> 4 byte integer (int32_t uint32_t)
8s1|8u1 --> 8 byte integer (int64_t uint64_t)        
                

struct {
int32_t    a;
uint16_t   b;
};
                

Events trigger actions. Actions do something; manipulate data, interact with the system, log messages, generate more events etc.

Actions can be associated with any model object but they are frequently associated with controls.  When an event is received it is matched first against the action's associated with the currently focused control. After the control's actions the processing passes to the actions associated with the visible layers on the screen followed by the screen and application actions. This cascade of action handling provides an opportunity for the action execution to take place in model context (application, screen, layer, control) that is most appropriate.

Actions always are invoked in response to an event to perform some sort of activity and are always executed in the context of the main execution thread. The trigger event may be a Storyboard standard event or a user event and may come from within the application, from a Lua script action, or be generated from an external program and injected using the Storyboard IO library.

More than one action may be invoked when an event is received.  In this case the order in which the actions are declared within the design is the sequence that they will be invoked when an input event is matched.  This action ordering can be used to ensure an order of operations execution, for example changing the value of a data variable before executing a script that references that data variable.

The following demonstrates how the a gre.press event triggers a gra.datachange action:

For more information on the Storyboard Engine execution pipeline, refer to the Execution Pipeline section of this document.

Actions are extensible components in the Storyboard framework and are usually implemented as plugins to be included in the runtime environment.  Plugins have the ability to hook into the Storyboard Engine managers and can provide a bridge between a Storyboard operation and the embedded system outside of the application.

A complete list of standard actions can be found in the Action Definitions section of this document.

Actions follow the same naming conventions as data variables, described in the Data Variables section of this document, with the following addition:

Valid data variables and control/layer/screen names must follow the following rules:

Controls and Layers can be manipulated at runtime by modifying their internal data variables.  These variables are created for each control and layer in the application use the reserved grd_ variable namespace.

These variables are generally accessed using ${model_object:varname}, for example ${control:grd_x} to indicate the x position of the current control

While the Engine defines a number of standard user interface types of events, there is no limit to the number of new events that can be created to control custom logic in an application.  There can be multiple event input event sources concurrently generating events, however the events are queued and the delivery of the events, and potential execution of actions as a result, is serialized by the main application thread. This serialization is discussed further in the Execution Environment section of this document.

The application changes its internal state in response to events via actions bound to particular Storyboard model objects.

As an event is processed, it is matched against what events the action handlers available in a particular screen context are expecting.  When there is a match between the received event and the expected event, then the action handler is invoked.  The action handler is invoked with arguments set in the deployment bundle as well as the context of the invocation.  The context includes the current screen, the current and focused controls as well as the event and its payload.

Multiple actions can be bound to a single event.  These actions will be serially executed based on the order in which they were declared in the Design environment. The first declared action is the first action executed.  

Actions can be declared on controls, layers, screens and applications. The event to action matching is performed first for controls based on screen display order (where applicable) and moves sequentially up the stack to process layer, screen and the application actions.  The execution of actions does not stop when one action handler is executed, with the exception that if a control is marked as opaque, then no additional controls will have their action handlers invoked .  If an event is directed (an event which has a position) the only layer which this control is part of will have action handlers invoked, otherwise the actions for all layers will be invoked.  Once control and layer actions have been invoked the following continues to the active screen and application context.  The following diagram illustrates the affects of control flags on action handlers for a directed event (an event which has a position):

The following diagram illustrates the flow for an event which does not have positional information.  These events may be targeted at a specific control or the focused control.

When an event is received the event is delivered to the currently focused control and this controls layer, and application.  This is true for all events except for directed events such as mouse/touchscreen events which contain positional data.

The focus control for a screen is determined using the focus index value assigned to a control.  A focus index is an integer value which places the control in the screen focus queue.  A focus index must be unique across a screen.  Focus can be navigated via the focus actions which allow for setting focus and moving through the focus queue (next/prev).  The focus actions are described in more detail in the Action Definitions section of this document.

The data change is a meta-stage in the execution pipeline.  It is one of many potential outcomes of executing an action handler in response to an event.  However, since most actions cause a change of state, and most state information and variables are maintained in the data manager, this topic deserves special discussion.

The data manager provides clients with the ability to be notified of data content changes through data change listener callback functions.  Since it is possible for multiple action handlers to be invoked as a result of a single event, these callback functions should be restricted to a bare minimum of functionality otherwise they may introduce undesired delay to the graphical rendering operations.

Data change listener notification/processing occurs after all of the actions have been processed for a particular event, allowing data changes resulting from multiple actions to be more efficiently processed.

The display render action is also a meta-stage in the execution pipeline.  As actions are executed, if any of these actions cause changes to data or state which is relevant to the rendering of content on the current screen, then the display will be refreshed with the updated content.  If there is no change in data content that would affect the current screen, then there is no display update required.

The Engine supports user defined animations using the animation action, gra.animate.   This action will start the executing of an animation immediately and will monitor the animation and apply the specified changes as they have been defined by the user in the Designer tool.

An animation is a named block of operations that will perform changes on Storyboard data values at a pre-determined frame rate. The individual data changes that occur within an animation are refered to as animation steps.

An animation step contains the following information:

Animation steps are all synchronized within an animation block so that their data changes will occur in a synchronized manner. While it is possible to specify arbitrary time offsets and durations, these values will be mapped onto the nearest synchronized frame slot. The frame slots are dictated by the frame rate of the animation block.

Animation instances can be labelled with a string id which is an identifier used to provide exclusive excution. It is possible for many animations to run concurrently, however if two animations have the same id value, then only the last one invoked will actually run. For example if you had an animation to shrink and grow a control, then you only want one of either the shrink or grow operations to occur at one time and this can be achieved by having the shrink and grow animation actions share the same identifier.

Animations may be stopped at any time by using the animation stop action, gra.animate.stop.

Animations can also be created in the more traditional method of setting a timer and operating on data on every timer firing. By manipulating any data in the timer callback, clients can cause any number of custom behaviours to occur as the data changes on variables will automatically be reflect through to the user interface as would be done at any other time.

The timer callback allows non-traditional data change rates to be applied as well as flip-book style animations where a sequence of images is pre-defined and changed on each timer iteration, although this could be achieved through a series of 0 duration animations as of Storyboard 3.0.

Animations are frequently used during screen transitions.  A screen transition is a way to move from the visible screen to a new screen which may or may not have common layers.  By default screen transitions can be invoked by using one of the following actions:

All transition actions which are time based take similar arguments that control the duration of the transition, the rate at which the transitions will occur, the orientation of the transition and the number of frames that should be used.   Using these arguments, the designer can both control the user experience (i.e. duration and effects) as well as the overhead incurred on the system (frequency of frame updates).  

During a screen transition 4 events will be generated to notify the system of the current state.  These events are:

The following illustrates the sequence of events:

The transitions are written such that if graphics hardware layer support is are available, then these layers, assuming they are available for use, will be leveraged to lower the processing overhead for the system during the transition period.  Experience has demonstrated that it is possible to achieve smooth transitions at almost no CPU cost when the hardware capabilities can be properly leveraged.