A render extension is the most basic piece of the user interface. It is responsible for rendering a specific type of content.  Render extensions must be bound to a specific control and are not permitted to draw beyond the boundaries of the control to which they are bound.  It is possible to include multiple render extensions on a control and the order in which they are declared in the deployment bundle will define the order in which they perform their rendering.  Examples of render extensions would be:

When designing a user interface, it is often possible to achieve the same look and feel by using multiple render extensions on a control or using multiple controls with overlapping behavior.  The general rule of thumb is that render extensions should be used when there is a common event to action binding that would occur for all of the visual components being displayed.  

A user interface is made up of many controls.  A control is a rectangular area of the screen which can render content and react to events.  Each control can be made up of many render extensions and react to any number of events.  Controls can be shown or hidden and active/inactive as the system requires.  A special type of control called a table is available in order to create a row/column based layout of information.  Each column in a table can have a distinct template for rendering.  This allows the efficient and easy creation of list style information.

Controls are contained in layers.  A user interface can have 1 or more layers which in turn are composed of 1 or more controls.  Layers are logical groupings of controls and can also react to events and execute actions.  As with controls, layers can be shown or hidden as required.  A single layer can be reused across multiple screens to allow common controls to be displayed.

A screen represents the users view at any given moment.  A screen is composed of one or more layers.  These layers are composited to the screen in a z-order (back to front) in order to combine to form the desired output.  Each screen in the system must have a unique name and by the use of actions and events a user can traverse the user interfaces many screens.

Here is an example of a screen containing three layers containing controls with images and text that combine to form the final screen:

Actions are most frequently bound to controls in a specific context, but they can also be bound to the application, screen or layer.  When an event is received it is matched first against the controls actions, then the layer actions followed by the screen and application actions.  This cascade of action handling provides an opportunity for application behavior to be driven in several different contexts, whichever is most appropriate to the situation.  

Actions always are invoked in response to an event to perform some sort of activity and are always executed in the context of the main execution thread.  The event can be a Storyboard user interface event or any other event injected into the system, potentially via the GREIO library or from a scripting interface.  

More than one action can be invoked when an event is received.  In this case the order in which the actions are declared within the deployment bundle is the sequence in which they will be invoked when a matching event is injected into the system.  This sequencing can be used to perform simple logic operations such as changing a state variable (using the data change action) and then invoking a different action whose behavior is governed by the value of that state variable (such as a timer action).

Here are some action examples tied to input events showing a data change (that could trigger changes to the screen), a Lua script callout and a screen transition:

Actions are one of the more extensible pieces of the Storyboard application framework and are often implemented using a Storyboard plugin.  This provides them with the ability to hook into internal behavior or bridge between Storyboard operations and information available from the outside system.

A complete list of standard actions can be found in the actions definition section of this document.

Actions must follow the same naming convention as data variables, described in the “Data Variables” section, with the following addition.

Events are the basic communication mechanism for passing data in the Storyboard framework.  Each event is named using a string identifier and contains a data format string as well as a data payload.  The data format definition is used to extract data from the event for abstract processing, primarily by actions and action definitions, but may also be used by other clients that are receiving Storyboard events.

When an event is transmitted to the Engine, regardless of the mechanism that is used (Scripting API, SDK API, GREIO API) once the transmission function call returns, the event data can be de-allocated.  Depending on the interface used, the event data will have been copied or the call will have blocked until the event data is no longer being referenced.

A complete list of standard events can be found in the events definition section of this document.

1s0     --> Special null terminated string
1s1|1u1 --> 1 byte integer (int8_t uint8_t)
2s1|2u1 --> 2 byte integer (int16_t uint16_t)
4s1|4u1 --> 4 byte integer (int32_t uint32_t)
8s1|8u1 --> 8 byte integer (int64_t uint64_t)        
                    

struct {
  int32_t    a;
  uint16_t    b;
};
                    

Events are named data payloads.  While the Engine defines a number of standard user interface types of events, there is no limit to the number of new events that can be created to control custom logic in an application.  There can be multiple event input event sources active concurrently, though event delivery and execution is serialized into the context of the main application thread.  This is discussed further in the discussion of the Execution Environment.

The application changes its internal state in response to events through action handlers.  These changes are achieved by mapping specific events (i.e. press, release, gesture) to actions (i.e. set data, call script, log) in the deployment bundle.  

As an event is processed, it is matched against what events the action handlers available in a particular screen context are expecting.  When there is a match between the received event and the expected event, then the action handler is invoked.  The action handler is invoked with arguments set in the deployment bundle as well as the context of the invocation.  The context includes the current screen, the current and focused controls as well as the event and its payload.

Multiple actions can be bound to a single event.  These actions will be serially executed based on the order in which they were declared; first declared is first executed.  

Actions can be declared on controls, layers, screens and applications. The event to action matching is performed first for controls based on screen display order (where applicable) and moves sequentially up the stack to process layer, screen and the application actions.  The execution of actions does not stop when one action handler is executed, with the exception that if a control is marked as opaque, then no additional controls will have their action handlers invoked .  If an event is directed (an event which has a position) the only layer which this control is part of will have action handlers invoked, otherwise the actions for all layers will be invoked.  Once control and layer actions have been invoked the following continues to the active screen and application context.  The following diagram illustrates the affects of control flags on action handlers for a directed event (an event which has a position):

The following diagram illustrates the flow for an event which does not have positional information.  These events may be targeted at a specific control or the focused control.

When an event is received the event is delivered to the currently focused control and this controls layer, and application.  This is true for all events except for directed events such as mouse/touchscreen events which contain positional data.  Focus for a screen is determined using the focus index for a control.  A focus index is an integer value which places the control in the screen focus queue.  A focus index must be unique across a screen.  Focus can be navigated via the focus actions which allow for setting focus and moving through the focus queue (next/prev).  These actions are listed in the API appendix.

The data change is a meta-stage in the execution pipeline.  It is one of many potential outcomes of executing an action handler in response to an event.  However, since most actions cause a change of state, and most state information and variables are maintained in the data manager, this topic deserves special discussion.

The data manager provides clients with the ability to be notified of data content changes through data change listener callback functions.  Since it is possible for multiple action handlers to be invoked as a result of a single event, these callback functions should be restricted to a bare minimum of functionality otherwise they may introduce undesired delay to the graphical rendering operations.

Data change listener notification/processing occurs after all of the actions have been processed for a particular event, allowing data changes resulting from multiple actions to be more efficiently processed.

The display render action is also a meta-stage in the execution pipeline.  As actions are executed, if any of these actions cause changes to data or state which is relevant to the rendering of content on the current screen, then the display will be refreshed with the updated content.  If there is no change in data content that would affect the current screen, then there is no display update required.

Valid data variables and control/layer/screen names must follow the following rules:

Controls and Layers can be manipulated at runtime by modifying their internal data variables.  These variables are created for each control and layer in the system and are namespaced as “grd_”.  The following variables are available

A table contains all of the control variables and also a set of table specific variables.  These table specific variables can be queried but not dynamically changed.  In order to change these values in a table actions are provided: gra.table.resize, gra.table.scroll.  The variables are as follows.

control:grd_rows (format = 4s1)

The number of rows in the table

control:grd_cols (format = 4s1)

The number of columns in the table

control:grd_visible_rows (format = 4s1)

The number of visible rows in the table

control:grd_visible_cols (format = 4s1)

The number of visible columns in the table

control:grd_active_row (format = 4s1)

The row index of the currently active cell

control:grd_active_col (format = 4s1)

The column index of the currently active cell

control:grd_row (format = 4s1)

The table’s current top left row

control:grd_col (format = 4s1)

The table’s current top left column

The most frequently used animation are those that occur during screen transitions.  A screen transition is a way to move from the visible screen to a new screen which may or may not have common layers.  By default screen transitions can be invoked by using one of the following actions:

All transition actions which are time based take similar arguments that control the duration of the transition, the rate at which the transitions will occur, the orientation of the transition and the number of frames that should be used.   Using these arguments, the designer can both control the user experience (i.e. duration and effects) as well as the overhead incurred on the system (frequency of frame updates).  

During a screen transition 4 events will be generated to notify the system of the current state.  These events are:

gre.screenshow.pre

This event is generated for the new screen being shown.  The event will be generated before the transition starts. This event gives the user a chance to change data via the gra.datachange action or Lua before ther transition content is updated.

gre.screenhide.pre

This event is generated for the previous screen being hidden.  The event will be generated before the transition starts.

gre.screenshow.post

This event is generated for the new screen being shown.  The event will be generated after the transition has completed

gre.screenhide.post

This event is generated for the previous screen being hidden.  The event will be generated after the transition has completed.

The following illustrates the sequence of events:

The transitions are written such that if graphics hardware layer support is are available, then these layers, assuming they are available for use, will be leveraged to lower the processing overhead for the system during the transition period.  Experience has demonstrated that it is possible to achieve smooth transitions at almost no CPU cost when the hardware capabilities can be properly leveraged.

Animation can be achieved through the use of timers and data variables.  A control can be moved by changing its x coordinate (control:grd_x) over time.

The Engine internally supports animations through the use of the animation action “gra.animate”.  This action invokes an animation context which is a list of variables to modify over time at each step of the animation.  As with other animations the rate of change of the variables can also be interpolated over the duration of the animation.