Application Design Concepts

It is beyond the scope of this document to fully describe the process by which distributed client/server systems are designed.

However, it should be apparent that the major components of such a system are the client applications, the server applications and the gateway applications. These are the parts of the system built using the Blue Application Platform, and which communicate using the Blue Application Platform.

This chapter describes some of the major stages of building a BlueAP application. It does not address the functionality of the applications, but rather describes how much of the design process is modular and how much of the process can be documented before programming begins.

Logical Processes

A system design resolves the issues of which applications are to execute where in a system, and how they are to inter-communicate.

Client applications can often easily be modularised vertically into those that provide several distinct and separate services. At a user’s workstation these applications can often be associated with a single window. These applications are often implemented as separate threads of a single executable.

In a BlueAP System they are implemented as separate logical processes, which means that the decision to run them as a single executable, or as a group of executables can be made to suit the particular system needs and be easily re-arranged to suit changing circumstances.

For example, separate executables mean that not all users have to run all the available applications, or that all logical processes can easily be run as a single executable in a test environment. Of course, running the logical processes as separate executables also makes provision for the use of workstations with multi-processors.

Server applications can also make use of the functionality of logical processes. The simplest case is to group server application together as a single executable for testing purposes; it is often easier to test some aspects of communicating processors in this way. For example, many servers (and their clients) can run on a single host.

In addition, some applications can be emulated by the use of logical processes. For example, a database or an external gateway. In this case a server application can be built that includes the logical processes for missing system components. In this way stand-alone testing can be performed whilst several system components are developed in parallel.

However, the most powerful aspect of using logical processors is the intrinsic use of the concept to design applications which are prepared for the future. For example, to cope with a growing workload or to manage an evolving functionality. A prime example of this foresightedness would be to design a Transaction System where each stage of a transactions lifecycle is managed by a separate logical process, e.g. for transaction entry, modification, update, etc. Each of these can be designed and implemented independently, and then executed either together or separately depending on the current requirements. If then, for example, a change was required in the lifecycle of a transaction only some of the logical processes would need upgrading, or a new logical process would slot into the design. This adds a type of plug-and-play feature to the design of complex systems.

Objects

A system design resolves the issue of what the ‘objects’ are in a system, what their ‘attributes’ are and what ‘actions’ can be performed with them.

The use of the Blue Application Platform cultivates an environment where the structure of the system design more closely resembles the model of the business practises it is emulating. The alternative is a system design constrained by the inflexibility of other platforms. These often suffer from the paucity of the design methodologies that their platform encapsulates.

The Blue Application Platform provides the breadth of functionality which allows systems which need to incorporate any, some or all of the following:

transaction based systems, as shown in Figure 7.
message based systems, using a request and reply methodology, as shown in Figure 9.

Lifecycles

Many object-based systems denote an object lifecycle as the progression through time of an object, as particular events occur to transform its state.

The Blue Application Platform is particularly suited to building systems which embody this lifecycle approach; they are usually typified by separating the transaction server from the application servers.

A transaction server is responsible for managing a transaction in terms of:

ensuring a transaction is never lost, and can always be recovered in the event of system failures
that changes in the state (of the lifecycle) of a transaction are appropriately logged (usually in a database)
that transactions are routed to the appropriate application servers when they achieve a certain state (in their lifecycle).

This means that a transaction server does not perform any business functionality but merely directs transactions to the application servers which provide the business functions at the appropriate time.

The Event/State methodology which underpins the Blue Application Platform is analogous to the Action/Lifecycle mode of transaction processing systems. For example, a message from an application server to the transaction server usually implies the completion of an event (say the modification of a transaction), which is used to update the lifecycle of the transaction and change its state (for example the transaction becomes ‘unlocked’ and available to other users) and is then forwarded to another application server (for example one which displays the transaction in a blotter). An example lifecycle is shown in Figure 8.

Events

Events are the core object of a BlueAP Systems application; all applications wait until an event occurs and then perform the necessary action.

All events have a unique identification type and may, in addition, be accompanied by data. For example, the commonest form of event is a message, which consists of the event descriptor and the application data. The contents and behaviour of the messages passed between applications forms the basis of a real-time system design and is often termed an ‘application programmer’s interface’, or API.

When a dialogue-based methodology is used (using the request and reply paradigm – as shown in Figure 9) all the events (messages) that are passed between applications must be defined. These include:

request messages, where a client specifies the name of the object it requires
reply messages, where a server provides the values associated with an object
control messages, where a server provides information to the client, for example, ‘bad request’, ‘object not available’ etc.
service messages, where the server provides information about the state of the service it is providing
system control messages, this are BlueAP specific messages, such as:
dialogue request
dialogue accept
dialogue close
system close
application fail

When a transaction-based methodology is used it entails the design of a two-tier BlueAP application (see Figure 7). The first level of application provides the transaction management functionality, while the next level implements the business requirements associated with the transactions. The design of these layers could incorporate the use of messaging. For example, a client application which requires a ‘deal’, so that it may be modified would probably use the request/reply paradigm. An example transaction-based system is shown in Figure 10 .

The Blue Application Platform also provides for types of events other than inter-process messages. These are:

delta timer events; these are used to generate a timer after a certain elapsed time. These are often used by applications to generate ‘time-outs’ when waiting for particular actions to be performed.
alarm events; these are used to generate a timer at a specified time of day. These are used by applications to perform appropriate actions at pre-determined times, for example the opening and closing of markets.
heartbeat events; these are used to generate a timer event at regular intervals. These are often used by applications to poll external gateway services for availability, or to produce regular statistics etc
and application events; these are events generated by an application for processing by itself.

In addition, the Blue Application Platform provides the ability to queue events as they arrive so that they may be processed at a later time. Generally, in a real-time system, events must be processed as and when they are received. Sometimes, however, it may be necessary to delay the processing; for example, during the start-up or recovery phase of an application.

States

BlueAP applications are provided with the ability to manage an overall process state, and an individual state for each dialogue.

When managing a dialogue most applications can rely of the simple state transition from dialogue closed, to dialogue open. For example, a dialogue starts in a closed state, the application makes a dialogue request to the partner process, and the dialogue goes into an open state when the dialogue accept message is received. If the partner process should subsequently fail (or send a dialogue close message) then the state would revert to closed.

Note that the designation of these states as open or closed, is determined by the application. They could just as easily be named waiting and running. The names are used with the BlueAP Tracing functionality to log the arrival of events and the state transitions that may occur as a result of them; the use of obscure mnemonics should therefore be avoided.

It should be noted that server applications designate a single event/state table for every client that requires a dialogue with it, but it is a different instance of the table (with a separate dialogue state), that is managed by the BlueAP Event Handler.

Most applications therefore rely on using the process state to more closely manage their behaviour. A common implementation uses the process state as follows:

to describe a start-up state, where the application connects to (opens a dialogue with) all its required peer applications (for example, a database server, an external gateway server), but where connections from clients are not accepted (or perhaps queued for later processing).
to describe a running state, where the application is connected to its peers, can accept connects from clients (and their subsequent requests for data), and generally provides the service it was designed for.
to describe a recovery state, where perhaps it has lost a connection with one of its peers, can only supply a partial service to its clients, and has to manage the re-connection of the lost application.

Although the simple use of just three states is sufficient for most applications, the Blue Application Platform allows for more complex process state management, which, in particular, allow for the design of more sophisticated start-up and recovery strategies.

Event/State Tables

The Event/State Tables of each of an applications logical processes, as shown in 11, completely describe the behaviour of the application.

Each application must define at least three event/state tables:

the System Event/State Table, for the processing of system messages, such as ‘close down’
the Request Event/State Table, for the processing of incoming dialogue requests,
and the Management Event/State Tables, for the control of a dialogue. An application must define one of these for each type of dialogue required. For example, a server application could define a table for the dialogue with a database, a table for a dialogue with a gateway server, and a table for the dialogue it uses for its clients;

The System and the Request Event/State Tables can only manipulate the value of the process state, while the Management Event/State Tables can manipulate both the process state and its own dialogue state.

sThe Request Event/State Table is only used to manage the arrival of dialogue request messages from other applications. When this event is processed, either by accepting or rejecting the connection, a Management Event/State Table is selected to manage any further events on this dialogue.

All entries in Event/State Tables have a common structure. They determine the following:

the event itself
the process state (or ‘any’)
a text descriptor (for tracing purposes)
the action routine to be used
the new process state if the action succeeds (or ‘no change’)
the new process state if the action fails (or ‘no change’)
the new dialogue state if the action succeeds (only for the Management Event/State Tables) (or ‘no change’)
the new dialogue state if the action fails (only for Management Event/State Tables) (or ‘no change’)

All Management Event/State Tables consist of a collection of sub-tables; one each for every dialogue state defined for this dialogue. When an event arrives on this dialogue then only the sub-table appropriate for the current state of the dialogue is used to determine the action-routine to be executed for the event.

When an event arrives each entry in the appropriate Event/State Table is examined to see if it matches; the event type must match that defined in the table and the current process state must match that defined in the table (unless ‘any process state’ is defined). The first entry in the table that matches on these criteria determine the action routine that the BlueAP Event Handler calls.

All action routines are defined with an identical API such that they always accept the event (and all its attributes – including the application data) and return a simple status – success or fail.

The BlueAP Event Handler takes the returned status of the action routine and uses the entry in the Event/State Table to determine if a new process and/or dialogue state transition is to occur. Such a state transition then automatically affects the processing of all subsequent events (if a process state change has occurred), or all subsequent events on a particular dialogue (if a dialogue state change has occurred).

Action Routines

The action routines are the C, C++ or Java functions called to perform the required actions when the application has received an event. The action routine called by the BlueAP Event Handler, is determined by the event, the current process state of the application, and perhaps the dialogue state if it is an event arriving on a particular dialogue.

All action routines share a single API which defines the input parameters (the event, with all its attributes) and its return status; either success or failure. In addition, good programming practise should ensure that every action routine is designed in such a way as to log their entry and exit point, to trace the contents of the application data part of the event, and to log the success or failure return state. The Blue Application Platform provides the necessary Tracing Library to achieve this.

Once the behaviour of an application has been defined and described in its collection of Event/State Tables then implementation can be completed by the action routines.

The action routines can call any other application function that may be required; or indeed any application library function. They can also call the functions defined in the Blue Application Platform libraries. These provided the following tools:

extensive tracing
generation of timer and application events
the use of event queues for delaying event processing
memory management
data-structure management

Messages

Because messages are the core events within any client/server distributed system, the Blue Application Platform provides many tools to aid in the design, transport and interpretation of messages.

These are:

an extensive macro library for mapping XDR (External Data Representation) data-structures so that messages can flow transparently between applications running on different hosts with different operating systems.
extensive tracing facilities that decodes the contents of messages into a readable form for debugging purposes.
an extensive library of codecs for translating messages to and from proprietary formats used by gateway servers.

Variables

The Blue Application Platform encourages a programming methodology which discourages the use of global variables in programs.

Each logical process in a BlueAP System is automatically assigned a data block of a size and structure determined by the application. This data block can be accessed by any of its action routines.

This functionality provides the following advantages:

improved memory protection between applications, and faster debugging of memory over-write errors.
code becomes re-entrant, so that duplicate logical processes (such as clients) can be executed simultaneously. This is particularly useful for the load testing of servers.
provides a standard programming practice which improves the overall readability and maintainability of a suite of server and client applications.

System Configuration

Once all the application servers have been designed and implemented a BlueAP Configuration File must be defined which describes the environment in which they will execute.

During the testing and development stages of a BlueAP Project applications can often be run in a standalone mode, or several logical processes can be joined together to form a single executable. However, during the latter stages of systems testing it becomes necessary to complete the system specification by defining the BlueAP Configuration File. It is usually necessary to define several separate systems – each with their own BlueAP Configuration File – that distinguishes the development systems (alpha and beta for example) from a production system.

The BlueAP Configuration File defines the components of a BlueAP System, and for each component the required run-time parameters. These parameters define the real-time behaviour of the BlueAP System and provide the assertions against which the BlueAP Kernel Supervisor process monitors and controls the system.

The BlueAP Configuration File defines the server hosts, the server applications (both BlueAP applications and Gateway applications), the databases, and the files of the BlueAP System.

The parameters associated with applications define the following:

any other runtime parameters which must be delivered to the application when it starts up.
the name of the application (as used by other applications)
the name of the executable to use
the primary and backup hosts where the application runs
the start-up and close-down times for the application; if these are identical the application runs as 24/7.
the transports, protocols and codecs that the application uses.