• The MicroGuide to Process and Decision Modeling in BPMN/DMN: Building More Effective Processes by Integrating Process Modeling with Decision Modeling
    The MicroGuide to Process and Decision Modeling in BPMN/DMN: Building More Effective Processes by Integrating Process Modeling with Decision Modeling

Build Survivable, Hardened Cloud Processes with BPMN

Integrating the cloud into your error processing strategy can be challenging. Fortunately, BPMN provides a powerful approach to managing failures, in the form of exceptions and compensations. Adding resources and services into the ‘Cloud’ adds error points that must be managed. The designer of these processes should strive to make their processes ‘survivable’ and hardened. The exception shapes and compensations allow continuous operation throughout intermittent error conditions such as those encounter in integrating cloud services and resources. Without a hardened process your participants might need to use a manual process for the exception flow. This can be an outcome of overfocus on “happy day” paths. Manual processes promote staff improvisations and undocumented “work arounds” and obviously should be avoided.

Figure 1 presents an example of BPMN’s modeling of survivable processes in a survey pattern. If an error occurs in the email-reading process, it can be posted to a log and systems administrators can respond. The survey can continue. Finally, the process can loop back or complete.

Figure 1:  BPMN for an exception in the survey process.

The aim of BPMN is to support “long-running” transactions so it is especially suited to the cloud scenario. For instance, a process-oriented car rental agency might create one process for each vehicle, each using a cloud-based machine health service. In this case, the process could run for years, with thousands of calls to the cloud-based service.

Three Error Types

There are three types of exceptions that may affect process execution:

  • Technical exceptions—the process server has failed
  • Transient exceptions—a needed resource (for instance a cloud service) for the process is unavailable
  • Business exceptions—the condition of the process is in error. Data is incomplete or has errors.

Unfortunately, there is no cure or notation for the technical exception. When the server halts, recovering processes are dependent on the technical nature of the process server.

Decisions, supported by business rules, can create named business exceptions. If the process is transactional, it should compensate for transactions that fail to complete. In compensation, the process cleans up or backs out records from ERP’s, operational data stores, or data warehouses – perhaps these can be cloud based.

Processes often must be completed within a specific time period, so there could be time-out actions noted on the diagram. Business processes usually receive a message and translate that message into yet another message for consumption by the next part of the process. BPMN is powerful for developing time-outs, exceptions, and compensations. In building a basic process in BPMN, you can attach time-outs, exceptions, and compensations directly to a subprocess task list. When time-outs and exceptions occur within the process tasks, the process server traps these at the point of the subprocess.

Figure 2 presents a process server “architecture”, including a cloud services, for understanding the three types of errors. If the process server fails, then it obviously cannot run and a technical error occurs. If the ‘on-premise’ database service fails, then the process cannot get the data it needs to complete. This is a transient error. You should be able to restart the process instance and complete it once the database is restored. However cloud services are more complex, you might not have the same level of control that you would have for the database.

Users send data for the process in application servers. If users send incorrect data, then a business exception has occurred. Finally, you use process steps to resolve the error. If the users have used a cloud-bases SAS system to create a request or order, then more complexity is added.


Figure 2:  Architecture for a process environment in a cloud-based ecosystem.

You can model business processes with advanced abilities that suspend processing, or communicate with disconnected, cloud-based processes. They can add new services and capabilities such as payments processing. These powers yield opportunities and fulfill the need for the enterprise to reduce the cost of their most complicated practices. Yet, to develop these cloud-based capabilities, you should understand how BPM tools use the three types of exceptions and what you must plan for when developing the production version of the process diagram.

Cloud-based- Error Handling Example

A process diagram should use exception handlers to redirect the process flow when a cloud based exception happens. So, if the subprocess in the figure below raises an error, then steps can be taken to accommodate the error.

Figure 3: using subprocess for exceptional flow

In BMPN, handling errors arising from a subprocess is known as ‘Exceptional flow’ and, for errors, this is always interrupting: the subprocess that generated the error will terminate, and the alternate “exceptional flow” is followed instead. Later, the process can merge exceptional flow with the main flow. The exceptional flow merge does not need to immediately return to the subprocess that generated the error. If required, a process model can merge several steps downstream.

Finally, cloud process integration diagrams in BPMN should include error handling, compensation, and transactions. All services have the potential for error conditions, understanding how to use BPMN to automate error handling and recover, or be survivable, is critical. 


A BPMN Pattern that Avoids Complex Merges

Arun Pareek In a blog post (here), described a solution to a process problem where three parallel tasks are executing. When one of the tasks completes with a particular condition, he uses a complex gateway merge to halt the execution of the other tasks. He called this ‘dead path’ cleanup. The original paper on dead-path elimination is here.

For you convenience, I will show Arun’s bpmn use case here:


The use case is for a security check that verifies Citizenship, Credit and Criminal checks. The topic of the dead path was the termination of one of the tasks Citizenship or Credit, when the criminal check had passed and one of the two activities, in pink, had completed.

In the Microguide to Process Modeling in BPMN, Rick and I do not support the use of the complex merge shape because they are confusing, and hide the logic that we are trying to build with our process modeling. All the logic from a complex merge shape can be defined in a subprocess. This example is a perfect demonstration of this. The fact that the ‘criminal background check’ result will cancel the citizenship and credit is hidden in Arun’s solution. Also hidden in the solution is the fact that the 'dead-path' elimination is an Oracle specific extension.

The recommended solution, with thanks for the help to Gilles Rabaud and Thomas Allweyer, shown here in an inubit (link) process modeler:



This BPMN:

  • Places the criminal check in a subprocess and the other activities in an enclosing subprocess
  • Starts these in parallel
  • Raises an exception to the border of the outer subprocess if there is a failure in the criminal check
  • For the credit and citizen check, the first activity to return a valid check escallates the inner subproces


 escallation and errors manage the threads in the same manner. The BPMN 2.0 spec explicitly provides for thread termination with the error event plus (conditionally) the escalation event.  

In my original post on this topic, several reviewers noted that I did not cover the exact use case described by Arun's post. The point of the complex merge was to cover the case where the Criminal Check passed and either the Citizenship or the Credit check passed (activities in Pink, here). In this case, you can place the Citizen Check and the Credit Check in a subprocess and when one passes then use the raise escalate the edge of the subprocess.

This shows the power of the escalate shape and makes it very clear what is happening.

While this approach requires more canvas, it is clearer what is happening and why. This said, I understand that Arun was merely demonstrating how to use a complex merge in the Oracle Tool.


Do you have a process for business process management?

During business process modeling, a BPM technical/management team creates or tunes a business process model that supports the project’s objectives.  What are the mental processes of the team members modeling and developing business processes and connecting business events and rules? They use workshops and communication. They interview the best-performing and the worst-performing actors in the processes. They review financial reports, documentation and the notes of the managers of the current process. They identify triggering events, business rules and the process needs for policies. They design the proper flow control. They create pictorial representations.

There are institutional or traditional ways of gathering information about a process. However, few organizations know when it is time to add process thinking and business process management to formalize activities—there is no process to create new topics.

Legacy efforts create reams of paper, cabinets full of files, and databases overflowing with useful information—from management directives to marketing papers to MIS memorandums. Much important information exists here. Yet, business process modeling is different from knowledge management, quality reviews or even data modeling. The purpose of data modeling is to develop a model of what is. The purpose of business process modeling is to develop a model of what should be. Yet, how does an organization decide it is time to gather ad-hoc and undocumented activities and add the structure of business process management?

It is self-apparent to look at a factory production line and know that there is a process present; how do we discover the need for a process? Symptoms of missing process needs include:

  • Poor business or financial performance in a rapid, surprised way
  • Low morale and high employee turnover
  • Management overwhelm, (many decisions, little data)
  • Poor communications

There are other symptoms; however, deciding and clarifying when to create a process can liberate these conditions and improve business results. A well designed process has these characteristics

  • Clear goals
  • Well defined roles
  • Improved clarity
  • Increased visibility

We have identified a series of steps to deciding and delineating a ‘green fields’ or occult process in the enterprise business process framework.


Does Your Modeling Capture the 'Swiveling Chair?'

The OMG’s Business Maturity Model (BMM) (link) suggests that organizations should rely on a system of strategic business initiatives from senior management whose implementation is delegated to subordinates (roles).  The delegation acts through processes and business rules and responds to events. However, when a bottom-up process discovery is conducted, the actual process varies widely from the way management has proscribed business operations to function.  Sometimes this is because of legacy process and antiquated technology; sometimes it is due to habit.  Sometimes the need for efficiency created the misaligned processes, and employees, under pressure use creative on-the-spot (ad-hoc, undocumented) solutions. 

Modeling Run away, Manual and Ad Hoc Processes

We often encounter organizations that spend months (or years) analyzing the daily, manual work of individuals.  At the work unit, they might find workers that do not know the business’s basic strategy; let alone the rules.  For example, data entry personnel might copy data into a green screen on another system from a flat file or printout. Often they do not know why.  Frequently there was no solution available to solve the problem.  Until it becomes documented, standard practice, management might not know this. As the business grows and more technology is added, what were 2 persons become 10.  Eventually, with 20 similar processes, 200 people are doing uniformed, manual labor. 

For instance look at the process fragment  below:

Figure 1: A Consumer Credit Check

Process modelers often mistakenly identify the ‘consumer check’ task as an ordinary task with a message flowing to the credit agency. This can be misleading. Often the process is actually manual and the sales personnel are utilizing an extramural application (swivel chair activity) to gather the data. If this is the case, the process below would be more correct.

Figure 2: The Credit Check as a Manual Process

The second BMPN fragment uses the ‘manual task’ to correctly identify the use of the extra mural activity. Perhaps the use of the ratings agency’s web service would be a good ‘to-be’ process.

Seeking Solutions

Organizations need a method which:

  • Creates Models that are easy to understand; higher levels models should clearly parallel management objectives
  • Accommodates improvements and optimizations without necessitating wholesale revisions, be the structure of the models should parallel the structure of the organization, and promote process reuse or federation
  • Easily models complex processes with core patterns and templates
  • Promotes rapid modeling of processes.
  • Defines a scope and context to every process discussion

Once managers have verifed the objectives, business analysts and experts should create scenarios that achieve enterprise goals. Moreover, because process should be linked to the structure of the organizations, its goals and objectives are directly linked to roles. Until 'swivel chair' processes  are properly documented, management might not know their existence. We describe better approaches to process modeling in our second edition of the Microguide to Process modeling.

Tom Debevoise


Ten Business Rules Usage Patterns in BPM and Business Events

In my opinion, Progress Software's acquisition of Corticon marks the beginning of the end of the monolithic ‘rules-engine’. The idea that there is a business rules engine that is ‘called’ by a ‘program’ is past and there is a more nuanced understanding of the role of business. By my estimate there are at least 10 separate usages of business rules within business processes and business events.

Firstly, business rules play an important role in the rules-driven process pattern. The common monolithic conception of business rules is that it is only an element of a decision that affects the gateways of one process.
There are various opinions about this and certainly I have mine. Yet, it as the BPM industry matures it is probably wise to use more standard research methods to develop an approach. Researchers at Georgia State University and Utrecht University have identified five categories of business rules that have a behavioral influence on the business process in terms of mitigating operational risk or achieving compliance to regulation.
The five usage categories are:

  1. Rules for task sequencing
  2. Rules for actor inclusion or task assignment
  3. Rules for effect sequencing or gateway conditions
  4. Rules for data / information registration
  5. Rules for detection control or event reponses

Briefly, these five usage categories are described:

Task Sequencing: these rules influence the position of one or multiple activities/events/decision (hence process elements) within a business process.  Most often, to create a process that is compliant with the business rules, process elements are simply added, re-ordered or removed. That is, the existing process model is updated to reflect the new rules.

Actor Inclusion/Interaction or Task Assignment: These are rules that influence the assignment of tasks or decision to specific actors. There are several approaches to creating a complaint process. Firstly, defined actors, in the form of participants, can be removed/added or appointed to different elements inside the process. In addition, a processes can call business rules to delegate the activity to the correct actor.

Effect Sequencing or Gateway Conditions: These rules influence the paths chosen by gateways or conditional sequences inside the process. This is the classic pattern of the values, created by the rules, setting the conditions at gateways and directing the flow of the process. That is: the path chosen is directed by an evaluation of business rules associated with individual transaction. This is more dynamic than task sequencing where rules influence the arrangement of the paths.  An example of effect sequencing is a shipping process where, depending on the needs of the shipment, different transportation process activities need to be executed. This is the most common perception of processes and rules. To make the process compliant, business rules need to be enforced during runtime.

Data / Information Registration or Event Responses: These rules influence the recording and viewing of data and information, and the authorization to access it. Most often, to create a process that is compliant with the business rules, internal controls govern timing: how long must the recorded data be kept accurate and the predefined format of complete or registered data. That is: the registered data must contain the following information and authorization, restricting access to predefined user and roles.

Detection Control or Event Reponses: These rules influence how a process responds to events. Events can be external or internal. External events might include weather, or financial events. Internal events arise from the direct or audited results of an activity or processes.  Most often, to create a process that is compliant with the business rules multiple solutions can be used: process elements can be added, reordered or removed, we can have the process respond to an ‘event channel’ or, a new business process can be created to perform the event control.

The last type can be expanded further. Business Events open another set of 5 patterns for business rules. In the Microguide we defined several classes of decision for event processing. These usually occur in the following order: 

  1. Detection, 
  2. Distribution, 
  3. Aggregation 
  4. Correlation 
  5. Assignment

Not every event processor includes all of these steps. Also, after detection, it is not required for these steps to be executed in this order. The list shown is ideal, in theory, for all event processing situations.

Detection: In event detection, logic is applied to data monitored by the event detector. A business-relevant or ‘event-of interest’ is discovered when the event process matches the logic with the data.

Distribution: In event distribution, the detected event is immediately alerted to the affected process or systems, according the logic and levels of participation. Rules logic decides the level of involvement and the destination of the distributed event. Distribution logic can simplistic, as in a burglar alarm during irregular hours. Complex distribution logic might deliver the event based on the scale of the data within the event.
Events are distributed through two distinct patterns; either through a message, or a broadcast. The message corresponds to the BPMN message. Event processing targets the event activated messages at a process instance. The broadcast distribution method is denoted as a signal event in BPMN, and is analogous to a radio signal.  While a message persists after it is sent, a broadcast is only relevant for a short period of time.

Aggregation: Many business events are only meaningful in combination with other events of a similar or related nature. Logical significance might arise from the timing or temporal nature of the other events. In aggregation, event processing uses business rules logic to identify significant events from a group of events.
Correlation: Event processing includes a correlation step. Event processing can play a role, by detecting internal or external events and correlating these with concurrent processes or data in the enterprise. Active business processes might be identified as intended recipients. Business rules define what data or process states are included in the correlations and the logic of the match.

Assignment: At the conclusion of the cycle, in assignment, the event is assigned to one or more processes. Business rules can choose which process is assigned the event for action. As with all rules, the assignment can be as straightforward as the unconditional assignment to a single process or it can be a time, capability and dependent assignment.

An example of this is shown in the figure below:

Process modelers and business decision modelers use the three metaphors of business processes, business rule and business events these to model solutions. In early incarnation of the business rules engine, the BRMS was built as a separate engine to be included in a batch or separate process. Our understanding of the use of business rules has become more nuanced and more focused on the intended usages. If you model your processes with these rules your rules will be more focused and more compact.

Tom Debevoise