Navigation
  • The MicroGuide to Process and Decision Modeling in BPMN/DMN: Building More Effective Processes by Integrating Process Modeling with Decision Modeling
    The MicroGuide to Process and Decision Modeling in BPMN/DMN: Building More Effective Processes by Integrating Process Modeling with Decision Modeling

Entries in Business Rules (20)

Saturday
Dec212013

Business Rules and Business Process Exceptions

Often a business process team will create a working prototype of a business process. These prototype might arrise from 'use-cases' or business cases.

This idealized process develops a deep understanding of the final version, yet much of the important work remains. You should even consider executing this development artifact in a test bed. Yet to complete a business process, the technical team should detail the process diagram with the necessary scheduling, exceptions and compensations. The purpose of the details is to erect a more complete process model. The business processes should act on all the messages, including the system failures. The process should handle every anticipated exception.

There are 3 types of exceptions may affect process execution:

 

  • Technical exceptions
  • Transient exceptions
  • Business exceptions

 

As we will see, business rules can create, named business exceptions. If the BPEL is transactional, the process should compensate for transactions that fail to complete. In compensation, the process cleans up or backs out records from ERP’s, operational datastores or data warehouses.

Often processes should be complete within a time period, so there could be time out actions noted on the diagram. Business processes usually receive a message and translate that message into yet another message for consumption by the next part of the process. BPMN is powerful for developing time-outs, exceptions and compensations. As seen in the BPMN fragment below, you can attach time-outs, exceptions and compensations directly to a subprocess task list. When time-outs, and exceptions occurs within the two process tasks, the BPEL traps these at the point of the subprocess.

 

 

 

 

In this scenario, a grant authorization system evaluates grants and records the data. The time-out BPMN (the clock) will raise a time-out exception and wait. The diagram reports any exception that occurs (the zigzag) to the exception task. The compensation takes care of a record in the grant database.

You can model BP’s with advanced abilities that suspend processing, or communicate with disconnected, mobile processes. They can add new technology such as RFID. These powers yield opportunities for the enterprise to fine-tune their most complicated practices. Yet to develop these capabilities you should understand how BPM tools use the three types of exceptions and what you must plan for when developing the production version of the process diagram.

A process diagram should use exception handlers to redirect the process flow in case a transient or business exception happens. However, no process can manage every exception. There are technical exceptions that will cause the entire process instance to suspend.

 

Technical Exceptions

Technical exceptions occur when the process causes the BPMS instance to fail. An internal server exception should be abnormal; however there are operational circumstances that cause these. Technical exceptions include

The common Java Null Pointer exceptions
Exceptions caused by Illegal Argument exceptions, (suppose an external operational system decides to change a field from 10 to 20 characters)

Server database exceptions, out of tablespace, violation of database constraints

These are outside and therefore invisible to the calling subprocess. Processes cannot handle them because the execution state of the process has stopped. When this technical exception happens, the current transaction is always marked for rollback. The process throws the exception to the scheduler or message router.

Transient exceptions

A transient exception is usually caused by code external to the server like database connections or connection pooling exceptions. Within the scope of the current transaction, a transient exception is visible and the process can catch it with an exception handler. If the exception is not handled in the scope of the current transaction, the subprocess marks the transaction for rollback and throws the exception to the scheduler or message router. The process is suspended.

The suspended process can be restarted and resume processing after the database connection or some other condition is restored.

Business Exception

A business exception is one that can be expected by the process. Examples of these exceptions include:

Exception raised by an activity, a specific, named exception is raised, for instance a business rule could raise a named exception.

Exception caused by receiving an error message (like an SOAP fault). 

 

If the exception is not handled in the scope of the current activity, the activity fails. When the activity defined a coordinated transaction, the transaction is committed. When no exception handling takes place Exception processing begins and leads to compensation and the failure of the process.

Business Rules Decisions and Business Exceptions

Many organizations use BRMS to validate and transform incoming transactions. Validations and transformation are important types of 'decisions' that a business process makes with the data within process flows.

This is a common decision pattern: evaluate a set or transactions, say from an incoming data set. Each record that fails the validation should be stored in an area for latter cleanup. You can either handle the validation directly, with values in the process flows or you can raise a business exception, create a reusable process for handling the exception and move on. Another common decision pattern validates user form input, in a workflow. Users might correctly complete a form and send it to a process flow. If there are business rules that violate complex data and business conditions, the process should collect data from the user at a latter date.

Figure 2 is an example of what I am describing. A decision service validates incoming data. If the data does not pass validation, the subprocess raises a named exception, otherwise the record is processed.

The basic idea is to design your exceptions, time-outs and compensations to enlarge the flexibility of the process and allow most process to continue unattended. Handle the process and move on.

 

Diagram complexity

The complete business process diagram has many important exception features that must be accommodated. Unfortunately, these additional boxes and notations make it difficult for business users to understand. It would be nice if BPM diagramming tools would permit users to disable display of exceptions, compensations and timers attached to subprocesses. The tasks following these should also not be displayed. This would clarify the core nature of the process and create a more useful business engineering tool.

Monday
Aug262013

Past, Present and Future of Business Rules (Redux)

It has been over 3 years since we released the white paper, the Past Present and Future of Business Rules. The paper has been widely distributed and we managed to generate many discussions, some of them were heated. My colleague, Christina Gruen suggested I update this and subsequently I have updated the paper and prepared a webinar. You can register for the webinar here:

To recap: the business rules approach (or operational decision management) BRA/ODM has principally evolved from three sources: Artificial Intelligence, Data Modeling, and Business Process Management. In the white paper, I explained the differences between the sequential logic approach and the inferencing approach adapted from expert systems.  I have experience with both and clearly my bias is towards sequential, graphical approaches. However, inferancing and functional programing is critical for developing decision analytics. Strict formalism in business requirements may sound desirable; yet it creates a lot of work of questionable value for the business analysts and the technician. Organizations are frequently ‘stuck’ in heavy; difficult to maintain environments in many large-scale business rules systems in finance and health care.

Having been involved with many of the standards and practices, I am hearing surprising rumblings that There are development on the horizon that I will be game changers. OMG’s decision model notation is becoming more prevalent, plus other efforts are poised to shift the industry.

Thursday
May022013

White Space in the Standards: Business Rules, Data and Process Modeling in BPMN

According to the Specification (here), BPMN is constrained to support only the concepts of modeling that are applicable to Business Processes. This means that other types of modeling done by organizations for business purposes is out of scope  for BPMN. Therefore, the following are out of the scope of the BPMN specification:

  • Definition of organizational models and resources
  • Modeling of functional breakdowns
  • Data and information models
  • Modeling of strategy
  • Business rules models

Since these subjects are critical to Business Processes, the relationships BPMN is supposed to be evolving. We also know that operational decisions need data to decide, so at this point BPMN is incomplete with respect to creating executable business processes. An executable BPMN diagram might be more complex — it is inadequate to create an executable specification.

BPMN has a data shape, the data object. A data object is a rectangle with the upper right corner folded over, as shown here:

The text label for a data object can be found underneath the shape.  Often the current state of the data object is shown as an attribute shown in brackets under the text label.  As the diagram progresses, the state of the data object can easily be read, as displayed in the Figure below.

Figure: Use of data artifact shapes.

As with the text annotation, the association line attaches the data artifact to another shape. Data Class shapes can be associated with tasks, gateways, events, sequence lines, or message lines.  In message flow, data objects portray the “payload” or content of messages.  

The use of data objects is optional.  Some diagrams may concentrate on flow, while others show the complete details Data artifacts do not directly affect the sequence flow or message flows.  Data objects provide additional information, some of it reflected in the XML schema, without changing the basic behavior of the process.  For instance, the Data Object elements can optionally reference a DataState element, which is the state of the data contained in the Data Object. The data object can detail the metamodel’s XML particularly with a ‘callableElement’. Uses the following additional elements:

  • ioSpecification,
  • inputSet,  
  • Data Input,

The data input and output are detailed in the purchase order approval example above. When a BPMN editor draws a Data Association to an Activity or Event it should generate this supporting invisible substructure.

As mentioned, the specifications are naive with respect to the origin mutation and alteration of data attributes. I've always maintained that this is the realm of business rules (mostly). That business rules through examination of data in order to decide Bolivian conditions controls specific aspects of operational decisions. Some of that data exploration includes analytics such as regressions, projections, and other techniques.

I believe the BPM industry needs to progress on standards with respect to business rules: the representation how they use things like Business Objects expressed in UM, and other expressions of the operational aspects of the business process. The BPMN specification pretty much admits weakness in this. So, even with the data input and output elements of BPMN, more needs to be done in order to specify a business process in a way that does not very important code aspects. The OMG spec on business rules (PRR) seems to be moving very slowly and it doesn't offer a clear connection to the process modeled by BPMN.

Tuesday
Sep182012

Advanced Business Rules Capabilities Address Complex Challenges

At Software Innovations, we have added some interesting new customers including Green Charge Networks and Prosper Software. In addition to other projects that we cannot mention, these development shows how Visual Rules, modeling approach can solve complex challenges in our Big Data, highly dimensional, customer-focused social ecosystem.

 

The Juila set is a highly recognizable symbol for complexity as develops in the study of fractals and chaos.

From our discussions on the internet of things ( and here) and other developments, we know that today’s world of mobile-networked-social media presents new opportunities for business models and customer engagement. Example models include freemium where a service is provided free of charge, but a premium is charged for advanced features. Another style is gamificationwhere game thinking is applied to non-game applications to encourage people to adopt them. These seemingly simple, yet powerful concepts often require deep layers of implementation business logic.

This ecosystem is also being extended by sensors and edge devices in the IOT. For instance; heart monitors provide data to doctors, or fitness appliances through ordinary smart phones. Sensors on bicycles provide detailed data. These products are a visible outgrowth of the internet of things and services (IoTS); yet, to create applications one must leverage the extreme granularity of this.

In addition to the exotic, new world of the IoTS, traditional data sources have exploded in granularity and accuracy. Most of the decisions of the inputs into the operational decision include time based vectors of:

  • Location, derived GIS stored locations, and proximities
  • Awareness of proximal individuals and groups of individuals,
  • Edge device feeds, heart rate, temperatures, video, weather

Emerging business models use applications: mobile, cloud and on-prem, to operationalize decisions. The decisions will tell customers what to do or make decisions- this is critical.  This means that companies must make effective use of this highly dimensional and time dependent data, often in nearly real-time. To achieve their objectives in this new world, these business require perception of environmental elements with respect to time and/or space or situational awareness.

The complexity arises in problems areas when ordinary, yet challenging business rules areas such as consumer retail and end-use applications, security command and control, complex financial transactions, and products with highly hierarchical catalogs, are combined with real time motions of people, vehicles or traceable financial instruments. The characteristics of this might include large, time dependent vectors of business objects. In this highly dimensional area, each data point exists in the time-location space defined by its attributes and by its relative relationship to all other time-location data points. That relationship is the domain of business rules and the pivot points for the decisions that we are describing.  In this realm, the goal of the decisioning and operational decision algorithms is to assign the data point to membership in the most appropriate cluster—even as the subject is in flight. Business Object Data Collections can be linearly separable or non-linearly separable. We have found, particularly these data are often arranged in a vector that must be efficiently traversed.

The company building applications in this complex, operational world faces the dilemma of hand codding solutions or using visual modeling paradigms. These problem often entail an huge number of logic steps with the need to make subtle changes at deep levers. At Bosch Software we believe that only the latter is sustainable.

In our experience, visual metaphors are mandatory for solutions in this ultra-complex, highly evolving world. The business analyst must be able to use their intense focus on business aspects while the technical teams provide a mastery of the technical aspects for a seamless transfer of logic changes. As a demonstration of this, visual rules, business modeling has recently solved problems in these domains:

  • An athletic consumer products company seeking to operationalize decisions about fitness equipment
  • A company with an innovative financial model built a multi-dimensional, dynamic pricing engine
  • A financial institution created dozens of credit score cards, each with over 90 equations and 190 dimensions.

In solving these problems for many customers we have learned the following:

  • Complex dimensional decisions can only be understood when they are ‘chunked’ into simple visual metaphors, otherwise business analysts get lost in code or simply give up.
  • Visual paradigms including decision graphs, decision tables, state-transition diagrams and composites of these are the only way to quickly solve these challenges.
  • Because they cannot traverse and filter vectors of business objects, Decision table-only approaches tightly constrain the type of problem that can be modeled and miss important dimensional aspects of these classes of problems.

In Summary, business operating in this new realm should expect change and build methods that create the proper agility.

Tuesday
Jun052012

Build Survivable, Hardened Cloud Processes with BPMN

Integrating the cloud into your error processing strategy can be challenging. Fortunately, BPMN provides a powerful approach to managing failures, in the form of exceptions and compensations. Adding resources and services into the ‘Cloud’ adds error points that must be managed. The designer of these processes should strive to make their processes ‘survivable’ and hardened. The exception shapes and compensations allow continuous operation throughout intermittent error conditions such as those encounter in integrating cloud services and resources. Without a hardened process your participants might need to use a manual process for the exception flow. This can be an outcome of overfocus on “happy day” paths. Manual processes promote staff improvisations and undocumented “work arounds” and obviously should be avoided.

Figure 1 presents an example of BPMN’s modeling of survivable processes in a survey pattern. If an error occurs in the email-reading process, it can be posted to a log and systems administrators can respond. The survey can continue. Finally, the process can loop back or complete.

Figure 1:  BPMN for an exception in the survey process.

The aim of BPMN is to support “long-running” transactions so it is especially suited to the cloud scenario. For instance, a process-oriented car rental agency might create one process for each vehicle, each using a cloud-based machine health service. In this case, the process could run for years, with thousands of calls to the cloud-based service.

Three Error Types

There are three types of exceptions that may affect process execution:

  • Technical exceptions—the process server has failed
  • Transient exceptions—a needed resource (for instance a cloud service) for the process is unavailable
  • Business exceptions—the condition of the process is in error. Data is incomplete or has errors.

Unfortunately, there is no cure or notation for the technical exception. When the server halts, recovering processes are dependent on the technical nature of the process server.

Decisions, supported by business rules, can create named business exceptions. If the process is transactional, it should compensate for transactions that fail to complete. In compensation, the process cleans up or backs out records from ERP’s, operational data stores, or data warehouses – perhaps these can be cloud based.

Processes often must be completed within a specific time period, so there could be time-out actions noted on the diagram. Business processes usually receive a message and translate that message into yet another message for consumption by the next part of the process. BPMN is powerful for developing time-outs, exceptions, and compensations. In building a basic process in BPMN, you can attach time-outs, exceptions, and compensations directly to a subprocess task list. When time-outs and exceptions occur within the process tasks, the process server traps these at the point of the subprocess.

Figure 2 presents a process server “architecture”, including a cloud services, for understanding the three types of errors. If the process server fails, then it obviously cannot run and a technical error occurs. If the ‘on-premise’ database service fails, then the process cannot get the data it needs to complete. This is a transient error. You should be able to restart the process instance and complete it once the database is restored. However cloud services are more complex, you might not have the same level of control that you would have for the database.

Users send data for the process in application servers. If users send incorrect data, then a business exception has occurred. Finally, you use process steps to resolve the error. If the users have used a cloud-bases SAS system to create a request or order, then more complexity is added.

 

Figure 2:  Architecture for a process environment in a cloud-based ecosystem.

You can model business processes with advanced abilities that suspend processing, or communicate with disconnected, cloud-based processes. They can add new services and capabilities such as payments processing. These powers yield opportunities and fulfill the need for the enterprise to reduce the cost of their most complicated practices. Yet, to develop these cloud-based capabilities, you should understand how BPM tools use the three types of exceptions and what you must plan for when developing the production version of the process diagram.

Cloud-based- Error Handling Example

A process diagram should use exception handlers to redirect the process flow when a cloud based exception happens. So, if the subprocess in the figure below raises an error, then steps can be taken to accommodate the error.

Figure 3: using subprocess for exceptional flow

In BMPN, handling errors arising from a subprocess is known as ‘Exceptional flow’ and, for errors, this is always interrupting: the subprocess that generated the error will terminate, and the alternate “exceptional flow” is followed instead. Later, the process can merge exceptional flow with the main flow. The exceptional flow merge does not need to immediately return to the subprocess that generated the error. If required, a process model can merge several steps downstream.

Finally, cloud process integration diagrams in BPMN should include error handling, compensation, and transactions. All services have the potential for error conditions, understanding how to use BPMN to automate error handling and recover, or be survivable, is critical.