Goal-Orientation in Process Management
As ‘goal-orientation’ has become another buzzword for BPM I will summarize here the state-of-the-art. The concept of ‘goal-orientation’ is not a new IT term but used in psychology, economy, organizational theory and recently in agent-based artificial intelligence research. Even J. Schumpeter wrote already in 1934 about it in ‘The theory of economic development.’ So it is not a new and fancy term, but relates to human behavior.
My first try at goal-orientation was for process analysis in our 2001 Inbound/Outbound Communication solution. In our projects we had noticed that organizations usually had no process owner and thus no goals defined, so we used the model below to define processes. It also taught us that business rules are not sufficiently capable of dealing with complex events, which led to the development of the User-Trained Agent that uses machine learning to discover pattern matches in business events.
Despite recent claims to the opposite, one can’t simply add goal-orientation to flowcharted processes, much as you cannot simply add complex business events, rule-engine based decision making, predictive analytics or social capabilities to a rigid flowchart engine and promise benefits. You will find vendors whose brochures boast ‘goal management’ or ‘self-healing processes’ to be buzzword compliant, counting on buyer ignorance. Using goals as explicit process controls requires a completely different functionality than for example BPEL engines provide. Standard BPMN does not support it either. If you see normal flowcharts only? Not goal-oriented!
Goals in Artificial Intelligence
Goal-oriented concepts became relevant in artificial intelligence research to guide autonomous agents who would communicate goals to each other. At the BPE conference in London a couple of weeks ago I had the pleasure to meet Dominic Greenwood, PhD. We had one of the most interesting discussions I had in a long time. Dominic is a brilliant mind who has to be credited with some of the scientific work around goal orientation in agent based systems and especially BPM. Dominic and me have many similar ideas and knowledge in agent technology but we use it for different approaches to different targets. I focus on guiding humans to deal with unpredictable processes in ACM and Dominic uses multi-agent technology to perform constraints-based optimization of predictable processes using goal definitions for BPM. I do expect that we will end up with something that consolidates both.
Goals can represent a desirable future state or situation, a trend value or a set of rules or constraints. Therefore one needs a powerful data mapping capability and rules embedded in the process engine and not external. For business, goals can represent strategic objectives (often quite intangible), customer outcomes (pure perception), operational targets (must be measurable) or process goals (ideally tangible). While process goals should in principle be tangible, that does not mean that the actions needed to arrive at that goal are equally tangible or predictable. That is the whole point. A process flowchart assumes all-out predictability, while goal-orientation is about guidance under uncertainty. As certainty is not achievable because any event can happen at any time in any process context, goal-orientation is the only real world approach to process management. The ‘outcome’ (can be output, handover or result) of a process is achieving goals that are causally connected to business strategy and capability. A causally linked set of goals represents the Strategy Map of a Balanced Scorecard.
Flowcharts Can’t Model Goal-orientation.
Let’s face it, orthodox flowcharted process management won’t survive the social and collaborative revolution because it is not really agile. Goal-oriented process behavior can be seen as an internal guide to a process that helps to clarify why we do the process and verify if we achieve it. There is no external, additional monitoring necessary. The BPEL ‘standard’, for example, is not only unable to fully map BPMN, it is further utterly unable to handle goal-oriented models. In a typical BPMS flowchart there is no need for goals, because the path to the endpoint is strictly conditional, even if the user can ‘dynamically’ switch the execution path (manually set gateways) like the tracks on a railroad yard.
Executable process goals linked to operational targets and strategic objectives are different. The top-level goal of an end-to-end process (i.e. customer outcome) has dependent and optional process-goals with linked optional activities. Some process goals may have dependencies with operational targets or required service levels. Service levels are not monitored overall but related to goal-achievement. The activities necessary to reach them can change at any point in time due to unpredictable events. There is no overall pre-defined flow and possible happy-path from end-to-end. Goal-orientation requries the ability to add new goals, activities, resources, rules, constraints and performers when required. There is no sense in having goals and then restrict the performers in achieving them. Adaptive technology allows to reuse this new ‘knowledge’ for future executions without needing a BPM bureaucracy for redesigning flowcharts.
Because in reality most processes have to deal with unpredictable complex business events, neither predictive analytics nor business rules will be able to realistically deal with them. Statistical analytics lack the ability to identify the hidden, time-correlated, data-pattern context of an event. That’s why I chose pattern matching agents who work in real-time during process execution and learn from human actions.
Goal-oriented Requirements Language
Researchers at the University of Toronto defined the so-called GRL Goal-oriented Requirements Language that can be used to specify goals, soft-goals, beliefs, tasks, resources, actors and agents as elements for goal achievement with special relationships such as means-ends, decomposition, contribution, correlation and dependency. In difference to BPM-flows, GRL does not assume that the goal WILL BE fulfilled implicitly when the process has been completed. The Papyrus object-relational model database uses an expanded GRL definition for business planning and links it directly into process execution. That offers a unique, radical step forward in business process management. Gartner Fellow Tom Austin said to me: ‘You sound like you work for McKinsey and not for a software vendor.’
The real world practically of goal-orientation as a human concept brings process management that much closer to the business than BPMN ever could. It is normal human behavior that different people choose from multiple optional plans that all have the opportunity to fulfill a particular goal. When we are in problem mode we might reconsider our approach by means of optimizing for minimal constraints, sometimes intuitively. Typically we follow our patterns of experience until something goes wrong or changes, meaning an unexpected event happened and then we adapt the plan as needed. It can be that something is out of sequence, in the wrong state, does not fulfill some parameters or an external dependency is not being satisfied. Think of your normal working day! And then it can be any combination of the above or something totally different. Once I can simply identify a new event and how I want handle it, it is no longer a problem to my process execution. But it does break rigid, dynamic, or ad-hoc process flowcharts.
Complex Business Events Require Goal-Orientation
While agent technology can perform constraints-based optimization, that does not mean that it can deal with unexpected events – rather the opposite. That has been one of the real challenges in artificial intelligence. Plan optimization requires at least a library of plan fragments and goes awry when things aren’t as expected in the plans. Self-healing processes for BPEL were a research subject, but never reached real-world usability lacking goal-orientation. Self-healing strategies with rules could handle known execptions but are quite useless for unpredictable processes. Encoding rules that trigger on incoming data is very complex because one has to know beforehand what the trigger conditions and context might be. It is not longer a complex event! The ability to define at problem time (not a-priori design time) necessary constraints and rules for a new situation that the unexpected event caused, must not be a special case but NORMAL. This can only work by enabling humans to add actionable knowledge to the process definitions (training by doing) without needing IT guys to encode it. I am not proposing that all performers will define rules and activities. To empower the business and give performers explicit goals for a coherent business exercise, processes must be defined in business terms and not IT terms.
The million dollar question for goal-orientation for any business:
‘What are the right goals for an outcome and the right actions to achieve them considering the current situation?’
‘How can goals be sensibly expressed, understood by business people and defined to follow company strategy?’
The Outside-In methodology considers customer outcomes and is thus one-dimensionally goal-oriented but doesn’t encode them, thus loosing the promised agility. To leverage social concepts for efficient and effective processes, goals must be expressed in an understandable AND executable way, using rules in natural language. That again needs a business language ontology that is used to express the customer outcomes, strategic objectives, operational targets, the resulting process goals and finally business rules and constraints – hence a Business Architecture.
For those who are interested to read more: Gartner Group’s Jim Sinur covered of the subject (in November 2010) in this post titled ‘The Future Target of BPM.’ Good post, while it is not a future target but available. I have covered ACM goal orientation and its link to Business Architecture on this blog before.
Sorry McKinsey, if we take the mystery out of business planning and execution …