As ‘goal-orientation’ has become another buzzword for BPM I will summarize here the state-of-the-art. The concept of ‘goal-orientation’ is not a new IT term but used in psychology, economy, organizational theory and recently in agent-based artificial intelligence research. Even J. Schumpeter wrote already in 1934 about it in ‘The theory of economic development.’ So it is not a new and fancy term, but relates to human behavior.
My first try at goal-orientation was for process analysis in our 2001 Inbound/Outbound Communication solution. In our projects we had noticed that organizations usually had no process owner and thus no goals defined, so we used the model below to define processes. It also taught us that business rules are not sufficiently capable of dealing with complex events, which led to the development of the User-Trained Agent that uses machine learning to discover pattern matches in business events.
Goal-Orientation for Inbound/Outbound Content in Papyrus (2001)
Despite recent claims to the opposite, one can’t simply add goal-orientation to flowcharted processes, much as you cannot simply add complex business events, rule-engine based decision making, predictive analytics or social capabilities to a rigid flowchart engine and promise benefits. You will find vendors whose brochures boast ‘goal management’ or ‘self-healing processes’ to be buzzword compliant, counting on buyer ignorance. Using goals as explicit process controls requires a completely different functionality than for example BPEL engines provide. Standard BPMN does not support it either. If you see normal flowcharts only? Not goal-oriented!
Goals in Artificial Intelligence
Goal-oriented concepts became relevant in artificial intelligence research to guide autonomous agents who would communicate goals to each other. At the BPE conference in London a couple of weeks ago I had the pleasure to meet Dominic Greenwood, PhD. We had one of the most interesting discussions I had in a long time. Dominic is a brilliant mind who has to be credited with some of the scientific work around goal orientation in agent based systems and especially BPM. Dominic and me have many similar ideas and knowledge in agent technology but we use it for different approaches to different targets. I focus on guiding humans to deal with unpredictable processes in ACM and Dominic uses multi-agent technology to perform constraints-based optimization of predictable processes using goal definitions for BPM. I do expect that we will end up with something that consolidates both.
Goals can represent a desirable future state or situation, a trend value or a set of rules or constraints. Therefore one needs a powerful data mapping capability and rules embedded in the process engine and not external. For business, goals can represent strategic objectives (often quite intangible), customer outcomes (pure perception), operational targets (must be measurable) or process goals (ideally tangible). While process goals should in principle be tangible, that does not mean that the actions needed to arrive at that goal are equally tangible or predictable. That is the whole point. A process flowchart assumes all-out predictability, while goal-orientation is about guidance under uncertainty. As certainty is not achievable because any event can happen at any time in any process context, goal-orientation is the only real world approach to process management. The ‘outcome’ (can be output, handover or result) of a process is achieving goals that are causally connected to business strategy and capability. A causally linked set of goals represents the Strategy Map of a Balanced Scorecard.
Flowcharts Can’t Model Goal-orientation.
Let’s face it, orthodox flowcharted process management won’t survive the social and collaborative revolution because it is not really agile. Goal-oriented process behavior can be seen as an internal guide to a process that helps to clarify why we do the process and verify if we achieve it. There is no external, additional monitoring necessary. The BPEL ‘standard’, for example, is not only unable to fully map BPMN, it is further utterly unable to handle goal-oriented models. In a typical BPMS flowchart there is no need for goals, because the path to the endpoint is strictly conditional, even if the user can ‘dynamically’ switch the execution path (manually set gateways) like the tracks on a railroad yard.
Goal Dependencies in Papyrus Business Architecture Model
Executable process goals linked to operational targets and strategic objectives are different. The top-level goal of an end-to-end process (i.e. customer outcome) has dependent and optional process-goals with linked optional activities. Some process goals may have dependencies with operational targets or required service levels. Service levels are not monitored overall but related to goal-achievement. The activities necessary to reach them can change at any point in time due to unpredictable events. There is no overall pre-defined flow and possible happy-path from end-to-end. Goal-orientation requries the ability to add new goals, activities, resources, rules, constraints and performers when required. There is no sense in having goals and then restrict the performers in achieving them. Adaptive technology allows to reuse this new ‘knowledge’ for future executions without needing a BPM bureaucracy for redesigning flowcharts.
Because in reality most processes have to deal with unpredictable complex business events, neither predictive analytics nor business rules will be able to realistically deal with them. Statistical analytics lack the ability to identify the hidden, time-correlated, data-pattern context of an event. That’s why I chose pattern matching agents who work in real-time during process execution and learn from human actions.
Goal-oriented Requirements Language
Researchers at the University of Toronto defined the so-called GRL Goal-oriented Requirements Language that can be used to specify goals, soft-goals, beliefs, tasks, resources, actors and agents as elements for goal achievement with special relationships such as means-ends, decomposition, contribution, correlation and dependency. In difference to BPM-flows, GRL does not assume that the goal WILL BE fulfilled implicitly when the process has been completed. The Papyrus object-relational model database uses an expanded GRL definition for business planning and links it directly into process execution. That offers a unique, radical step forward in business process management. Gartner Fellow Tom Austin said to me: ‘You sound like you work for McKinsey and not for a software vendor.’
The real world practically of goal-orientation as a human concept brings process management that much closer to the business than BPMN ever could. It is normal human behavior that different people choose from multiple optional plans that all have the opportunity to fulfill a particular goal. When we are in problem mode we might reconsider our approach by means of optimizing for minimal constraints, sometimes intuitively. Typically we follow our patterns of experience until something goes wrong or changes, meaning an unexpected event happened and then we adapt the plan as needed. It can be that something is out of sequence, in the wrong state, does not fulfill some parameters or an external dependency is not being satisfied. Think of your normal working day! And then it can be any combination of the above or something totally different. Once I can simply identify a new event and how I want handle it, it is no longer a problem to my process execution. But it does break rigid, dynamic, or ad-hoc process flowcharts.
Complex Business Events Require Goal-Orientation
While agent technology can perform constraints-based optimization, that does not mean that it can deal with unexpected events – rather the opposite. That has been one of the real challenges in artificial intelligence. Plan optimization requires at least a library of plan fragments and goes awry when things aren’t as expected in the plans. Self-healing processes for BPEL were a research subject, but never reached real-world usability lacking goal-orientation. Self-healing strategies with rules could handle known execptions but are quite useless for unpredictable processes. Encoding rules that trigger on incoming data is very complex because one has to know beforehand what the trigger conditions and context might be. It is not longer a complex event! The ability to define at problem time (not a-priori design time) necessary constraints and rules for a new situation that the unexpected event caused, must not be a special case but NORMAL. This can only work by enabling humans to add actionable knowledge to the process definitions (training by doing) without needing IT guys to encode it. I am not proposing that all performers will define rules and activities. To empower the business and give performers explicit goals for a coherent business exercise, processes must be defined in business terms and not IT terms.
The million dollar question for goal-orientation for any business:
‘What are the right goals for an outcome and the right actions to achieve them considering the current situation?’
And:
‘How can goals be sensibly expressed, understood by business people and defined to follow company strategy?’
The Outside-In methodology considers customer outcomes and is thus one-dimensionally goal-oriented but doesn’t encode them, thus loosing the promised agility. To leverage social concepts for efficient and effective processes, goals must be expressed in an understandable AND executable way, using rules in natural language. That again needs a business language ontology that is used to express the customer outcomes, strategic objectives, operational targets, the resulting process goals and finally business rules and constraints – hence a Business Architecture.
For those who are interested to read more: Gartner Group’s Jim Sinur covered of the subject (in November 2010) in this post titled ‘The Future Target of BPM.’ Good post, while it is not a future target but available. I have covered ACM goal orientation and its link to Business Architecture on this blog before.
Sorry McKinsey, if we take the mystery out of business planning and execution …
I am the founder and Chief Technology Officer of Papyrus Software, a medium size software company offering solutions in communications and process management around the globe. I am also the owner and CEO of MJP Racing, a motorsports company focused on Rallycross or RX, a form of circuit racing on mixed surfaces that has been around for 40 years. I hold 8 national and international championship titles in RX. My team participates in the World Championship along Petter Solberg, Sebastian Loeb and Ken Block.
Welcome to the Real World! 
Great post Max! There is so much content here to pay attention to, but I feel this is a key topic that needs to be progressed in both BPM and ACM.
Whenever I talk about goals, I fear that half the crowd will immediately jump to the (mistaken) idea that goals are just tasks, and they will in their mind draw a chart of tasks. They look at your goal dependency chart, and see a work-breakdown chart, which feels like the same thing, but has essential differences. We need more examples of this.
Which brings me a question: Your dependency graph is a hierarchical tree. Can you have two goals that both depend upon the same subgoal? Can you have cycles where goals depend upon each other. I am just curious about this.
-Keith
LikeLike
Keith, thanks for reading and commenting.
Yes, goal-orientation is not an ACM only subject. I just see little benefit of goals for predictable, flowcharted processes.
Goals are in principle a kind of work break down too so that is not wrong. The difference is that you don’t define the work in terms of activity, but the work in terms of its outcome. You specify AND verify and don’t assume that certain activities will achieve the goal. If people accept that as a first step we have already achieved a lot. The point is that the BPM analyst already decided what is necessary to achieve the goal by defining the process. Only when the engine uses the goal to for example select tasks from a set of optional ones then it can be usable.
To your second question, one can clearly have one goal being a prerequisite to more than one higher goal. But you wouldn’t see that in your normal hierarchical view of the general outcome. If you look at a Balanced Scorecard you have by default four perspectives with different goals each. A causal dependency map can hava any number of links. I was already reminded that my posts are too scientific so I didn’t show the GRL charts with any number of links. The hyperlinks show the GRL and BSC causal maps. We can model the same, but they are even less understandable.
LikeLike
Max, I think you might be interested in “goal oriented risk modeling”: http://bit.ly/pdYDLS .
LikeLike
[…] Goal Orientation Processes – Max J. Pucher Let’s face it, orthodox flowcharted process management won’t survive the […]
LikeLike
Max, I really appreciate your mention to Artificial Intelligence as an enabling technology in process management providing scientific background to the goal-oriented buzzword. I fully agree. Let me add that not only agents technology but also hierarchical planning and scheduling technology might also be very appropriate to this end.
These are algorithms able to “digest” actionable knowledge representations about processes and reason about the sequence of steps that will drive us towards the goal. This basic reasoning about actions may also be complemented with cause-effect and temporal reasoning, resource allocation or problem decomposition and some other capabilities.
You might want to have a look at this short paper: Arturo González Ferrer, Juan Fdez-Olivares, Inmaculada Sánchez-Garzón and Luis Castillo. Smart Process Management: Automated Generation of Adaptive Cases based on Intelligent Planning Technologies. Proc. of BPM Demonstration Track 2010, Hoboken, USA, September 14-16, 2010, Vol. 615. (http://decsai.ugr.es/~faro/LinkedDocuments/SmartProcessManagementBPM2010_rev1.pdf).
In fact, we will also talk in more technical detail about this issue in forthcoming posts in the blog of my startup IActive Intelligent Technologies focused exactly on this: http://blog.iactiveit.com/?p=224 We could also discuss on how to reason with those complex decompositions pointed by Keith.
Thanks again for driving so much discussions and sharing your thoughts. Regards.
LikeLike
Luis, thanks. I think it is great that you are driving the principles of AI for process management forward.
I know JABBAH but not in detail. For me it is too flowchart-oriented and not really practical to deal with the knowledge worker dynamics. My main concern is that it does not take the business data and content into account for its analysis and recommendations. XPDL is not able to carry the data models that are necessary. Cause-effect and temporal reasoning is not practical when one has to deal with arbitrary events and intutive knowledge worker decisions that make each case/process different than any other.
I have tried to work out how it actually uses the goal-orientation in the XPDL environment. Maybe you can focus on that in practical samples? Thanks, Max
LikeLike
Max, thanks for your comment. I agree with your point on the unpredictability of knowledge worker dynamics. In terms of AI it means that they may be more “reactive” (next best action: Markov Decision Processes, Agents theory among others) and less “deliberative” (plan for a flowchart: Planning, Scheduling).
But I have to say that this may depend both (1) on the type of problem and (2) on the approach chosen to deal with this uncertainty.
(1) The type of problem is not a two-choice decision: either deliberative or reactive, but a continuum, with extremes problems (fully reactive or fully deliberative) and many problems not so extreme, with features of both worlds, would require a hybrid treatment.
(2) The approach chosen may be different along this spectrum. One may decide not to plan, just to react to the environment, of course. But one might also decide to try to plan, anticipating some decisions, structuring a kind of flowchart or process and gathering benchmarks about it, if you want, and revise/replan in the case of changes during the execution.
This is the case of on-line planning and scheduling (have a look at the European Network of Excellence on Planning and Scheduling notes about that http://www.informatik.uni-ulm.de/ki/Planet/TCU-olps/). Cause-effect analysis is obviously needed to anticipate the future and project actions to that future in a deliberative or hybrid approach. In a fully reactive environment it makes no sense.
Let me tell you that the study of this range of problems and approaches is a vivid and passionate discussion in the Planning and Scheduling academic community (to which I belong, I am CTO of my startup, but I have also been associate professor of the University of Granada during the last 17 years).
Extremely interesting to see this topic in the business arena too!
Thanks Max.
LikeLike
Luis, thanks for reading and commenting.
I know that all this is an exciting dicussion in the scientific community. I am involved in it. But we need something that is practical in the real world right now. Which is why I developed and patened pattern matching engine that does work in state spaces with a mixed planned and reactive workload. It matches human actions and events to state space patterns.
My main criticism: Cause and effect analysis is not possible for emerging capabilities. You can not decompose the ultimate logical reason for emerging functionality. Yes, Markov probabilities can pinpooint areas of probability, but the trick is to involve humans and let them teach the ‘machine’.
Hence, what I call transductive training. I learn from human actions that react to events and the pattern matching considers business and process information as one.
All the best, Max
LikeLike
I like it. It sounds very well, a machine learning approach to learn patterns, as Markov policies, and continuosly adapt to the reality without having to compute these policies (a map from states to actions) explicitly. Something related to “reinforcement learning”?
Regards.
Luis.
LikeLike
[…] La transformación del negocio tendrá que basarse en una plataforma de procesos que utilice las capacidades, los valores y sus mensajes a clientes como parte de los objetivos finales. La Arquitectura de Negocio resultante tendrá que mejorar todos los puntos de contacto y facultar a las partes interesadas para crear, ejecutar, evaluar y adaptar procesos a su propia discreción, es por ellos que se centra en objetivos de los social media generales, objetivos de procesos y objetivos de negocio. […]
LikeLike
[…] La transformación del negocio tendrá que basarse en una plataforma de procesos que utilice las capacidades, los valores y sus mensajes a clientes como parte de los objetivos finales. La Arquitectura de Negocio resultante tendrá que mejorar todos los puntos de contacto y facultar a las partes interesadas para crear, ejecutar, evaluar y adaptar procesos a su propia discreción, es por ellos que se centra en objetivos de los social media generales, objetivos de procesos y objetivos de negocio. […]
LikeLike