The Problem with BPM Flowcharts

I have several points to make and will cover them one by one. (Note: This merges content from various previous posts for clarity.)

Why is BPMN 2.0 as-is not usable as design tool for business users?

Disclosure: I am not against BPMN, because we use it with extensions for adaptive, goal-oriented processes.  But BPMN with its 520 pages of specs does not enable an untrained business user to describe processes. Why? Just look at all the issues that make it hard for an expert. You think that a business user can handle any of it? Despite the progress from 1.1, the enhanced and additional definitions still are prone to very ambiguous models. An ‘Activity’ can still represent any number of different functions and the new event types are lacking in detail on how they interact with the flow. There are now artifacts, but without the ability to describe methods, attributes and states. Not surprisingly, there are no business rules. The proposed UML-like data modeling has not made it into the spec. All inbound and outbound content, as well as GUI artefacts and rules will still have to be done outside the BPMN model. Hence, no model preservation, no roundtrip, no usability by the business and thus a lot of project management bureaucracy.

Because of the above BPMN can only represent a small part of a complete process and it is neither usable as-is for very dynamic processes, nor is it easy enough to use for the business users to describe their processes. Especially the interaction of various processes is extremely difficult to design and coordinate. Business users cannot create event handling exceptions for an intersecting set of processes. All these issues need to and can be solved as extensions before BPMN becomes usable, and that is the great thing about it. It does however not resolve the general problem with flowcharts.

Flowchart models have only the acting agents (users) as real-world entities whose decisions to perform functions on artifacts can’t be modeled. Even BPMN 2.0 with its additional artifacts is STILL functionally blind to the inner functionality of the major elements of a business process (content and data in context with rules) and therefore to its decision logic. It remains basic routing. Flowchart modelers see the world (a business) as a ‘complicated’ system that can be decomposed into a sequential causal chain of functions ‘to be executed’ and user actions to be assigned and a limited set of logic switches that causally control the execution. The real-world process knowledge is however hidden in a) functions performed by different agents who influence state changes on business entities and b) patterns of entity states and attributes that cause different agents to perform certain functions, and c) a variety of complex business events that can change entity states at any time. Flowcharts are unable to represent that even if one could extract and analyze all the information from the agents and the entities! Why?

Flowcharts can’t model Complex Adaptive Systems (CAS)!

A classical model of physics (i.e. a watch) is complicated, but the economy or a large business that consists of individually acting agents is complex (Anderson, Arrow, Pines – 1988). The flowchart fallacy is to see a business as complicated rather than complex. Holland et. al (1986) proposed a method of real-world modeling in which the world consists of various states S where a transition function F(S) changes a given state at time t into a new state a time t+1.  The caveat is that in a complex system the modeller using a modeling function can neither accurately describe the state space with all its entities nor can the function F – and its causality performed by the agents – be accurately known. So clearly, all models are wrong, but some models can still be useful. Is that the case for flows in CAS?

Only in a very limited sense. The Law Of Large Numbers allows us to calculate a statistical representation of the real-world across a large number of entities. While it enables us to see some data distribution at certain snapshots in time, this observation does however not describe causal rules that could be linked to a causal chain. It cannot be decomposed into why the various agents came to a particular decision. The individual agents have only a certain probability to interact with certain data patterns in an observed way. It is neither financially feasible to analyze all possible variants and all complex processes by questioning agents and ensure that they are inline with needs over time. Therefore ‘standardized’ processes are designed that hopefully cover 80% of process variants over time and thus would produce the savings. This is not the ‘Pareto Principle’ BTW, that would in difference suggest that  for example 20% of processes will produce 80% of profits.

The ‘getting 80% right’ proposition fails however, because it does not take process interdependencies into account. Let’s assume that one process owner has only ten processes to achieve his goals and he manages to get each one to cover 80% of all variants (as proposed by BPM methodology) then the ten processes will intersect and produce also a combined set of possible variants. The question is how much of the total process space (as given by the joint graphs) is covered by this main variant. To determine the ratio of the overall coverage we use the cross correlation matrix for all processes. If we make the assumption that all variants are independent (the cross correlation matrix is the identity matrix), we can calculate the answer: 1 / (1 + n * (1-p)) with n=10 and p = 0.8 we get ~33% (Thanks to my colleague Dr. A. Adensamer). Only A THIRD of all possible variants for this process owner are covered! My own calculation using simple probability is even lower. To achieve 80% variant coverage for all ten processes, each one would have to be 97% correct! If all process owners of a business are only getting a third of their processes right by automating them, then the cost of achieving quality at process handoffs will be much higher than the optimized 80%-process seems to suggests. Is anyone surprised that the flowcharts may not be satisfying the goals of customers and that short term cost savings do not continue long term?

Reductionist decomposition cannot model emerging properties!

The reductionist modeling hypothesis suggests that the more a decomposition in smaller elements takes place the more accurate the model would be. Anderson proposed in 1972 that reductionist models are misleading for complex systems because they cannot map and predict bottom-up emerging properties, such as a group of people collaborating to combine their knowledge. Decompositon does further not consider the outer context of a system. Russel Ackoff pointed out that taking a British car to pieces (analysing it) does not explain WHY it has the steering wheel on the right side. Thus reductionism cannot be used to reconstruct the business logic from the decomposed tasks of a process analysis.

Therefore BPMN 2.0 and flowchart models cannot represent real business processes, because a large business clearly is a complex adaptive system that consists of individual acting agents with its employees and customers. Trying to simplify a complex business into a ‘complicated’ one by decomposition into small steps, makes the model unable to adapt to the customer agents or to other environmental changes. Agent interaction is emergent and not designed. That’s why it needs Centers of Excellence to redesign the blueprints from scratch. Because of the programmed data, content and rule functions this requires long projects to implement, and thus flowcharts reduce the agility of a business rather than improve it.

How can processes in complex adaptive environments be supported?

In this post I do not even discuss the issue that human decision-making is not following rational future utility, but uses emotionally intuitive heuristics that are much more effective in uncertain environments than boolean logic based on more data. Model that with a flowchart …

I thus propose bottom-up process creation, where knowledgable or skilled actors assign real-world entities with state-changes and rules to process goals that are linked to top-down business targets, will produce a much more realistic and most of all adaptable model.  To use BPMN for that it has to be extended. Then it can support human decision-making and makes it transparent rather than replace it. Users find it much easier to interact with real world entities and their states than with purely abstract flows. Rather than to enforce  agents to perform in a certain way, the process definition only enforces basic rules of conduct, which creates substantial transparency and therefore flexibility and adaptability. Process management has to offer complex real world models of people acting as a social group on business entities. Using SocialBPM to define flowcharts still creates the above limitations.

Does not enforcing a flow as in Adaptive Case Management mean that there is less control, less efficiency, or less compliance? Absolutely not! Rather the opposite. In difference to ad-hoc or dynamic BPM, there is no intentional point of process leniency in the flow, because the focus is empowerment. Control is only applied where absolutely necessary to allow in-process innovation and optimization by the people in the know. And they do not need to understand a BPMN Designer to do so.

Basic flowcharts are ok for very simple, repeatable processes if the design is done by experts who can also add all the other elements needed. In very rigid, bureaucreatic environments, such as government agencies or hospitals, flowcharts can be usable but they are still not necessary to execute control. While BPMN is not a tool for business people, the representation can be helpful for understanding smaller process structures once the processes are organized around business goals and mostly controlled by rules and not gateway flow-logic. The trick to make BPMN 2.0 usable for business users is to hide it during design, but offer it as optional visualization of the activities to achieve the defined PROCESS GOALS during execution.

I am the founder and Chief Technology Officer of ISIS Papyrus Software, a medium size software company specializing in communications and process management. I wrote several books and hold a number of patents. My quest is to bring common sense to IT, mostly by focusing in human quality issues rather than cost saving, outsourcing and automation. I am also Chief Architect at VIPorbit software which provides mobile relationship management.

Tagged with: , , ,
Posted in Adaptive Process, BPM
16 comments on “The Problem with BPM Flowcharts
  1. Max,

    You are spot on & thanks for giving some very solid arguments from CAS.

    The more I see how processes emerge in community platforms when you put together a bunch of people working towards a business objective, the more I feel that its best to leverage the amazing capability of activity streams and rudimentary workflowing capabilities in such platforms that allow you to create various kinds of objects and define various states and actors can just join in based on the particular case.

    I have been wanting to write this for a month or so now, but been unable to. But your post gives me some new triggers. Hope to come up with something soon.

    Thanks for the post again. Am definitely with your proposition!

    Regards,
    Prem

    Like

  2. Max, you’re right of course.
    At best, flowcharts provide a ‘suggestion based on perception’ of one possible course of action out of a number of process variations which we know exist but tend to treat as unique exceptions – costly, time-consuming and wasteful.

    Plus flowcharts (for whatever reason) tend to wake the beast in modellers (let’s not call them designers) to show how much effort they put into their models by making them complex and too elaborate. Seems that many have forgotton that it’s not the model they should be focussing on but the (business) process.

    BtW, the 30% (If all process owners are only getting a third of their processes right by automating them…) seems quite optimistic, compared to the results in our TestLab. Let’s not forget that a single error of logic can prevent a process being implemented.

    Will BPMN help in that? Somehow I doubt it.

    Thomas

    Like

  3. […] BPMN 2.0 – Max J. Pucher The trick to make BPMN 2.0 usable for business users is to hide it during […]

    Like

  4. Really enjoyed this Blog, Max. I had similar thoughts while attending SAP TECHED recently. I just wrote a blog which relates to your concept of CAS.

    Adaptative processes are all over the place in Aerospace or Mechanical engineering. The theories of System Dynamics which try to model these processes have been pretty good so far. Stochastic modelling for system dynamics go even further in the modelling accuracy.

    BPMN 2.0 is fine with relatively linear processes or with slow dynamics. It probably shows its limits with more dynamic processes. Emerging technologies like BAM combined with BPM will help. Complex Event processing engines are at the core of the orchestration of this model.

    Thierry

    Like

    • Hi Thierry, thanks for reading and the comment. I do completely agree. Yes, flowcharts are ok for systems with little dynamics. I don’t se BAM as a solution to the dynamics, but embedded processes and much of the guidance can come from identifying the complex events that represent decision points within unstructured processes. It is not a certain step in the flow that defines that but a pattern of states. Regards, Max

      Like

  5. As ever I enjoy reading your blog posts. I will be honest and say, early on I wasn’t too much of a believer in CAS, and was very much routed in the BPM flowchart approach (mainly because thats where I started work).

    However, after looking at so many of the projects I have been involved in, and reading your posts on my own blog, I have started to guide my own company away from BPM type solutions. We still brand as BPM but in essence, we will be implementing CAS, pretty much due to your insight.

    I have long thought that designer tools and maps, looked great for a demo to customers, and are great for very simple processes (the classic that I see time and time again is expense claims) but for the real world, and the majority of processes, workFlow, BPM, call it what you will, is simply too fixed.

    It is hard though to communicate that, CAS for me, is an evolution of Case Management and BPM, into something that effectively allows better management of business and processes while empowering the end user. I now find myself in many arguments with BPM professionals with regards to this, many of which either dont get the concept, or take it too far and believe CAS is some form of AI that learns our processes and makes life too complicated….

    I hope posts like this make it more clear, and that the BPM community will start moving away from their maps. If not, I think I will need some rebranding of our own products and drop the term BPM completely…

    Like

    • Dear Andrew, thanks for reading and commenting. I am glad that not everything I say comes across in a strange way as opposition to everything IT and other vendors do. I have a focus on human aspects and thus on the natural dynamics of life. I simply don’t see flowcharts in any way suitable for that on a larger scale. Large scale, yes. Fairly simple sub-processes, yes. But overall in between we need dynamics and adaptability without a complex governance bureaucracy. Yes, one can see CAS or ACM as an evolution of BPMS and many have added those features. But the majority of processes can’t be handled with augmented BPM. Billions have been spent in selling the notion of encoded processes and vendors won’t step down from there. Analysts have linked their reputations to these concepts and will defend them no end. That is life too.

      It needs a new mindset that is related to what we see in Social, which is kind of the swing of the pendulum into the other extreme. Yes, we can use BPM, ECM, CRM, CRM and some MDM and integrate it all together with E20 and what will be produced is exactly the opposite of what is needed. It certainly won’t be adaptive.

      Thanks, again!

      Like

      • I agree that everyone will defend their position even if they are wrong…I think I am a rare thing, I have come down and moved away from my previous position on BPM…

        I think the term BPM is now lost in flowcharts and thats a real shame. Its a little like workFlow seemed to get lost in BPM, yet there is no difference really…

        My own company will be dropping all forms of mapping processes, and has opted for a far more adaptive and agile solution, something that brings together a number of silos and hopefully, empowers users rather than restricts them….Thats the goal…

        Like

  6. […] In terms of case management  Craig has produced some great research like: ‘Dynamic Case Management- An old-idea catches fire.’ He does teleconferences with AIIM on ‘Support Your Information Workers by Understanding and Implementing Case Management’ or Forresters own TCs as well. So it is all heating up. Forrester lists ActionBase, Appian, Cordys, EMC, Global 360, IBM, Pallas Athena, Pegasystems, Singularity and … whoa … even CRM maven Sword Ciboodle as their entrants for the next DCM Wave. The likes of Fujitsu, HandySoft, Ideate, OpenText, and Oracle (!!!) will only make it into the ‘Ripple’. Us and Whitestein Technologies are also listed in that second group, but we are the only ones who actually provide GOAL-oriented processes! There are vendors with products that have to be integrated, others who are simple application tools, email-collaborators, but hardly anyone who does embedded content. But therefore OpenText is jumping now on the ACM bandwagon and promotes ECM as case management. Virtually every BPM vendor now wants a piece of this DCM market especially as the frustration about hyped-up benefits claims of BPM sets in. Read about Dr. Rashid Khan’s assessment of BPM Simulation and Optimization that is a perfect add-on to my own perspective on the problems with flowcharts. […]

    Like

  7. […] outcomes. The problem is that when combined with other processes, that 80% degrades, according to Max Puchar, down to around 30%. Meaning, that even when we’ve flowed out a process, when the process […]

    Like

  8. […] processes needs to be considered as CAS, as Max brings to our attention in his latest post on why BPMN & Flowcharts are insufficient. Or as mentioned here, not a CAS rather an analogy of CAS.Whichever it is, I am decided that I will […]

    Like

  9. […] to the idea of using flowchart style languages for ACM, and this is well covered in a post “The Problem with BPM Flowcharts“.  He says “BPMN 2.0 and flowchart models cannot represent real business processes, […]

    Like

  10. Hi Max,
    Wish I had read your blog post some years ago. I have considered a blog post under the title “Flowcharts considered harmful” but I’ve hesitated as the arguments are hard to make clear. I therefore appreciate your point that flow charts make complex business systems unnecessary complicated.

    The key is to express business rules and let the system guide the user on what to do next, where end users can choose among all possible options in the system. It is somehow similar to using a GPS while driving. When you start you enter the goal and the GPS calculated the route. As a driver you’re not limited to follow a specific path (unless of course there is only one road) and sometimes GPS even provides you with alternative paths.

    During you voyage you can choose to change path and take the next right. The GPS will re-calculate a new path for you. The GPS keeps guiding you despite the fact that you changed the route and didn’t follow its initial guideline.

    The difference between flow charts and business rules is the flexibility similar to what the GPS does. In a system with business rules the system can provide guidance to the business user on their voyage towards the goal. The users can choose their own way forward.

    I also very much appreciate your comment that the absence of flow charts doesn’t lead to less control or less efficiency – quite opposite. The reason is fairly simple and relates to your calculation. As a flow chart system often only model a fairly small path of the complex business system, in your example 33%, while setting a much higher expectation of compliance, the end-user very fast loses faith in the system as they realize it only support a limited part of their need. Further, what is the value of a strict and rigid control when it can only be applied to a very small part of the complex business system? Very often the rigid controls become a problem for the knowledge worker as the system enforces meaningless rules.

    Best regards
    Morten

    Like

  11. Might be a good idea if you join the discussion on LinkedIn – BPMN group. Discussion Why BPMN ?

    Like

    • Hi, thanks for reading and commenting. I am very active in most of the LinkedIn BPM groups. There are too many and because I am not an avid BPMN supporter, I do not feel I can interact with the people there in a productive and positive manner.

      Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Max J. Pucher
© 2007-16
by Max J. Pucher. All rights reserved.
Real World Statistics
  • 200,855 readers

Enter your email address to subscribe to this blog and receive notifications of new posts by email.

Join 8,157 other followers

%d bloggers like this: