Agile-, AdHoc-, Dynamic-, Social-, or Adaptive BPM

I am pleased to inform you that we have just published ‘Mastering the Unpredictable’ with the substantial support of Keith Swenson, and it is not too soon. The subject of adaptive processes is heating up as you might have read. In the book I discussed the differences between BPM types. Forrester groups products into Integration-, Document-, or Human Centric, which is not very helpful to figure out what you actually need. Vendors position their solutions as such: agile, dynamic (aka flexible), and ad-hoc (often also called human). I started to use the term “adaptive” for process requirements a few years ago. It has been used before but as a substitute for agile. A recent arrival is Social BPM. What does it all mean?

Agile Process — arrived with more powerful design tools and BPMS functionality that gave a business analyst a faster way to design flowcharts. The agile BPM concept requires  the complete analysis, modeling, implementation, simulation, monitoring, and optimization lifecycle. Therefore it is laden with bureaucracy, overhead and long implementation lead times. Processes are mostly laid out in intricate detail, encoded with business rule engines, linked to complex SOA backend orchestration, more complex data and user interface mappings, and in most cases sees content as an afterthought — except for document-centric capture solutions.

Dynamic Process — has been around for a few years now in various incarnations.  It is a variant of agile process and enables the business user to make changes to the process execution on the fly, by for example selecting a different sub-process, but requiring predefined exit points. User changes do not change the defined process template, and a change is seen as either an exception or an unforeseen variant. That hits the limitations of  most BPM systems, because users do not have access to the metadata definitions of process and business variables and therefore these dynamic user changes fall through the monitoring grid.

However, most users who taste Dynamic BPM blood report that they love the flexibility as a new kind of freedom. Such processes can proliferate and if the system isn’t really capable of handling it, a lack of transparency can be the consequence. That is not the problem of the dynamic process but the limitation of the BPM system. I question the need for process standardization as a reaction to the limitations of orthodox BPM systems. If dynamic processes are the cause of distress in your BPM bureaucracy or system they most probably ought to be executed as adaptive processes and not be forced back into rigidity.

Ad-Hoc Process — those are typically simple processes with a couple of steps relating to  content state. The most commonly used ad-hoc process environment today is Microsoft SharePoint. There are however substantial problems to be considered. User-created SharePoint processes and content are a Wild-Bunch mix of Word/Excel/Access pieces interspersed with macros. That makes them neither easily manageable nor reasonably upward compatible to the next releases of MS-Office and SharePoint. But too many business managers are looking for quick-fixes rather than sustainability. When I talked to IT managers in terms of process management with SharePoint the general response was negative, despite users being reasonably happy with the ease-of-use.

I propose that these ad-hoc processes also ought to be managed through adaptive processes, which provides them with the business metadata definitions, the standardized backend interfaces, business rules, and easy-to-use content and user interface creation and adaptation.

Social BPM — For me, process management is about people communication, therefore I always saw the link to ECM and CRM and most obviously to social networking, despite the realization that it can’t be enforced. Social networkers are collaborative t-shirts with backpacks who love what they do because it is their own free choice. BPM proponents believe in upfront design and people control. And in that gap — so some believe — may lie salvation through Social BPM, but in which lifecycle phase could social be of benefit? Design, modeling, implementation, simulation, optimization, deployment, execution, monitoring, or tuning? No clear definition …

Open collaborative effort on the creation of process flowcharts covers at most 20% of the complete process functionality to be designed. Once the flowchart is encoded with the other BPM elements it stops to be social, agile and will never be adaptive. That is not the same as empowerment, which provides authorization and access to resources. A business hierarchy is most effective when the process teams can each focus on their own things. There is little need for an open social collaborative community to DESIGN processes. Surely, two process owners need to discuss their handovers. If the process tools are web-enabled they are automatically ‘social’. Social communication is not enough, because the business needs a central repository to manage and deploy all the process and business metadata.

Adaptive Process — sidelines Business Process Design and Management as a strategic initiative. Process design must rather map the business strategy into a business architecture with capability maps, use cases and process teams. It is however quite impossible to sensibly derive detailed business processes from a strategy without bottom-up participation. Social BPM could help but the result would still be rigid. The process owning team alone must create and maintain the (secondary) service processes, while the management defines the goals in a capability map that links the service processes and the support processes together. That has to be communicated in an easy-to-use business architecture for the process teams directly in a metadata repository. The infrastructure  provides the necessary transparency for management to define goals and outcomes and to monitor their achievement, while business users still retain their ad-hoc freedom. These processes can be enhanced to stricter definition and more complexity any time.

So what does adaptive mean? It refers to internal changes caused by outside conditions that become permanent and make the entity more fitting to those new conditions. Those changes are performed by means of the entity itself and not by some external force. So if you need consultants to design or change your processes it may be agile but not adaptive. With Adaptive Process, endusers do not just collaborate in flowchart design, but they actually create the real-world process on the fly. Not just a simple ad-hoc activity, but with substantial complexity using metadata models from the repository and business rules in natural language for well defined goals. Being adaptive is not about predicting how a process WILL work or to agree on all possible mutations. Adaptive means that real-time knowledge from the last process execution can influence the execution of the next.

Misconceptions — Adaptive process is empowerment, but that does not mean decision-making authority for anyone about everything. Authorized users can however add business rules to a process during runtime. Centrally managed business rules can make an adaptive process as flexible or rigid as needed without ever touching a flowchart. Process adherence in BPM too often forces users to do the wrong thing right. A process adherence culture is thus not a valuable business trait. It requires complex process exception handling to be analyzed, implemented and executed. Give up the rigid process and gone are the exceptions. There is no need to evaluate to see if processes are dynamic, because that is the norm anyway. Rigid processes are the odd and rare ones.

Strict role/policy security must control who is allowed to make what changes to a process. Everything that actors want for a customer is a good thing for the business. Actors do not always know the consequences of that and that’s where transparency comes into play. Adaptive Process is not just a more flexible kind of BPM, where users can make more choices during execution! That is Dynamic or Ad-Hoc BPM. Adaptive Process is about DOING AWAY with the flowcharting process and allow business users to interact with all above process artifacts in real-time and create those processes WITH ANY CONTROL, SEQUENCE AND STRUCTURE NEEDED interactively — without needing further implementation work. I don’t see much of a danger. The difference is that it happens in real-time, by the real people, and for or with the real customers. Therefore there is no need for simulation, as you just adapt over time the process controls to the minimum amount necessary.

For many organizations this power is new and strange, but users love the freedom when they get it. The management loves the transparency and immediate control over processes. Changes can happen more or less immediately. The ones that are the most cautious about using adaptive processes are BPM analysts, IT architects, and IT production managers. This might change once they see it is a fantastic chance to put a well defined process and business architecture in place.

So the risk of Adaptive Process is not in the implementation or in business users resisting process adherence. The question is whether management or IT are willing to hand that much power to the business users, because Adaptive Process enables them to also create protected processes in agile, ad-hoc, dynamic and social BPM style.

20 Comments on “Agile-, AdHoc-, Dynamic-, Social-, or Adaptive BPM

  1. Pingback: Tweets that mention Agile-, AdHoc-, Dynamic-, Social-, or Adaptive BPM « Welcome to the Real (IT) World! -- Topsy.com

  2. Max, this seems like an interesting summary of some of the vague terms that are popping up around BPM.

    I’d like to address your question: ‘tell me where the danger is?’. I have too many times seen people forced to do the wrong thing right or the right thing wrong by incompletely implemented solutions, so flexibility or adaptability in a process makes sense, but is incredibly hard to implement in traditional BPM tools.

    The fact is, making controls enforceable can be done with rules or decisions inside a process, but if you mess with some of the operation before or after, some would say that you have the risk of invalidating the control – its intent or how it operates. Reality is, I can only think of a few cases where this might be an issue, and outside of regulated industries none have come to mind.

    How would you address this real concern of companies implementing solutions?

    It looks like we may have similar ideas for what adaptive process may be all about in reality. I look forward to exploring this more.

    Cheers
    Phil

    Like

    • Adaptive Process also enables object-level security. If a step is mandatory you can’t remove it. If something has to be checked at any time, then create a global rule to do so and protect it as part of the template. You can easily create a process that is as rigid and secure as any flowchart ever could be.

      You are absolutely right about orthodox BPM tools. They are stuck in flowchart paradigm hell … thanks, Max

      Like

  3. Hello Max:

    So many definitions makes me wonder that it will cause confusion on people mind. I remember when OMG released BPMN 2.0 Draft.

    OMG call:

    Private business process – internal use (?!)
    Private business process can be: executable or non-executable (?!)

    Public processes – represents the interactions between a private Business Process and another Process (?!)

    A Choreography is a type of process, but differs in purpose and behavior from a standard BPMN Process. A standard Process, or an Orchestration Process. (?!)

    Max:

    Ad-hoc, Agile, whatever, doesn’t fit in Adaptive definition? Keep it simple.

    Like

    • AlbertoManuel: I didn’t create those definitions. I am just trying to clarify them. I am not the one to call it all BPM. I wanted Adaptive Process to be outside the BPM domain. Others however have different interests.

      Like

  4. Max:

    I don’t agree when you say: “Allow business users to interact with all above process artifacts in real-time and create those processes WITH ANY CONTROL, SEQUENCE AND STRUCTURE NEEDED”

    This looks like a marketing press release.

    5 years ago a leading BPM vendor I worked with promised the possibility to break the barrier of accessing information – they called also artifacts, assessed trough pc/mobile/phone/ whatever people real need to work with. Today this barrier wasn’t breaked. Your prophecy (and end-user expectation) it’s about the capability to connect immediately to data/forms if a user wants to specially when these artifacts aren’t available – that makes people to slow down increase cycle time, error, etc. . This is not possible (in real time). If it is please share some real world examples.

    One last thought.

    If you don’t have process controls you cannot manage it. Not every one are responsible knowledge workers.

    Like

    • AlbertoManuel, thanks. The product to enable this uses an object-relational database, so there is no problem to show anything in real-time. If the artifact isn’t there, it isn’t. You don’t see it and you can’t access it even if there is a template for it. The user interface only shows what is available and authorized for this user role. For GUI forms, the user can pull up a data-object (i.e. from a SOA IF) from a search and drop it into a widget. He can then pull out a set of attributes and drop them onto another GUI widget for example using data filters. He can take the data and drop them into a document template and the variables can be used in the document. All without scripting or programming.

      I do not understand the ‘barrier’ that was not broken. What are you referring to? I am not talking about a prophecy. I talk about what we implemented to do Adaptive Process.

      We do have all the process controls you want. The complete BPMN set plus natural language rules. They are available from a library and the user can drop them into the process, write rules, define actions to be taken, link tasks in sequence, pull in artifacts of any kind, and so on. At the end whoever is autorized can decide to make this process a new version or a new variant of the process. Even if they don’t, all actions taken during the process were trained into a machine learning agent in real-time, so that next time the agent sees a similar situation for this role, he will suggest the same action for this user role.
      As I said, the process can be as rigidly controlled as with flowhcharts or a completely open case container. The process owner decides that.

      Hope that explains it. Max

      Like

  5. Max.

    I understood you statement regarding accessing, manipulating, using what you call (correctly) artifacts, something more visionary like the possibility of a end-user can easily construct such a thing. I mean, BPM suites have the possibility using wizards (sometimes) to connect to data, and you can access it using a form. That’s OK.

    But 99% happens that people do not know how a database is structured (what is a database anyway), what the database attributes mean (custnm is costumer name or costumer number?), where is the information stored. Thus how can an end user manage to do it? A BPM wizard is not enough. Sometimes is necessary code to extract and manipulate data stored in a database table (this knowledge is completely out of reach of 99,99% end users).

    I thought you vision was something that could help people overcome this huge difficulty. A layer running on top of everything that could lead people to access and construct things used in business process. That would be a layer with “artificial intelligence”.

    Picking up again the “custnm” attribute imagine a layer that tells the user, in real time: this is a costumer name filed that belongs to a customer table of a costumer database what do you want to construct with it? The layer could if the user want help finding related like an walkthrought. I thought that this concept was what you were talking about. I think this is the real thing!

    Regards

    Like

    • The points you mention are exactly my criticisms about BPM, CRM and ECM products. The actual implementation is the problem not the dumb flowchart. users can not only construct artifacts but also all elements of real data and process. And yes, the end user has NO understanding of databases, tables and and complex Boolean or otherwise operations. But a business user does understand the concept of a ‘Customer’ and if then that concept exhibits ‘attributes’ in normal user terms such as ‘Family Name’ then there is no problem. These are the elements of a Business Architecture’ and that is the key for making it accessible to the business user. He does not care that the data field is actually pulled in via a complex SOA interface from some silo. Therefore Papyrus is used as an umbrella layer that hides all the IT complexity from the user.

      The problem is that IT does not want to get rid of those ‘fuel guzzling old clunkers’ of CRM, ECM and BPM and continue to pour manpower and repair work into them rather than to dump them for something more efficient and sustainable. ‘We have a huge investment into xxx’ is the rationale, which is actually rather stupid. It is NOT any investment, but rather throwing good money after bad.

      Once again, I am not talking about a VISION but about an existing SOLUTION. Our latest install is a $3m solution for 2000 insurance users. It was maybe a VISION ten years ago, when I started to develop the product. In 2001, I was laughed at for introducing Inbound/Process/Outbound closed loop communication. Gartner Group and Forrester said in 2007 that an artificial intelligence agent is not BPM if it has no flowcharts. But finally the tide is changing.

      Thanks again for commenting. Max

      Like

  6. Pingback: The word for 3/31 is “adaptive” « Thoughts on Collaborative Planning

  7. Pingback: Column 2 : links for 2010-03-31

  8. Max,
    Happy Birthday.

    I too am really excited about the ‘Mastering the Unpredictable’ book coming out (thanks to Keith Swenson)- I am hoping it will start real, meaningful discussions about what it means to manage unstructured, unpredictable, human process.

    Like

  9. For those not familiar with ISIS Papyrus: I know from my own past experience that what we do sounds to good to be true. There is one remedy for this: “See for your self and visit our OpenHouse or attend one of our roadshows” or even better get in contact for an appointment. As IT you have to be prepared to get a paradigm shift, as business you see how you would like to work.

    My story started 3 years ago when I was Senior Manager at Accenture and responsible for Java, EAI, ECM and managed 130 consultants. Having a strong background in OO, Java and AI myself and being certified WebSphere Architect and familiar with products like Filenet, Documentum, IBM Process Orchestrator, Cordys, and others from real-world project implementations I dare to say that I have seen a lot.

    When I visited the ISIS Roadshow in Brussels with my ECM Architect, I saw an upcoming paradigm shift with new possibilities. I decided to give up my career at Accenture and to join ISIS Papyrus. 3 years later I still get excited every day about the things we have achieved and the new stuff that will change the world.

    Like

  10. Pingback: uberVU - social comments

  11. Max

    I agree with the dynamic nature of processes and the constraints that modeling introduces. Whether they are “unpredictable” is still open for debate but the complexity that all the permutations of real world processes may have are hard, if not impossible, to model.

    We’ve built our tools to manage dynamic event based processes and in the past we’ve been critisized that “the arrows in our flowcharts have no meaning”. It was always designed like that but hard to explain. I’m glad there is an awareness of the adaptive and dynamic nature of processes. Your Adaptive Process diagram depicts it very well.

    For me the real value of an Adaptive Process is:

    1. Controlled decision making delegated to the user with the best knowledge to action a process.

    2. Process decisions made in the context of the circumstances of a specific process instance – not a one size fits all process or single path workflow.

    3. Self learning or “emergent” processes that evolve as new knowledge of the process or its requirements develop.

    4. Process exceptions handled on demand or as they happen.

    5. A repository of controlled process activities that can be sequenced dynamically or “on-demand” to drive a process outcome suited to the specific situation.

    Keep up the good work!

    Pieter

    Like

  12. Pingback: links for 2010-04-06 « steinarcarlsen

  13. Pingback: Library clips :: Have we been doing Enterprise 2.0 in reverse : Socialising processes and Adaptive Case Management :: July :: 2010

  14. Pingback: Dynamic Exception Handling or Adaptive Goals? | Adaptive Case Management

  15. Thanks for your valuble information but i want know about what is Adhoc report in BPM? why they use these report? please let know b’coz i always confuse with it?

    Like

    • HI, Ad-Hoc tasks are activities that users can add to an existing process. Some BPM systems support this kind of user action. It serves the purpose of allowing activities outside the planned process. In most cases it is difficult to make those sensibly interact with the predefined process and influence process completion. It is more like a NOTE function than a well defined process task. Hope this helps, Max

      Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

Join 5,327 other followers

%d bloggers like this: