SOA and BPM Agility: Perfect Complements or Illusions?
In 2007 I wrote a quite ranting newspaper article ‘Why SOA does not deliver’ that I also posted here. It is still one of the more popular posts on my blogs (next to ones on climate change). As the subject is still as relevant – and doing SOA/BPM is still as difficult – I decided to create an updated, compacted post here. Am I saying that both SOA and BPM are useless? Not at all, but I propose that they are just two elements of a larger solution requirement. Implementing SOA and a BPMS does nothing than waste a lot of money.
Service-oriented architecture (SOA) was first described by Gartner in 1996 in an SSA Research paper. Over time it turned into a huge marketing hype that only recently slowed. BPM has gone through many definitions and technology changes and just became more complicated over time. BPMN 2.0 has become a nightmare of ambiguity and still does not allow directly embedded data and content models. My problem with SOA and BPM is the unproven claim that they improve the agility of a business. I am serious: Agility is a human property and not a system functionality. What stops current organizations from being agile? No, not software but small-minded resistance to true innovation in IT. SOA is being used to sell huge software stacks and agile BPM promises ‘puppet-on-a-string employees’.
Agility will only come from executives being willing to innovate technology and empower their employees by doing away with rigid business processes and the straightjacket like impediments of BPM, CRM and ECM silos.
Business Process 1911’s Style
Most SOA/BPM proponents are stuck in 1911’s-Taylorism. Frederick Taylor believed in specialization and proposed rigidly structured corporations. Hence, each and every IT application represents such a business process fragment, because if not, according to Tom Davenport, we would not need it. So what is a business process? Rummler and Brache (1990) proposed that ”a business process is a series of steps designed to produce a product or service for a customer.” And that is quite simply, incorrect. In ‘Reengineering the Corporation’ Hammer and Champy made a very important suggestion: “Not the individual task or process is important but only the customer outcome.” BPM systems and BPR projects miss the ability to simply adapt to goals. Substantial bureaucracy is needed to manage the complicated changes in a BPMS to modify a process. Dynamic ad-hoc capabilities improve things, but they offer only limited runtime flexibility and no user-created processes. BPM requires however primarily a Business Architecture and Strategy that allows transparent processes.
Hardly anyone seems to realize that the need for an agile enterprise is created by the EVENT-driven dynamics inherent in process related business communication. It is the state of the communication content that controls the process and not its abstract, meaningless steps. Business communication is not just a document or an email, but can be anything: a selection menu, a web page, a sticker on the document, a data record, images, or even a voice recording or video. It clearly can also be Social networking messages that are utterly disruptive to a process. No matter how much time and money is spent on business process analysis, there will always be one more communication item needed for a business process once it gets going. This is why collaboration tools, email and social media are now so pervasive. They don’t require analysis to communicate.
I propose that BPM and SOA should focus on improving human decision-making within the processes that make up the organization and not on cost-cutting by reducing humans. And here is a core problem of BPM and also business rules. Human decision-making IS NOT related to Boolean logic, but most likely an emotionally weighted pattern-matching ability of the brain. Rationality is only used for post-decisions justification or verification. Antonio Damasio – today Professor of Neuroscience at the University of Southern California – has long researched neural systems for memory, language, emotion, and decision-making. In his 1994 book, “Descartes’ Error: Emotion, Reason and the Human Brain,” he documents his discovery that “humans with dysfunctional emotional centers face grave difficulties in decision-making.” So the idea that business decisions can be automated is a grave fallacy.
To improve process management we need technology to support and improve intuitive human decision-making far beyond logical rules. Before we can achive that, we must provide a relevant information context to the decision maker and not just more data and therefore information noise. According to Peter Drucker (Managing in a time of great Change – 1995), the worst thing we can do is to encode future decisions into the process.
Governance or Change Management?
In terms of business agility the core issue is IT change management and not governance. Change management is not about project management but METADATA: structured data about data that are required to define the properties of business entities and logic functions. Most understand metadata as the search criteria of an archive but they are in fact the core descriptors of Business Achtiecture entities. We need managed metadata because the IT problem starts when things change. Maybe you can change the process with BPMN, but try to change a data field and the process structure and logic simply – but catastrophically – collapses.
Even if you use all-out ‘open standards’ such as DTDs, XSLs, XSLTs, XPATH, WSDL, UDDI, BPML and BPEL and a lot of Java code, a business entity attribute in a SOA service interface is NOT automatically propagated to all user interfaces, all Java modules, all process definitions, all XML transforms and all databases. SOA does not improve or speed up in any way what users get in terms of functionality. As SOA does not add common versioning and deployment, we are right where we were before – in a ‘Big Ball of Mud’ – just more complex. Add SOA and/or BPM to any kind of infrastructure and you are making it harder to manage! And therefore less agile!
So clearly, without a Virtual Metadata Repository (not an archive) to define the Business Architecture, there won’t be any change management! The VM-Repository propagates metadata changes from its stored Business Architecture throughout all processes, business rules, user presentation, content and interfaces. We can now empower business users to adapt processes, rules, content and presentation to their needs without complex IT projects. Clearly, business agility is not provided by SOA or BPM methodology. An agile organisation will be empowered by user oriented Change Management!
The Truth About IT Innovation:
IT people in large organizations – more than any other people I know – shy away from innovation, because it is disruptive and risky. Risk is not desirable when 99.999 uptimes are required. We thus have tested-to-death, frozen application icebergs.
SOA is not innovation but a bureaucratic evolution of object-oriented messaging that took a wrong turn because vendors needed to sell what they had. Doesn’t anyone realize that SOA has only been invented so that the existing silos can stay and don’t have to be thrown out? SOA is not investment protection but the spiderweb that turns the existing silos into A PERFECT TRAP. Who in his right mind can believe that SOA enables them to replace vendors or silos? It is so complex and expensive that all related services are outsourced. Excuse me, but who could believe than an outsourcing contract makes IT more agile?
Fact: You can’t lead by walking in someone else’s footsteps. If you don’t innovate, your IT is not leading edge no matter how buzzword compliant you are. Benchmarking IT against others who don’t innovate will only pull everyone down to the same low level. But then … you can show that you are among the best in your benchmark! I am always asked for reference installations (but then strangely enough, no one wants to be one!) which is impossible for innovative software. Hey, it’s NEW! That means there can’t be a huge number of users. Yes, being innovative carries some risk. Finanical institutions, who take substantial risks with their client’s money with derivatives are the worst when it comes to trying innovative software. How strange is that?
Agility in business people will be supported through innovative technology and can most certainly not be found through best practices. This new technology will have to be self-explanatory, easy to implement and adapt to the needs of the business on the fly. I call it Adaptive Process and it will replace all the CRM, ECM, BPM, BI and TLA (Three Letter Acronym) silos and if at all, use SOA as one of many possible backend interfaces. I propose that your business will either innovate or get stuck in the SOA/BPM trap …