How to get over the SOA hangover: new design processes

publication date: Jun 24, 2008

How to get over the SOA hangover: new design processes

Over the past few years the acronym SOA (Service Oriented Architecture) has been associated with a number of promises: building large scale systems in a rigorous way, ease of integration through standards adoption, separation of concerns of processes through orchestration tools via declarative rules, even ‘programming’ from certain analysts, Not to mention the promise of leveraging legacy systems that can get new life breathed into them thanks to the wider global role they now play. Big promises all.

With all the hype there have been some casualties along the way. Certainly the road to a successful SOA project is filled with challenges, many of which lay around stakeholder buy-in, the ability to apply governance both deep and wide through the lifecycle and enterprise-wide, pure complexity, integration testing issues and the old chestnut of what exactly a transaction means in an enterprise setting!
As technologists, we often focus on the technical challenges around projects of this kind, around standards and the methods we use to design software. We also look at the development of service patterns, applying different methods to formalise the software lifecycle like, for example, in the debate between Rational Unified Process (RUP) and agile design.

The problem with promises is that they quickly need to show benefits on the balance sheet. This is especially true of large strategic projects where investment is often nervously given. Successful large projects only become so because they remain accountable from inception onwards and they must ensure this rigorous software methods are employed since SOA means integration of either new or wrapped legacy systems: the complexity of the entire system is multiplied by the complexities of all its parts as well as the dependencies between those parts. Indeed, the dependency issue can force a counter agile approach. The key design tenet is to supply self-contained sentient service components. They can wrap the desired functionality; supply both technical and business level monitoring information so that higher-level services can consume them. A greater level of abstraction often enables the main integration points to grow more organically rather than with up-front design. Not to say this wider reaching design doesn’t or shouldn’t occur, far from it. However, iterative design feeds into it rather being the recipient of it. From a technical viewpoint this enables development and testing to begin earlier. It also allows mapping to happen internally through abstraction levels reducing the need for pan-enterprise standardisation. Whilst this does not necessarily mean that a common meta model is not required, it does mean it can grow incrementally and/or can be mapped from/to by the service parts. Users can see a common thread of agile design feeding into wider designs here. Service Component Architecture is the term often used to describe this approach of building services from smaller components.

Moving onto governance of the design process, this is the absolute key to success. It is unlikely that all sub parts of the system have the same deadlines and aren’t subject to localised pressures of their own. By breaking service contracts into smaller self managed parts, a more agile and thus more accountable gradual delivery can begin. This isn’t to say that the more traditional Service Inventory Analysis is given up, far from it. However, logical views of a large complex system are all too often far removed from reality and analysis often gets diluted away when the technical designers pragmatically begin work on the system. The only way to succeed is to iteratively take business process models into raw service components via test driven development until the user has a full service inventory, allowing individual deliverables to continue. 

At Lab49 we are starting to see design methods such as RUP and OpenUP adopt more agile practices as well as more agile methods being applied in a wider enterprise setting. It should also be noted that wider enterprise architectures methods such as TOGAF (The Open Group Architecture Framework) are starting to encompass agile techniques. A mistake is often made that agile means ad-hoc, it does not. It actually enforces a higher degree of rigueur because of the much higher accountability.  Various software engineering practices have attempted to increase ROI through re-use [of?], de-coupling [of/], component coherence, abstraction and so on.
Having talked about design applications and issues of control, we have seen that developers tools are not enough to build an SOA system. Support tools and service discovery tools will always be required to help find, understand and (re) use all components – all too often the wheel is reinvented sometimes through ignorance.

As an example, it is possible to design a system that would control a number of physical devices some of which could respond with an error condition hours or even days later. In this case, a classic rollback is out of the question and a set of cleanup or compensation actions must be set up. All of this can often be identified and delivered within these smaller components because the user is local to the problem domain quite quickly over a few iterations.

So to the future: because there are few formal definitions for what SOA actually is, it will always be a chaotic path for some developments. Whilst governance and standards are vital for any interchange and integration projects, a stepwise but formal iterative process mixed with the higher level views (either top down or bottom up) will provide a clearer path because of the frequent reset points. A final example of this is managing transactions in a SOA environment. Because a failure in a synchronous system usually involves a rollback operation, undoing what was done during the operation, within a workflow system, a firm may have operations paused for long periods of time and found to be in error much later. To avoid these situations, one needs to design skeletal compensation statements right at the beginning.