IT Ecosystems - Part II

Yesterday I posted the first part of a discussion I started on a technical mailing list about IT ecosystems. Here is part two of the discussion, which pertains to SOA.

2. What is the role of SOA? Based partly on the FAST vision, I am beginning to think less about using SOA for data integration and more about just using it for process integration. In other words, I'm not sure I want to integrate customer data at all. I might just want to leave it in the CRM system, use search to retrieve it in other systems, and use SOA to operate on the data.

Dave Watts:

SOA isn't about data integration at all, in the "raw data" sense of things - it's not intended to be simply a remote query mechanism. It is supposed to be about process integration.

Brian Meloche:

We use SOA heavily.  Web Services are great for inventory calls, order status and transactions, but it's slower than your standard query a DB and return results.  Timeouts are common, and you need to code your CF to EXPECT for timeouts.  From my experience, your mileage may vary, it's best to use SOA for only what is absolutely necessary.  You can get more performance out of triggers that update your web DB, storing as much as you can in your web DB, and using aggregate tables whenever possible, than to rely on heavy SOA integration for everything.  It's like a spice... use too much and it will spoil the meal.  Use the right amount and it will really make for a great system.

Robi Sen:

Web Service != SOA.  People seem to forget that SOA came from EAI architectural patterns in the late 80’s pre XML.  Almost all so called SOA architectures I have seen are simply people using XML to create a API or essentially do RPC calls.  No auto discovery, no federation of services, no process coordination etc.

Robi Sen (in a separate comment):

SOA's used to be called Federated Service Architectures as well as Service Oriented Architectures back in the late 80's and early 1990's (I meet people all the time who think SOA is a very recent term) and its actually a concept that came from the Enterprise Application Integration world.  SOA is not just about integration but federating service by exposing the fundamental services of disparate applications and allowing them to function as if they are one large application.  I think your short statement captures much of it although it seems most people think any application that exposes something via XML is SOA.  Its wonderfully fun talking to people about your SOA that has 0 XML or WSDL involved in it.  You can usually tell if they understand the architectural concepts of SOA by how often they ask.. 'Well where are the web services'?

Rob Brooks-Bilson:

We've implemented SOA in a different direction than most.  Instead of creating lots of services and then trying to build composite applications from them, we've approached SOA purely from a reuse and abstraction perspective.  As you know, we are big proponents of XML messaging and the Enterprise Service Bus (ESB).  For us, we started by implementing the ESB as a way to replace an existing customer support/b2b delivery system.  What we ended up with was the infrastructure to make SOA work for us.  Our ESB provides the foundation for us to tie any of our systems together - ERP, shop floor, portals, B2B, etc.  Once we had the ESB in place (initially using POJOs for a lot of the end-point connectivity), we introduced the idea of services to the corporation.  Some of those services are web services, while some are not.  The main point is that we are abstracting API's and providing a common entry point to all of our systems (the ESB).  The interesting problem we're working on right now is around SOA governance.  As Dave W mentioned in his reply, there is a mountain of paperwork involved in managing the full life-cycle of services in a SOA.  You absolutely need a governance framework around this if it is going to scale beyond a handful of services controlled at a single location, controlled by a centralized group.  For us, we have efforts going on all around the world, with strong IT presence in many of them.  We have to have a way to manage that and provide governance - especially in the age of things like SOx.

SOA is starting to look like a big part of a real solution to the decade-old problem of enterprise system integration. Still, enterprises are just barely scratching the surface of SOA, and many internal enterprise IT staff are as yet unfamiliar with what SOA means for their businesses. As emerged early in this discussion, there is a big distinction between using SOA for data integration and SOA for process integration. A perfect IT ecosystem would contain only SOA-enabled applications that communicate seamlessly across a common bus and share data and functionality as needed without redundancy or replication. That scenario, however, is not the reality in modern enterprise IT ecosystems. Architects and developers are therefore left with the challenge of integrating systems that were not designed with integration in mind.

In the case of these application silos, data integration is a necessary. Data can be batch-synchronized to start, and after achieving an initial sync, systems can rely on SOA to maintain data integration across systems. Note that data integration, even in the case of application silos, leads directly into process integration. If a customer changes their business address on a web site, the web site will invoke the ChangeAddress service to notify the appropriate application silos that an address has changed and must be updated.

Also, note that technology is only one component of a good SOA strategy. Governance is the other major component of SOA. Without good governance, a technology like SOA that spans multiple systems and areas of responsibility is destined to fail.