For the past three years, I have been working on and off on an open-source CFML-based Web application. It  started out as a simple system to store arbitrary structured and unstructured content. I started using it to build more and more complex Web applications, and over time it grew in size and scope. I thought about it for awhile as a content management system, but content management is not what I was aiming for, and not where the platform has really evolved.

After struggling with terminology and purpose, I started thinking about the application as an application platform. What is that? I see it as an implementation of typical application patterns in an integrated package that allows a develoepr to use it in whole or in part, building on the core libraries to create a new solution. 

Once I had the concept clear in my head, I started casting about for a name. After lots of pondering and brainstorming sessions with my colleagues on the CF-Community list, I decided to call the platform Colony. To me, Colony is all about staking out new territory on the Web, building compelling new services, and advancing the state of software.

Colony is also about shared effort and shared reward. To that end, we have just released the platform in alpha under the Apache Software License 2.0. You can get the alpha code and see more about the platform at www.cfcolony.org. The site is graphically challenged and light on content at the moment, but that wil cahnge soon. 

ZDNet's Joe McKendrick has a blog entry up about different kinds of architects in an enterprise IT environment. You can see the post here, What’s the difference between an ‘SOA’ and ‘enterprise’ architect’?

I like Joe's distinctions between enteprise architects and SOA architects. In particular I like the analogy of seeing an enterprise architect as a city planner and the SOA architect as someone charged with delivering city services. 

Back in January I went to the FASTForward Conference in San Diego. One of the keynote speakers, Andrew McAfee of Harvard Business School, spoke about Enterprise 2.0 and how it will transform business in the next decade or two. One of the things that stuck in my head after the conference was how emergent systems will drive value by allowing users to build structure themselves rather than imposing structure on them.

As I have thought about that concept over the last few months, what has occurred to me is that we have a similar opportunity in the enterprise architecture space, using technologies like SOA, to design systems that are self-organizing. Take, for example, setting up an employee directory service rather than just posting the directory on the corporate intranet as an application silo. Once the directory service is posted, capable users from all over the business are able to read from the service, and could provide novel and interesting uses of the data. HR issues aside, I believe the effect of providing major functionality as a service will provide tremendous value for businesses over time. I see this trend as a long-term shift toward emergent architecture in the enterprise. 

Today I present the last part of my four-part piece on IT ecosystems. This last part deals with the human /organizational side of SOA - understanding corporate politics, building coalitions, gaining consensus, and providing governance and process to support your infrastructure.

4. In an ecosystem where every property has access to every process or service, where do you draw the lines about which system exposes which functionality? The nirvana of process decomposition can devolve into the ugly reality of political turf wars, with individual middle managers trying a land grab to "steal" each other's roles by incorporating functionality from one system into another.

Dave Watts:

The bigger issue I've run into isn't people trying to stake out other
people's stuff, but rather the opposite - people not being willing to
support what other people are using. If I build a system that depends on
services from three or four different organizational units, who gets blamed
when those services fail? (And they will.) There's a lot of paperwork that
goes into any serious SOA effort I've seen, involving who has to support
what, how they have to provide support, etc, etc.

Mark Kruger:

Looking around at blogs on topics like SOA and integration I think this point - interdepartmental cooperation - is missing from the discussion… "Banging heads" is rarely productive. Getting buy-in is a proactive process that requires salesmanship, collaboration, congeniality and integrity. Everyone has "spheres of influence" and a CTO may temporarily get what he or she wants by running rough shod over those carefully crafted power structures, but in winning the battle he or she will more firmly entrench such structures and quite possibly lose the war.

Rob Brooks-Bilson:

This is where governance comes in.  There are lots of vendors out there now selling solutions to help you manage that governance, but at the end of the day, a well defined Enterprise Architecture with the backing of management is necessary (I believe) to be successful.  Enterprise Architecture can be a wonderful tool for limiting the scope of turf wars.  The governance layer helps you maintain the inventory (it's more than just UDDI for web services) of what's available, where, and who's responsible for it.  You need to build a culture of reuse.  It's difficult getting people to move away from the idea of "I need to get a list of customer numbers, so I'll just write my own query to get them from the db" to the idea of "let's create a single service for retrieving this information, exposing the process through the ESB".  

When you say you want to influence people to do the right thing, that is certainly a valid approach, but it requires a heck of a lot of effort given the potential number of players in an organization.  An alternate approach is to tackle this through an Enterprise Architecture initiative.  You can involve key players from various groups, but the main person you need to influence here is the CIO/CTO.  You get them on board with the concept of EA, get their backing, and the getting everyone else to fall in line - especially when one of the goals is to better align IT with the business - should be a whole lot easier.

A good SOA effort, like any enterprise-wide initiative, requires the active, ongoing cooperation of lots of people. Like never before, IT needs to work hard in a collaborative role, both inside the IT department and in the rest of the enterprise, in order to successfully execute an SOA strategy. “Banging heads is rarely productive”, as Mark noted (in response to a comment by me about the innate urge to do just that). As Rob B. notes, a governance layer can provide the mechanism by which disagreements are settled and rules finalized. Much like governance in a change management group can ease tensions among business users, application owners, and infrastructure staff by providing clear rules about the production release of changes in IT infrastructure, SOA governance can provide a clear set of rules and a neutral forum for resolving disputes among business users, application owners, and other stakeholders.

Finally, all of these questions pertain to just a small part of current IT ecosystems. IT evolution touches phone systems, network infrastructure, desktop computers, collaboration systems, hardware infrastructure, external systems from customers to vendors – virtually every piece of technology in the modern enterprise. Successful management of an IT ecosystem is an ongoing campaign to measure the effectiveness of current systems, look for opportunities for improvement, and guard against risk. Thanks to everyone who responded to my original questions and contributed to this discussion. 

Here is part III of my discussion piece on IT ecosystems. This post covers system decomposition and reuse with SOA, and the associated security issues that companies may face in implementing such strategies.

3. Decomposition is all the rage with enterprise systems- we could for instance use the ecommerce system's web services layer and the SOA/ESB to decompose the system into its individual processes, then expose those processes as needed across the enterprise. Again, I wonder where the useful limits of that metaphor are. One idea is to provide a common header in the content management system that contains an AJAX component to expose the customer shopping cart on any external-facing web property in our infrastructure. Nifty idea in principle, but it raises some serious questions about security.

Dave Watts:

…issues with session hijacking, etc, are no
different for exposed services than for any other web applications.
Typically, in my experience, a big part of making SOA happen is wrapped in
authentication/authorization services. On the other hand, the more exposed
something is, the more likely you will discover problems with it.

Rob Brooks-Bilson:

I think this is where process mapping comes in.  If you first define the business processes you are interested in, you can then map those process across system boundaries.  This can be fairly simple to do, or a royal PITA depending on how mature your business is, and how well defined existing business processes are.

Security is an issue here (especially for externally exposed services).  It's not overly difficult to do, but there are a lot of options depending on what type of services you expose (web servivces, RPC, gateways, etc.).

Security is becoming a bigger and bigger component of IT efforts in companies from startups to multi-national conglomerates. SOA strategies create two serious issues for enterprise IT. By decomposing applications into their component processes and exposing them to other applications via SOA, IT may create a situation where applications are both dependent on one another and are linked across a common platform.  

The dependencies create a scenario where applications could be subjected to outages because other applications on which they depend are compromised. Critical business functions like invoicing need to be fault-tolerant and hardened against indirect attacks. Second, linking applications across a common platform creates a situation where an attacker needs only to find one good attack vector into the SOA itself in order to potentially compromise many systems.