World Wide Webber

I couldn't agree more, but have one slight caveat: I see ESB as an SOA infrastructure (well, if it's done right it is!), but that doesn't mean it has to be tied to Web Services. So, your last sentence about SOAP as the bus is right if you're only interested in a Web Services-based ESB. But that need not be the case for all users. In fact, a minimal middleware system would be UDP based with (perhaps) XML.

Jim, I agree with your observation. The ESB is a one-day fly, it serves a specific purpose but will not be around for long. http://loekb.blogspot.com/2006/03/esb-is-one-day-fly.html

I got to my "Cancel Rant" button just in time... 

But still, the middleware vendors are feeling the cold breath of obsolesence in their necks. 

Jim, we ought to go out there and run a global MMCS campaign. 

CV

I am afraid I couldn't disagree more with your comments on smart endpoints and lack of a need for ESB (or some smart bit in the middle of SOA). My blog item is here: 

http://ronanbradley.typepad.com/ronan_bradleys_soa_weblog/2006/03/the_enterprise_.html

Hi Ronan, 

I'd read your blog entry, and I think we're talking at crossed purposes. From the point of view of an application on the Internet, all the smarts are in the endpoints. I agree there is infrastructure like DNS and routers involved, but all of this is completely transparent to the application (or in the case of REST intermediaries are explicitly part of the architectural constraints).  

Conversely an ESB which acts at the application level is not transparent. It meddles with application payloads and complicates the architecture and implementation. 

I am not dogmatic enough to say there is no need for ESB technology. When I encounter a situation where an ESB is the simplest, smartest thing to use then I will.  

However to assume up front that ESB is the answer to all SOA is clearly inappropriate since each organisation's architecture will differ. 

The ESB vendor community has to stop pronouncing it has the answer when it hasn't heard the questions yet. 

I will still advocate minimal middleware systems as the starting point for any engagement. If other middleware can be shown to add business value, then it should be considered. Otherwise it's business as usual with WS-Fabric. 

Jim

It strikes me that contract mediation (which is the ESB story) is one of the major reasons why SOA is even being discussed. Note that I work for BEA, but I don't speak for them. 

Here's the logic: 

- A service is a combination of implementation, interface, and contract. It is not necessarily a web service. It is effectively anything uniquely addressable with implementation autonomy, an explicit boundary, and a set of clear usage policies. 

- By service contract, I mean every "rule of engagement" required for interacting with a service -- from runtime service levels, to data formats, to schema. 

- A service is not necessarily a web service. An FTP drop point can be just as much a service as an HTTP/SOAP web service, if it's appropriately specified. 

- A service's contract must always be built with particular classes of consumer in mind.  

- Over time, new classes of consumers will come along that you didn't account for. 

- The point of an intermediary (such as an ESB) is to mediate between different contracts of producers and consumers where the differences between contracts differences aren't so drastic as to require implementation changes. 

This is not about making the network more intelligent. An ESB is an endpoint just like any other, it's just another implementation technology, but with a specific bundle of features that orient it towards certain use cases. It has nothing to do with Java, and nothing to do with Java-to-WSDL binding (have you even looked at what AquaLogic SB does?) 

IF your solution is to "not build services in such a crummy way", I'd ask you to elaborate. Does a service have to account for every possible consumer class? Does it have to expose every possible interface type? Does it have to support every message exchange pattern (pub/sub vs. request/response)?  

Or should a service producer just make some realistic assumptions about target consumers, and then delegate some other intermediary to mediate this contract to other potential audiences? For example, that odd endpoint that really wants to interoperate with Service A through COBOL copy books dropped on an FTP site). 

These are not abstract, theoretical questions, these are the real things I get asked by customers all the time. 

I fail to see how an ESB complicates the architecture or implementation. It's just another way of building an intermediary.

Jim, 

I think we are probably agreed on the principle: use as little as possible and keep things simple as possible. The statements that you made *could* be interpreted to mean that intermediaries/ESB are necessarily wrong and should be never used. From your comments, it seems that our positions are closer: it is a case of horses for courses. However, where we disagree is how often an ESB (or other intermediary) will be needed. My view is that in almost every case of a wide-scale enterprise integration problem you will need one. Your view is, I assume, that in most you won't. 

So long as the people doing the implementation start from a position of understanding what an ESB can do for them and when they may not need one, the chances of SOA success will be increased and our positions are compatible. The danger has been that the discussion has become almost theological about purity of service definition versus the evil of intermediation! 

You make a secondary point that an ESB is fundamentally different from DNS/internet infrastructure because it is not transparent and meddles with the payload and complicates the architecture and implementation. Assuming your main objection is around transparency, I don't understand the point. The services do not/should not know about the 'meddling' (i.e. it is transparent). In fact I would argue that the meddling makes the architecture simpler as it removes the complexity from each of potentially many end points and aggregates it into intermediaries which can service multiple end-points with common mediation capabilities. I know that we are not likely to agree on this one but it is worth the debate. 

Ronan

Hello Stu, 

You've raised some great points. I think you are probably in amongst a small group of people who understand the ESB is valuable as an endpoint toolkit rather than as the thing that sits in the centre of your enterprise, which is a view I have some sympathy with. 

You're right to suggest that not all services are Web Services; however in the context of this discussion it's useful to compare WS technology to buses since they are competing technology approaches to solving similar problems. Irrespective of the type of service that you're building however I would disagree with you that you need to have knowledge of that services consumers. Servers (from client-server) are built that way, not services. Services are utterly ignorant of their consumers to ensure loose coupling, which means that the service is blissfully oblivious when new consumers come along because they, like the old consumers, are nothing to do with the service at all - all they are endpoints that act as sources and sinks for messages (which the service is very much interested in). 

As you point out intermediaries sit within your service boundary, not on the network (intelligent networks are a bad idea we know). I agree with this. But it does not mean that a service has to account for every possible consumer type at all - any service is only interested in the messages it sends and receives and looks after only its own house. Consumers of a service see its contract metadata and can decide for themselves whether that service meets their needs. 

Now a service itself often has little choice in the MEPs it chooses to implement. These MEPs usually come from business processes that the service hosts. If your customers are anything like mine then they are learning to become comfortable with the fact that services are a technical mechanism for hosting business processes (or parts of processes). The MEPs that a service supports then naturally stem from the logical exchange of documents from that process, not from some arbitrary technical aspect like "request response" or "pub-sub". WSDL is somewhat to blame here since it corrals developers into a very operation rather than workflow centric viewpoint. I'd suggest taking a look at SSDL (www.ssdl.org) for a contract language that is a better fit for process automation. 

I guess my take on an ESB complicating matters stems from several issues. Firstly there is the obvious one where, without even knowing what the customers' problems are, ESBs are touted as the solution (Ronan touches on our opposing views here elsewhere in the comments for this entry). Not so. ESBs are not always necessary or desirable for delivering service-oriented systems. In fact something like Web Services may be preferable in most cases because they allow us to build out our integration fabric incrementally driven by business needs, rather than trying to second-guess ourselves and install the "integration server" up front. Better that our integration is decentralised so that it can evolve over time as needs change without being overly disruptive (loose coupling again).  

Secondly there's the matter of governance. Now that I have the possibility of putting logic on the network (transforms, orchestrations, business rules, etc) it becomes very difficult to actually ascertain at any given point what my systems are doing. The conduits between my services are no longer transparent but instead the bus becomes an active point in my architecture. For end-to-end processes this is a significant challenge because the owners of the bus are free to do work within that domain which may impact the processes that I have running across the bus. This is a case of tight coupling since a change in the middle can break applications which communicate via that mechanism. This to me seems more heavyweight and complicated than simply allowing services to exchange messages. 

However there may yet a place for ESBs in most large enterprises, but there are (at least) two caveats: 

1. The ESB is a toolkit whose constituent components are used behind service endpoints only; 

2. The ESB is not the first thing that gets bought when a project starts. Only if the need for an ESB toolkit becomes apparent then the purchasing decision can be made. Don't make expensive purchasing decisions up front - that way we avoid guessing wrongly and paying the penalty financially and technologically. 

Jim

The only thing I do not like about ESBs is the Service Bus concept. They can perform much useful tasks in an SOA: transform, route, compose, script, queue, publish/subscribe. What they should *not* do is to try to become the center of any SOA, but just integrate with it as any other service. 

ESBs try to limit with the complexity of the distributed environment of an SOA by putting a centralized behemoth into it. But this does not scale: as the SOA grows you have to feed the ESB with CPUs for everything to work, and if it fails you're dead. The solution is distribution, not centralization. 

To avoid hardwired point-to-point the solution is not an ESB, but dynamic URL resolution by means of UDDI (why is it not used in this way?), or distributed agents like Amberpoint and the like. Interface-oriented design (instead of service-oriented design) also helps: you do not mind who implements the interface as long as it does it well, so you can redefine services easily. 

More about this in http://javicamara.blogspot.com/2005/12/will-esb-kill-uddi-star.html

I have been an advocate of the ESB strategy within the context of SOA for some very specific type of message handling that will take you back to the days of Message Brokering. In an SOA there are normally two parties defined in the interaction (requestor and provider) which goes back to the point-to-point concept and precludes the engagement/involvement of "interested parties" to the business event that has occurred. Enter the MB or ESB which will, based on message type, be able to inform multiple of other interested parties in the binary interaction of an SOA two party event. This in my estimation is crucial capability as I would not want to develop into a service the smarts to know who all the interested parties are to a business transaction and would rather that the event outcome be distributed to those interested by a third party resource, namely, the event broker or ESB. We should not overlook the provision of this brokering capability as a "service" endpoint within an SOA. This is not to say that perhaps the vendores are overloading the value prop of an ESB but it is up to the market to discern which capabilties make sensee and to determine the true value of the ESB. My complements to many of the very sound and wise comments that I have read here.