Tuesday, March 9, 2010

Why Web Services Failed at SOA (and why REST may fail too!)

First, let us review what is the concept behind services, with particular mention of the web services themselves.

A service is a business functionality exposed by a simple interface. Note that it is not an object, nor a method, nor a module, not a procedure. It is pure business functionality.

Services are consumed, not called, nor executed nor invoked. We define the service consumption as the interaction between agents, a client and a provider, which will communicate using a messaging system, where the data unit is a document. Consumption may be local or remote (meaning a service may not require remote communication).

A service may live in the web, where they are modeled as resources, identified by a URI. If not living in web, as suitable endpoint implementation should be provided.

A service is described at business level with a contract, where a glossary of the terms is agreed upon, and the semantics of all business concepts and processes are clearly stated and known by interacting parts (this is important). The service implementation is defined by the service name, its contract, an endpoint (unique entry point/port) where to send messages, the operations the service offers, the message interaction flow, the message composition (including the document definition) and optionally a binding information.

Nice definition. What is wrong with it?. Not much, actually. Just that when Web Services Architecture group decided to make those concepts into a standard, they did a great job, but also committed a couple of fatal mistakes.

The first one was the actual selection of SOAP as the messaging protocol. Yes, SOAP is a big mistake, not because it is SOAP, but because of what it was created for. You can read here a very complete explanation of the SOAP origins by one of the people that worked on it, Don Box. There, we can understand the main problem to solve was creating a protocol to better support RPC (actually RMI) in a object distributed system.

Yes, SOAP (Simple Object Access Protocol) was not created to support services documents and messaging. It was created to provide a technology neutral data/class definition system, plus supporting remote method access (that actually meant Remote Method Invocation).

So, why is this a mistake? Well, unless your idea of services was, from the beginning, that of distributed objects and the exposition of their methods, SOAP was no fit for a Services architecture. Why is SOAP there, not sure, but my common sense tells me it was not its place. If the idea was to support document oriented messaging, an RPC protocol was the weirdest decision I can think of. You will need to tweak the protocol to support what it was not supporting before. The only reusable aspect is that it was meant for HTTP transportation, which by the way is not a constrain of the WSA (it is the only implementation though).

History tells that soon after, SOAP was modified to allow document style definition, relaxing the literal RPC format. The SOAP acronym meaning was actually changed! So it is not an object access oriented protocol anymore, but a service oriented one. That should prevent people using it for RPC, was the hope.

Ok, if that was fixed (SOAP adjusted), we may have no more problems with that, right?
What is the second mistake?

Well, a subtle one: they kept the RPC idea in SOAP. Backwards compatibility? Not sure, but the RPC literal was there, in SOAP. Worse, the WSDL included that option and WSA did not discouraged the practice! I may think it was because there were tools supporting this and the idea was not banning anyone from using them. So, people started (or continued) using RPC and calling them services!
Is that wrong? Well, a little:
a. The problem is RPC is not Services,
b. RPC does not share the service metaphor,
c, RPC has different implementation side effects,
d. RPC usually forces fine grain while Services try the contrary
e. RPC couples (per method name, parameters and interaction)
f. RPC is in the IT domain, while services should be in the business one (this is a big one).
g. RPC usually works in the distributed environment, but services not are always distributed.
h. RPC forces request-response flow, while services may have any combination of flows.
i. RPC is usually blocking, synchronous. Services are asynchronous per messaging basic concepts.
j. RPC parameters usually require serialization of objects (to be passed as parameters). Services do not even know what objects are nor the parameter concept, they deal with plain messages. 
k. In HTTP, as a particular implementation, RPCs mapping of concepts adds overhead to the call, while Services uses HTTP just as a transport and should not add more overhead.

Ok but, what about that? In case the RPC was kept due to tools, the existing tools at that time may be kept as RPC tools, and new tools to work with services and their concepts should had been created. Easy solution.

But that didn’t happen.

Actually, the tools took over and started to fight back! There was almost no support for document style. There was an RPC style non encoded. There was no standard way to represent a document (that you can feed the service call with).  Yes, the tools claim they supported document style. But when I did some research projects to compare the tools, I found the services created using document styles didn’t work at all! I created a service, then I created a client using the same tool, and the client failed talking to the service! I made it work by manually structuring the XML document to have the first element named as the method that implemented the service! It was RPC all along, and since nobody complained, I assume no one was expecting to work document style.  Well, there were a few.

But the worst comes afterward, with the creation of the Wrapped and Bare modifiers. They are no were in the WSDL nor the WSA. The wrapped is a way to use document style, but doing what I did manually in the lines above: formatting the XML to reassemble an RPC format message! You know what, testing the BARE modifier yield what I suspected: didn’t work.

That means the tools and server vendors are still selling RPC, creating RPC “services” and RPC clients. The ones that supported the document style, did a great job making people stay away from it, like Axis, whose example of RPC is a couple of lines long, but the same example using document style is gross and long, not even I could read.

Ok, granted, the web services implementation are just RPC in disguise. So what? We’ve been using RPC all our lives and we are happy with it, right?

Well, SOA is a style that looks forward to have its structure and components made out of business functionality. Under that assumption, we can build lots of additional features, like business composition (business processes), orchestration, governance, etc. But, for that we need services as business functionality, decoupled by an uniform interface. RPC adds coupling to the mix, since we are adding the object or library concepts (for many, those two are the same!) , the method invocation interaction, the data coupling in the arguments lists, and the stiffness of non-variable method definitions  (just change one argument and all breaks!).

Point (f) above is important. All other features of SOA are built on top of the concept of a business functionality exposed as a service. Governance, composability, visualization, all those work with services, not with function calls. So, as mentioned in point f, what surfaces is IT, not business, and thus the features may not work as expected.

Add to the mix that, to actually create the services, you must be a programmer. Since they reflect the RMI concept, we assume distributions, method and parameter passing knowledge, etc. Even more, the actual code is converted into exposed services, bottom up approach. This will create services shaped as the actual legacy implementation, which may not match the actual business at hand. The top down approach, which should start with the definition of the service at the business domain level, is discouraged. Tools offer not help there (or very little), and only a few architects follow that path. So, the RPC influence is breaking the possibilities of actually taking advantage of the Services metaphor.

Can I say that the whole failure of SOA as a agent of change, an encourager of business over IT in systems, a provider of business valuable concepts like business processes, governance and product composing, is just the intrusion of RPC in the whole dance? May be, but there are other causes as well. Still, SOA may not raise from he tomb if no paradigm shift is achieved.

Wait a minute. REST is in the title, it is another problem in your list? Or is it just the enemy of SOA?

Well, not really. REST is another architectural Style. It was made for a very different problem and I can assure you it is not the replacement of SOA.

It is in the title because of two reasons:
1. People think the problem of SOA were the Web Services using SOAP,
2. and also people believe REST is a simple way of creating the next generation of services.

So, they are adding another point of failure to the new SOA by trying to construct SOA using what they call “RESTFull Services“.

Why is it a fail point? I’ve heard REST services rock!

Well, first I own REST an apology. You see, actually REST has nothing to do with this. The problem is with the wrongly named REST services, which are no more than an API based on HTTP. Now, there could be REST services, but they are much more complex that the ones most people use. And bigger. So, let’s simply call those “Not-So-REST Services” as Web API Services, (WAS? Nice acronym, a provider for one of those would be a WASP!).

Well, our WAS are, as usual, more oriented to functional calls (yes, RPC). There could be a way the WAS achieves a truly service meaning, when using HTTP as the messaging transport, and the Hypermedia as the documents. Yep, it is a nice match. But that requires people thinking on documents and interactions through messages, and they are not doing that. If you check, it is the same old RPC story again.

So, what to do then? Are we doomed, I mean, is SOA doomed?

Not really. We need to understand what a service is about, and provide a suitable implementation that allows developers to work with it without mapping. I mean, why does a service need to implemented as an object method call? It breaks the metaphor and thus breaks design and implementation, making it flawed.
I’m working on a DOSE (Document Oriented Service Engine) implementation in Java, WAS and WSA oriented (WSDL 2.0, that is), with no RPC. The programming model should be quite different that the proxy and RPC model, plus a set of articles to show how design using document orientation for services should be done.

So, stay tuned.