World Wide Webber

The First International Workshop on RESTful Design, the (inflammatory-titled) WS-REST will be running alongside the WWW2010 conference in Raleigh, North Carolina on 26th April 2010. As a member of the program committee, I'd like to draw your attention to the Call For Papers and the Twitter stream for the event, and encourage you to submit something.

Guilherme has written an interesting and thought-provoking post on the RESTEasy framework, comparing it to his own Restfulie framework. Overall I think the theme of his analysis is accurate: RESTEasy doesn’t provide support for hypermedia out of the box, and that means it’s anything but RESTful.

But I don’t believe this is RESTEasy’s fault. In fact in the bizzaro-world that RESTEasy inhabits, I imagine it to be an exemplary citizen. Caught between the enterprise and the JCP, what choice would a sensible Java vendor have, other than to follow (and influence) the appropriate JSR? So it’s not RESTEasy (or Jersey, or CXF) that are at fault here, but the committee-driven process where folks with different goals and levels of understanding about REST have convened to drive out a lowest common denominator called JAX-RS (a poor name since at best JAX-RS provides a programmatic layer over a Web server).

So I won’t put the boot in to RESTEasy (or its brethren). They are what they are, and it’s too late now to worry about it. What Guilherme has shown though is just how fast and powerful innovation can be when it’s freed from committee constraints, and left in the hands of folks who are knowledgeable about the domain and passionate about their community. Restfulie (for both Ruby and Java) may well be early in their development, but they’ve already eclipsed JAX-RS (and WCF too) in terms of trying to provide support for RESTful services by embracing hypermedia.

I hope Guilherme will be offered a seat on the JSR for JAX-RS 2.0 so that it can earn the “R” in its acronym. And if a mature Ruby version of Restfulie makes it into Rails at some point, that would be nice too.

The slides for the REST tutorial that Ian Robinson and I gave are now online. Enjoy.

Guilherme Silveria and Cauê Guerra have been busy. Based on some of the ideas in the book Ian, Savas, and I are writing (which focuses on Java and .NET examples), they've written a really neat framework called Restfulie which brings hypermedia front-and-centre for both Rails services and their consumers (which is normally the hard bit).

This is brilliant stuff guys, well done!

I'm super-happy to be speaking with Ian Robinson at QCon San Francisco this year. We're going to be doing a full day tutorial  based on our book (tentatively titled "REST in Practice" that we're writing with Savas Parastatidis). The tutorial will cover Web Architecture, CRUD Services, Hypermedia and REST, Atom and AtomPub, scalability, and security issues in building distributed systems on the Web.

The tutorial runs from 09:00 to 16:00 with plenty of REST breaks on Tuesday 17th November. See you there.

I’m going to be giving briefings in London and Manchester as part of ThougthWorks’ continuing series. This time around I’m going to talk about Web-based SOA with a talk called "The Enterprise Architecture you've always dreamed of has been hiding in plain sight since 1991."

I’ll be in Oxford Street, Manchester on the 7th December for an evening briefing including a bite to eat and a few drinks, and in London on the 9th December for a breakfast briefing (almost certainly less drinking involved) in the Liverpool Street area.

You can sign up for either session now at the ThoughtWorks web site. See you there.

John Moe has written up a taxonomy of various SOA styles including Guerrilla SOA. I think John's is a useful taxonomy, I completely agree that Guerrilla SOA is preferable to Gorilla SOA. Following on from John's essay, Dave Chappell argues on the SOA mailing list that much of the cost in an SOA project stems not from middleware, but from people. And that actually SOA without middleware leads to reinventing wheels which is then left behind by Guerrilla consultants.

I'll take that as a nod in my direction since last week Dave and I participated on a panel at the SOA symposium in which similar language was used. Let's start by breaking the argument down into its two constituent parts:

1. Cost of middleware versus cost of project

On the cost front, I can provide some anecdotal evidence based on recent experience. I was part of a team approached by a telecomms service provider. These folks are all about carrier grade service, and massive throughput - a challenging and frankly daunting area of computing.

Prior to our first project starting, that client had already undertaken some analysis of their future architecture (which needs scalability of 1 billion transactions per month) using a blue-chip consultancy. The conclusion from that consultancy was to deploy a bus to patch together the existing systems, and everything else would then come together.

The upfront cost of the middleware was around £10 million. Not big money in the grand scheme of things, but this £10 million didn't provide a working solution, it was just the first step in the process that would some day, perhaps, deliver value back to the business, with little empirical data to back up that assertion.

Understandably, this pitch fell flat. £10 million spent and no change in production systems was simply unacceptable. So instead the chief techies in the organisation reached out to my crew.

My (small) team spent a couple of weeks working through some analysis of the problem, including some simple mathematical models of how the problem domain scales over time, and how the solution would have to compensate. We took the time to understand how to incrementally alter the enterprise architecture to release value early, and we proposed doing this using commodity HTTP servers at £0 cost for middleware. Importantly we backed up our architectural approach with numbers: we measured the throughput and latency characteristics of a representative spike (a piece of code used to answer a question) through our high level design, and showed that both HTTP and our chosen Web server were suitable for the volumes of traffic that the system would have to support.

Armed with that empirical data, we went ahead and built the first service in the ecosystem. It took us in total around four months with a modestly sized team (less than 10 folks, day-to-day), and as we delivered software we tested it heavily against both functional and performance requirements.

I'll repeat that, because it's important: we performance tested the solution every single day to ensure that we would always be able to meet the SLAs imposed on us by the business. We were able to do that because we were not tightly coupled to some overarching middleware, and as a consequence we delivered our first service quickly and had great confidence in its ability to handle large loads. With middleware in the mix, we wouldn't have been so successful at rapidly validating our service's performance. Our performance testing would have been hampered by intricate installations, licensing, ops and admin, difficulties in starting from a clean state, to name but a few issues.

Based on the success of the first service, we  began work on the next project. Since we'd already proven the technical patterns in terms of scalability and reliability, the green light for the next service was immediate. For the next service, the problem domain was slightly more intricate, and the team slightly larger, and the software took a 5 months to go from nothing to production (that is: serving customers and earning money). Again we were able to continuously build and performance test the system, and show it running - weekly - to our stakeholders. Nothing was invisible because there was no bus which could hide problems.

When this service went into production, it immediately started to generate revenue. The combination of the two services we deployed incrementally took strain off other parts of the system, and allowed uptime to soar. The last I heard a few weeks back, the system as a whole was dealing with several hundred percent more transactions per second than before we started.

But what's particularly interesting, coming back to the cost of people versus cost of middleware argument, is this: we spent nothing on middleware. Instead we spent around £1 million on people, which compares favourably to the £10 million up front gamble originally proposed. To be explicit:

Total cost of working software in production: £0 (middleware) + £1,000,000 (staff) = £1,000,000
Total cost of middleware approach: £10,000,000 (middleware) + £? (staff)  = > £10,000,000

If Dave's point is accurate, that people cost more than software, then the minimum cost for the middleware solution would be £20,000,001, or around 20x the cost of the Guerrilla approach. And that doesn't even account for the revenue lost (or risk) with big-bang deployments as opposed to rapid release models. But I'm nice like that, so I'll let it slide - 20x is already a substantial enough differential.

2. Frameworks versus reinvention

In terms of frameworks, well I'd say I'm a big fan. I like WCF and Jersey, I like MVC and Spring, I like Jetty and Tomcat, I like the .NET framework and the JVM. In fact every piece of software I have ever delivered into production has relied on frameworks for heavy lifting - we used Jetty and Tomcat/JBoss and a bunch of mature open source libraries (thanks Apache!) for the services I described above.

What's different about the frameworks I favour, is that they don't imply a large up-front commitment (or gamble as it should be called), but instead allow choices to be deferred until a more responsible moment. Using these frameworks also allows delivery of tested and, importantly, testable code (thanks to Michael Feathers for educating us all about this!). And this is important because code which isn't tested is legacy, and legacy is risk.

The teams at the telco services company are still working in the same Guerrilla SOA way, underpinned with agile practices. And they are still delivering high-quality solutions into production, and generating revenue for their business. There's no "magic" in the system - no proprietary middleware that only some arcane specialist can deal with. They just have code and commodity frameworks which allows them to keep on delivering value to their customers.

Guerrilla SOA doesn't reinvent frameworks, it embraces them. But it only embraces those frameworks that accelerate delivery, and not those which require up-front gambles that are detached from business imperatives. The Guerrilla Consultant is all about commoditisation: using commodity/open frameworks and commodity skillsets (.NET, Java, Ruby versus specialists for products like Oracle Fusion), and all about deploying rapidly and repeatably onto commodity hardware.

It is anything but the trench warfare approach espoused by enterprise middleware.

Only 2.5 years after this .NET developer moved to London, I’ll be attending (and presenting) at my first London .NET user group. In tomorrow’s talk (registration required), I’ll be talking about hypermedia formats and protocols and showing the neat framework Savas wrote for building hypermedia services in .NET.

Hope to see you there, and for a couple of cheeky pints afterwards.

The mulit-talented Guilherme Silveira with Adriano Almeida and Lucas Cavalcanti, has been coding up a storm on the RESTful services front. Based on the Restbucks ordering example from the book Ian, Savas, and I are writing, he’s written a sample hypermedia service that exposes the ordering protocol.

More importantly, they’ve written up a generic client that can be used to explore that protocol. They’re hosting the demo service on GAE, and have released their code for all to enjoy on GitHub. Fabulous work guys, and very timely too.

Now, back writing that book before someone else writes an entire framework around those ideas :-)

The SOA Manifesto (aka Snake Oil Agreement, or SOA) has been published, attempting to set out a value system for SOA deployments modeled on the famous (and useful) Agile manifesto. Like the Agile manifesto, the SOA version was written by a team of people some of whom are luminaries and personally people I respect hugely. Unlike the Agile manifesto however, the SOA manifesto is nothing more than snake oil to give the vendor community a thin veneer of respectability atop their increasing bureaucracy, deteriorating levels of innovation, and increasingly painful pricing models.

But don't take my word for it, let's walk through what the Snake Oil Association wants us to believe. Let's start with a definition of SOA:

"Service orientation is a paradigm that frames what you do. Service-oriented architecture (SOA) is a type of architecture that results from applying service orientation."

Wow, that's profound. Sitting in my study looking out of the window, I can't help but see the similarities to my own immediate environment. For example, "Comfortable office chairs are a paradigm that frames what I do (or in this case simply frames my butt). Chair oriented architecture is a type of architecture that results from applying chair-ness to seating situations."

What a crock. Those bon mots are nothing but recursive dribble which say nothing about SOA, and much about the validity of the manifesto. But it continues:

"We have been applying service orientation to help organizations consistently deliver sustainable business value, with increased agility and cost effectiveness, in line with changing business needs."

In what sense? I don't think I see too many signatories of the Snake Oil Agreement that work with organisations (though there are one or two, I'll call these "the good, but chronically optimistic, guys"). I see a lot of people who sell to organisations, but that's helping in much the same way that if you suffered an injury, I kicked you in the general direction of a hospital. I don't see much in the way empirical evidence to support this claim, whereas there was a lot of train-wreck data to support the Agile manifesto.

Then the fun really starts, where the undersigned state concoct a bunch of values which embody the goodness that is SOA.

1. Business value over technical strategy

Whose business value? The customers, or the middleware vendors? And does this mean technical strategy has less merit? I find that hard to believe when several of the signatories are prominent luminaries in technical strategy for SOA (for example ESBs).

Let's chalk that one up to typical industry newspeak, run it up the flagpole, and move on.

2. Strategic goals over project-specific benefits

That is, don't think about getting this piece of software into production and realising business value, instead chain yourself to an improbable strategic vision. This contradicts the first assertion. It's also disingenuous, because it implies that projects cannot add to a strategic vision, which is pretty dumb really. It’s really saying: we don’t trust IT staff with anything important which is rude and narrow-minded beyond belief.

3. Intrinsic interoperability over custom integration

Another disingenuous point I believe. This is telling you to buy kit for "intrinsic" interoperability rather than doing it yourself with anarchic(!) technology like the Web. If you really believe in this value, then bin your ESB and embrace a platform which has intrinsic interoperability by dint of a few commonly accepted formats and protocols. If you’re unsure: see RFC2616 and friends.

Furthermore it’s ironic that “intrinsically interoperable” systems like ESBs need development effort to get them installed and running and plumbed into their victims, which seems to me a lot like custom integration between system and ESB anyway. Whoops!

4. Shared services over specific-purpose implementations

Premature optimisation, pure and simple. Re-use is great, but I'll start with use and move on from there. The fallacy of services is that they're designed to be shared - they're not. It's the business capabilities which services implement that may or may not be shared. It's tail-wagging-dog to think that service architects should design for sharing, when its the business capabilities and processes which dictate that particular outcome.

5. Flexibility over optimization

This is one I happen to agree with. Unfortunately I don't see how the Snake Oil Association has reached this conclusion, unless the signatories honestly believe that their wares support flexibility (ha!) or fail to deliver optimally (more likely). An ESB isn't flexible, nor is it optimal.

6. Evolutionary refinement over pursuit of initial perfection

Again I agree with this principle, in fact it's the very embodiment of Guerrilla SOA. However it conflicts with the second assertion that the Snake Oil Association values strategic goals over project-specific benefits. Those project-specific benefits are the vehicle for those evolutionary refinements that we want to pursue.

Over the last year I thought I'm come to terms with dangerous SOA sentiments, I had even started to shelve some of my thoughts on ESBs thinking the world had progressed passed 20th century network topologies and sales models. The Snake Oil Agreement and the Snake Oil Association have demonstrated that my hope was premature, and that my own moral relativism has lead me to mistakenly believe that I should live and let live alongside what is plainly a retrograde model for large systems engineering.

I have now dismissed my moral relativism. No longer will I hold try to see things from the Snake Oil Association's point of view: that view is simply commercially and morally wrong. The Snake Oil Agreement has distilled everything that is wrong with enterprise software delivery, with vendor politics, and with golf-course decision makers. I say it is not a rallying cry to repeat the manifest mistakes of the past, but a focal point for a revolution in the way systems are built and evolved.

La lucha continua!

The (Second) International SOA Symposium is where all the serious SOA-types hang out for a few days, discussing the critical issues around ESB adoption and which vendor rollouts looks best on a CV or in a PowerPoint presentation. Miraculously I've been invited back to talk about the exact opposite. This time around I'll be talking up the Web, based on some recent experiences producing massively scalable systems (who'd have thunk), and about the hypermedia aspects of REST (my current favourite thing).

I'm also going to participate in a couple of panels. One is on whether Agile and SOA can work together (short answer: yes, see Guerrilla SOA); and one (that I might have to skip because of flights!) on What's new in ESBs (short answer: nothing, see Guerrilla SOA).

I'm really looking forward to this event, not the least of which is because I'll be amongst good friends (honest, I'm not enemies with everybody in the whole SOA universe) like Thomas Rischbeck, Arnaud Simon, Mark Little, and Ian Robinson.

In a warm up for this year's QCon San Francisco, I'll be giving two tutorials in Denmark around the same time as the JAOO meetups (but three weeks later than the main JAOO conference). These tutorials are a condensed version of the sessionIan Robinson and I will be giving at QCon San Francisco, and will cover the same topics, albeit in a little less depth. The first tutorial will be in Århus on the 20th October, and I'll be repeating it on the 21st October in Copenhagen.

Both tutorials will cover Web Architecture, CRUD Services, Hypermedia and REST, Atom and AtomPub, scalability, and security issues in building distributed systems on the Web.

[Update: These sessions have been postponed until 2010. We’ll figure out when based on feedback at the JAOO meetup sessions]

Once I finish my current assignment in Bangalore, I'm spending a packed week in Europe starting with the Trifork folks (the brains behind JAOO). They've organised two  meetups - one in Århus and one in Copenhagen where I'll be talking about Hypermedia in REST, and how we can use hypermedia to develop Domain Application Protocols for Web services (of the Web, rather than WS-* variety).

Did I mention these are free? Then register for Aarhus on the 19th October, or Copenhagen on the 20th October and come along.

[Update: these events aren’t part of the JAOO 2009 Conference, they’re running a few weeks later. Unfortunately I will miss JAOO again this year].

Brilliant! My friend Anna Liu (ex-Microsoft, now University of New South Wales), got Slashdotted for her work on evaluating cloud platforms. Great work Anna, ad see you at the Aussie Architecture Forum next week!

• Mainland Europe: 8:00PM
• UK is: 7:00PM
• EST in the US: 2:00PM
• PST in the US: 11:00AM

Prior to last week’s excellent Falando em Java conference in São Paulo, I spent some time with the lovely folks at Caelum. They invited me to talk at their in-house tech day where I spoke about hypermedia as a contract language for describing business protocols. After some prompting from Emil, my talk has gotten some twitter love, so it’s worth pointing out that it’s available on the big Web too.

Cue Pink Floyd ripoff, “we don’t need no static contracts… hey, WSDL, leave that Web alone”

On the 28th May (1pm EDT) I'll be delivering a new version of my RESTful Web Services tutorial as part of the new InfoQ Virtual training events.

This tutorial is based on material first given at AIIT, and then refined and compressed for QCon, and GIDS. This time around the material has been refined again to fit in with the new live-over-the-Internet format that InfoQ are pioneering.

The tutorial follows in the footsteps of the book I'm writing with Savas and Ian, and will cover:

• Lo-fi Web integration with URI tunneling and POX messaging
• Using the Web for remote CRUD operations
• Hypermedia and RESTful services
• Security
• Atom and AtomPub
• Scalability and caching

So if you want to learn about the Web as a platform for building distributed systems, sign right up...

If you’re into thinking about services, and you fancy being in lovely Prague, Czech Republic on the 5th June 2009, then you might consider submitting a paper to the Workshop on the Applications of Software Services.

This has been a public service announcement on behalf of the workshop organisers.

For a number of years Mrs World Wide Webber has been using Yahoo! for email. It wasn't an educated choice, she just needed an email address when she left university and Yahoo (and others) provided those for free.

Over time though Yahoo! has become a pain! in! the! arse! The user interface is lousy and the calendaring is laughable. Her calendars were recently moved to Google, and the only thing that staved off the death knell for Yahoo! email was that it's easily accessible from my wife's iPhone.

But finally last week things came to a head. My wife wanted two pretty reasonable things: to be able to read her email in Mail on her Mac and to use a non- @yahoo.com address for professional correspondence.

And here's where the pain begins. Yahoo! don't offer IMAP, in fact they don't offer anything other than POP which isn't going to help since my wife wants to access her mail both from a computer and from her iPhone, leaving the email stored on the server. Worse still the POP service costs money - $20 per year to be upgraded to the Yahoo! Mail! Plus! service. My wife's data had been kidnapped, and the kidnappers wanted a $20 ransom to release it!

What! a! rip! off!

But you know what? I paid it[*]. I paid it and I used the POP access to slowly but surely download each and every email my wife had stored with Yahoo! and upload them to gmail which happens to support IMAP for free.

My wife now has a real email address which she owns, that Google supports (for now) and which she can migrate as she sees fit. It works with her iPhone and she can check it on the Web when she needs to.

Good bye and good riddance Yahoo! The $20 sting in the tail to escape your clutches was extremely rude (can’t you make money any other way than taxing dissatisfied customers?), but in a year's time I will have forgotten about it, and you.

* YPOPS crashed on startup every time I tried, so no go there.

Savas is blogging about an idea we've been batting around for a little while around exposing hypermedia state outside of regular resource representations. In essence the idea is that by using the Accept header clients can ask a resource about other resources it knows about, effectively supporting the whole hypermedia as the engine of application state constraint in a generic way which isn't coupled to specific formats (e.g. Atom, XHTML). I quite like the idea, but it'd be nice to know what other folks in the community think too.