My blog has moved and can now be found at

No action is needed on your part if you are already subscribed to this blog via e-mail or its syndication feed.

Sunday, February 11, 2007
« SOA Magazine February 2007 Issue | Main | Keys to Successful Governance with SOA »

There is currently a war of words going on regarding the performance of some of the Java web service stacks including Axis2, XFire and JAX-WS 2.1 FCS. 

Instinctively, I think that this type of testing is asking the wrong questions and I am trying to articulate why that is so.

To start with, these steps seem to completely sidestep any of the design considerations that are associated with the development of any serious enterprise class web service. Those design considerations [Microsoft PAG: Improving Web Services Performance] include:

  • Design chunky interfaces to reduce round trips.
  • Prefer message-based programming over RPC style.
  • Use literal message encoding for parameter formatting.
  • Prefer primitive types for Web services parameters.
  • Avoid maintaining server state between calls.
  • Consider input validation for costly Web methods.
  • Consider your approach to caching.
  • Consider approaches for bulk data transfer and attachments.
  • Avoid calling local Web services.

Secondly, this type of benchmarking tends to focus people on the immediacy and synchronous nature of web services rather than designing the system for asynchronous operation.  In a real production system, all too often the chunk of time that is taken up by the processing associated with the business logic that the web service is fronting may be a significant factor in the performance overhead associated with the web service. And as you go through your perf optimization, it may very well make sense to optimize that piece e.g. database calls (as that gets you the biggest bang for the buck) as you do your end to end performance engineering.

Lastly, this all seems to be about the relative performance of the various data binding frameworks that are out there (ADB, JiBX, XMLBeans etc.. etc..) which in turn brings up all the nasty interoperability issues related to serialization/de-serialization that deal with the impendence mismatch between XML Schema and the language of your choice (Java, C#, ...).  This is something that I have had a great deal of interest in, especially in trying to find work arounds to ensure interoperability. But more and more, I am becoming frustrated by this particular aspect of web services and am moving more towards avoiding serialization entirely and processing a message directly. This would also allow me to utilize some of the more powerful capabilities like XSLT/XPath etc.

Of course, this also moves me away from the web services mainstream and the "ease of use" argument that can be made due to the tooling support for XML to Object Mappings by the various vendors.  One of the things on my list of near term to-do's is to explore how hard/easy it would be to go down this path using some of the modern web service stacks such as WCF and Axis2. I really think that in the long term, it would be much more beneficial to me to go down this path and will more than likely also help out as I try to come up to speed on REST (Another one of my to-do's).

2/11/2007 11:59 AM Eastern Standard Time  |  Comments [1]  |  Disclaimer  |  Permalink   
Monday, February 12, 2007 8:29:01 AM (Eastern Standard Time, UTC-05:00)

Despite being one of the perpertrators of this performance testing, I do agree. A good analogy is comparing two cars based on their 0-60 performance. It says very little about how long its going to take you to get to Glasgow on a rainy day with two kids in the back and plenty of traffic!

On the other hand people see these measurements as helping understand the maximum theoretical performance (of their car, or WS stack).

As regards the pure XML option, I think that is a very interesting question. For example, many messaging systems quote performance in TPS, but in those cases there is no data-binding. Its simply the maximum number of messages that can go through a system.

You might like to know that Axis2 was designed very much with the raw XML messaging model in mind. I often write raw XML clients and services. If you know XML well it actually makes it much simpler than the WSDL/databinding approach.
Comments are closed.