September 6th, 2007
Study: Good SOA testing needs a lot of sign-offs
Last month, I talked about Aberdeen Group’s latest survey that contrasted the SOA practices of “best-in-class” companies versus those of the “laggards.” In that survey, it was found that the top companies were seeing greater payback and customer satisfaction as a result of their efforts because they were “enterprise-aware.”
Now, in a related study, Aberdeen concludes that effective testing and QA also depends on this enterprise-awareness. A new study of 240 end-users found that so-called “best-in-class” companies are taking a multi-pronged approach to the SOA testing and QA problem by incorporating automation in the testing lab and process change at the organizational level. Best-in-class companies also tend to promote the involvement of business users in all phases of the development lifecycle. The key is being able to look at multiple business processes during the testing/QA process, because that’s how SOA really works.
Because it’s more of a mesh than point-to-point interactions, SOA involves a geometrically growing set of application-to-application and user-to-application interactions that are really tough to simulate, especially with more manual approaches. Plus, it involves applications out of your domain, so others need to be involved.
How can a company evolve into a best-in-class organization when it comes to testing and QA? The leading companies “have attacked the problem from multiple angles,” Aberdeen notes. “It isn’t enough to just deploy automation, and it isn’t enough to simply rely on functional tests QA for composite applications needs a horizontal, process-oriented view, not the vertical unit-test methods used in the past.”
Aberdeen notes that the typical best-in-class company gets business users involved in all aspects of quality.uses automation to increase test coverage, and sees quality as more than just an end-of-the-lifecycle task.
As a result of these best practices, 61% of the best-in-class companies saw a reduction in the number of defects discovered in production. By comparison, only 18% of the “laggard” companies could say they were able to reduce defects. Overall, 94% of the best-in-class companies reported an increase in the quality of deployed software, versus 17% of the laggards.
Best-in-class companies “are almost three times as likely as industry average and laggard organizations to have redesigned their entire testing process,” the report continues. “The best-in-class organizations have made the commitment to services-based applications and have retooled their key processes to reflect their new direction.”
Now, the rub: Everybody in the enterprise needs to get involved and sign off on the process. Of course, getting management and the enterprise on board is 98% of the challenge, and this is where probably even the best-in-class companies have tripped up more than their fair share of times. The report stipulates that “the enterprise as a whole must be willing to set aside the time for key players to be involved in projects as they move from phase to phase, and to supply them with the appropriate tools necessary.”
Joe McKendrick is an author and consultant with deep knowledge and insights regarding trends and developments in the technology industry. See his full profile and disclosure of his industry affiliations.
Subscribe to Service Oriented via Email alerts or RSS.
