December 23, 2008

A new virtual horizon

James Whittaker points to virtualisation one of the key things that will emerge in the future of testing and in my mind the Hyper-V revolution is definitely shaping up. At work here, we're already planning on ways to cope with private and shared VM "whole deployment" test environments as we iterate through the development cycles. Perhaps this is old-ish news to teams in product-focussed businesses. For Intergen as an IT service provider working in this space, with our small (days to weeks) test windows and many concurrent projects, this is huge.

I am also about to start execution of performance tests against a P to V (production to virtualised) environment - I remember talking to a client 18 months ago about how performance testing against a Virtualised environment should be treated as purely indicative, as the capability of VM's under load just didn't really approach that of real servers. Perhaps this is still true under stress, but the difference here is that we're talking about achieving acceptable "production-ready" response, and throughput, with only the CPU utilisation explicitly excluded from the set of acceptance criteria we'd use against the base metal. Funny how quickly things change. In any case, no doubt we'll learn something new.

I have a weather eye on the emergence of cloud computing Windows Azure / Amazon's EC3. Presuming that the application scales at all, I predict that these services will make it possible to scale out an app even if it is inefficient, just by buying more capacity. While you could worry that it threatens the 'Performance testing' discipline in some ways, I predict that it will make the justification for measurement and tuning more direct.

In the old days if it your app works acceptably well on the machines you already had (with some headspace) that was all you cared about. With the cloud, however, there is no fixed asset, so any performance improvements will save you operational cost pretty much immediately, and there will probably need to be a more regular assessment of your needs, especially if you have variability in your weekly / monthly / yearly demand.

In any case, this whole area is going to be new frontier, with a myriad of concerns not only technically but morally as well (e.g. legal jurisdiction, security/privacy). Exciting!

Feel free to share your thoughts on how VM technology is going to affect the tester.

No comments: