August 25, 2005

VS2005 Load Testing

Ok, so the ACT testing didn't go so great. Basically the Testing window was 4 hours (from 3p.m. to 7p.m.) and there were some significant responsivity issues I saw during that period. Anecdotally we saw a huge improvement after changing (a seemingly un-related) registry setting regarding the Active Directory implementation, but who knows? (we didn't re-test...)

I've been asked to look at the performance for another client and have spent quite some time with the VS2005 Beta 2 and getting a series of tests automated. It is a huge step up from the ACT tool although grappling with the most beta aspects is still a challenge.

Things I really like:
  • The web test recorder works with HTTP 1.1, SSL and NTLM etc...
  • The automatic handling of VIEWSTATE (thankyou, thankyou, thankyou)
  • Out of the box extraction / validation rules and their extensibility (very powerful).
  • Working in an Intellisense IDE.
  • Visual representation of the requests, querystrings, form posts etc
I'll just quickly list some of the areas I've had some pain - if there is anyone out there who needs more info about any of these, i'm happy to share....
  1. XML data binding - Nope not supported just yet as far as I can tell - use an SQL table if it's required.
  2. Web /Load Test renaming and moving - also works inconsistently. I get warnings and misnamed files quite regularly when doing these prety basic operations. I've learned to be careful about this and usually any problems are obvious immediately. I just kill the copy and re-try until the tool gets it right (which is does eventually).
  3. Load Test Results - Don't seem to be saved in the same way as other test type results are. When I end a test session and shutdown VS2005, the result view basically disappears. I suspect that there is something more I could do if we had the ability to publish the results to the Foundation Reporting service, but I'm not sure. The backend data can be pulled from the DB still though, but the front end presentation is so good. What I'm intending to do to work-around this is run the tests in VS2005 but have perf mon record the counters separately.
  4. Test Result DB connectivity - This required some in-depth research to get running. There is information out there around this ( but the conection string dialog is still flakey. I did have to re-install the tool to get back at the appropriate setting when I wanted to re-set it.
  5. Random crashes - these happen a lot with test creation and de-bugging -although I must admit that I haven't lost data as a result, and I haven't had to abort a load test run in this manner either....When running web tests it is common to get the system "freezing".
  6. The API is largely undocumented as yet- a pain if you want to do anything beyond what MSFT have provided for.