This is where the real performance testing begins – scripting, execution, re-execution and analysis of results. Generally I attempt to get into the scripting phase as soon as possible – not all phases of the Performance Testing lifecycle have to be done in strict sequence, if I know a business process has to be simulated, but aren’t yet clear on the exact workload model for that process – scripting can still begin. The Performance Tester needs to really start preliminary scripting ASAP as this gives an accurate idea of what actually needs to happen – when you start scripting you realise how little you actually know. Initial phases of scripting can be slow – the Performance Tester will discover that he can’t just load test the system – they will need to work out how to create X thousand users with their associated passwords, they may have to set up different profiles for each of the users and in some cases work out how to add dummy cash to their accounts. All of this takes time – and quite often takes a lot of co-ordination with different stakeholders within the organisation. It’s a good idea to get onto the system and start scripting in parallel to the requirements gathering stage – this allows you to ascertain exact requirements.
After the scripts are complete I split the testing into two distinct phases:
Informal Performance Testing: The act of executing the scripts, reporting technical results; flagging issues and helping development resolve them. This is the where most of the time is usually spent during a performance testing cycle. A Performance Tester will resolve:
- Environmental issues (DB, Network, OS Configs)
- Incremental builds with significant changes that require testing
- Investigation of genuine performance bottlenecks and testing of fixes
- Incorrect of poor deployment procedures
Formal Performance Testing: Formal execution of scripts against a known baseline that is intended for release.
By splitting the performance execution stage into two distinct phases I find it helps stakeholders within the overall project understand what is happening within performance testing. If this isn’t split into two phases then it could appear that the Performance Testing phase has taken weeks and is the bottleneck when the reality is very different. Issues found within the informal test phase require help from specialised skill sets that are often in demand – and the load testing issues are often hard to isolate and resolve. I’ve found that the number of issues here are directly related to the quality and maturity of the build and deployment process.
When the build, environment and scripts are complete and correct – Formal Performance Testing can begin. All planned scripts will then be executed and reported against a known software baseline that is intended for release. There should be no real surprises lurking here as these should have been uncovered and solved in the Informal Performance Test Phase. The results for this test should be used as a basis for the management summary.