Monday, November 9, 2009

Headless Performance Testing


What is headless testing? It is any testing that does not have a GUI. We are all used to creating a performance script from recording a business process. This is accomplished by having a GUI for the business process that can be recorded. But do you always have a GUI to record? No, but many performance testers just refuse to test anything that doesn't. They say that without a GUI an app is not ready to be tested. I know most developers would disagree with that statement since they have to unit test the code many times without having a GUI.
So what happens? Performance testers say they won't test without a GUI, and developers generally don't have a testable GUI until the end of their release; so that means that performance testing will be done at the end. Hang on! I know that one of the biggest complaints by performance testing teams is that they are always at the end of the cycle and they never have enough time to test. Even if they do get the testing completed, if they find problems, the app gets released anyway because there was no time to fix the issues.
One of the biggest, if not the biggest, problem that performance testers have is that they are being relegated to the end of a release; yet they are perpetuating the problem by not testing earlier drops that do not have a GUI. This seems quite strange to me.
So what can performance testers do? Start doing headless performance testing. Start testing components before a GUI exists. The earlier you are testing, the quicker the problems are found, the more likely it is that the problems will be fixed and that means higher quality releases.

How to do it? For SOA there are products that will pull methods from WSDLs, and they will help you manually create scripts for the service. If it is not a service or a WSDL is not available, you can work with the developers to create test harness that can be used to create performance scripts. Many time the developers do not have to write a new test harness because they may have already written one to unit test their code or component.
Is a test harness enough? In some cases, yes. But in most cases, you will also need to employ stubs or virtual/simulated services. Stubs are pieces of code that "stub" entities which do not exist. When you are trying to test a service, you might not have a front end (GUI) and you probably do not have a backend to the service. The service may talk to other services, servers or databases. And if those backend entities do not exist, then you have to put something in place to simulate that backend so that the service you are attempting to test will function and react properly.
I've mentioned services quite bit. These seemingly innocent components are exacerbating the need for headless testing but are also allowing it to be more feasible. Before, with siloed applications, it was a problem to only test at the end; but hey that's what happened. Now with SOA, services are being deployed and many, many applications are utilizing them. It is no longer ok just to test one app to ensure that it will work as designed. Services need to be performance tested by themselves to fully stress their anticipated load. Just because a service seems to work with one application or another, all the applications combined that are using a single service may overly stress that service and cause it to fail.
The good news is that since these services should be well defined and encapsulated, it becomes possible to properly test them with a GUI--again, either by utilizing a WSDL or a test harness from developers. Headless testing will not only help ensure proper performance of an app in production, but it also enables testing earlier in the lifecycle. Before well-defined components, testing earlier was doable but so hard to do that most just refused. SOA allows early testing to become a reality.


Performance testers need to understand this new reality. It is time that we embrace the services and not just rely on developers to test them. We have been complaining for years that we need to start testing earlier and now, that it can become a possibility, we need to jump at the opportunity.

What will happen if performance testers don't perform headless testing? Well, performance testers will start to take on a smaller and smaller role. Developers will have to fill the void and "performance test" the services. We know that QA exists because developers can't test their code, but someone will need to test the services; and if the performance testers will not, the developers will have to. The developers will then claim that, since they have tested the parts, testing the whole is not that important. I have witnessed this happening today.
Is Business Process testing and End-2-End Testing going away? Of course not. They are important, and they will always be. Being able to test earlier will just allow many more performance issues to be found and corrected before applications get released. Testing individual services is needed because, many times, services are released before applications begin using them. I don't think anyone wants any component released into production without it being properly tested.


What have I been trying to say? Performance testers need to step up and start testing individual components. It may be difficult at first because it is a new skill that needs to be learned; however, once a tester gains that expertise, they will make themselves more relevant in the entire application lifecycle, gain more credibility with developers, and assist in releasing higher quality applications. Leaving component testing to developers will eventually lead to poorer quality applications being delivered or a longer delay in releasing a quality application. I can easily say this because QA was created to prevent the low quality apps that were being delivered. If developers were great testers then QA would not exist. That being said, QA can't abdicate their responsibility and rely on developers to test.

Headless testing: learn it, love it, performance test it.

No comments:

Post a Comment