So what happens? Performance testers say they won't test without a GUI, and developers generally don't have a testable GUI until the end of their release; so that means that performance testing will be done at the end. Hang on! I know that one of the biggest complaints by performance testing teams is that they are always at the end of the cycle and they never have enough time to test. Even if they do get the testing completed, if they find problems, the app gets released anyway because there was no time to fix the issues.
One of the biggest, if not the biggest, problem that performance testers have is that they are being relegated to the end of a release; yet they are perpetuating the problem by not testing earlier drops that do not have a GUI. This seems quite strange to me.
So what can performance testers do? Start doing headless performance testing. Start testing components before a GUI exists. The earlier you are testing, the quicker the problems are found, the more likely it is that the problems will be fixed and that means higher quality releases.
Is a test harness enough? In some cases, yes. But in most cases, you will also need to employ stubs or virtual/simulated services. Stubs are pieces of code that "stub" entities which do not exist. When you are trying to test a service, you might not have a front end (GUI) and you probably do not have a backend to the service. The service may talk to other services, servers or databases. And if those backend entities do not exist, then you have to put something in place to simulate that backend so that the service you are attempting to test will function and react properly.
I've mentioned services quite bit. These seemingly innocent components are exacerbating the need for headless testing but are also allowing it to be more feasible. Before, with siloed applications, it was a problem to only test at the end; but hey that's what happened. Now with SOA, services are being deployed and many, many applications are utilizing them. It is no longer ok just to test one app to ensure that it will work as designed. Services need to be performance tested by themselves to fully stress their anticipated load. Just because a service seems to work with one application or another, all the applications combined that are using a single service may overly stress that service and cause it to fail.
The good news is that since these services should be well defined and encapsulated, it becomes possible to properly test them with a GUI--again, either by utilizing a WSDL or a test harness from developers. Headless testing will not only help ensure proper performance of an app in production, but it also enables testing earlier in the lifecycle. Before well-defined components, testing earlier was doable but so hard to do that most just refused. SOA allows early testing to become a reality.
Performance testers need to understand this new reality. It is time that we embrace the services and not just rely on developers to test them. We have been complaining for years that we need to start testing earlier and now, that it can become a possibility, we need to jump at the opportunity.
Is Business Process testing and End-2-End Testing going away? Of course not. They are important, and they will always be. Being able to test earlier will just allow many more performance issues to be found and corrected before applications get released. Testing individual services is needed because, many times, services are released before applications begin using them. I don't think anyone wants any component released into production without it being properly tested.
What have I been trying to say? Performance testers need to step up and start testing individual components. It may be difficult at first because it is a new skill that needs to be learned; however, once a tester gains that expertise, they will make themselves more relevant in the entire application lifecycle, gain more credibility with developers, and assist in releasing higher quality applications. Leaving component testing to developers will eventually lead to poorer quality applications being delivered or a longer delay in releasing a quality application. I can easily say this because QA was created to prevent the low quality apps that were being delivered. If developers were great testers then QA would not exist. That being said, QA can't abdicate their responsibility and rely on developers to test.
Headless testing: learn it, love it, performance test it.