ROI: You Get What You Pay For

We've all heard that saying. But how many times do we really follow it? We have bought, ok I have bought, cheap drills, exercise machines, furniture, only to be sorry about when they break prematurely. Or you find a great deal on shoes only to have them fall apart on you while you are in a meeting with a customer. I'm not saying that happened to me, but I know how that feels.

Cheaper always seems like it's a better deal. Of course it's not always true. I can tell you that now I pay more for my shoes and I'm much happier for it :). No more embarrassing shoe problems in front of customers (not saying that it happened to me). In fact when my latest pair of shoes had an issue, I sent them back to the dealer and they mailed me a new pair in less than a week! That's service. You get what you pay for.

The same holds true for cars, clothes, hardware, repairs, and of course software testing tools. You knew I was going to have to go there.

I hear from some people that Performance Center is too expensive. I'm always amazed when I hear that. I'm not saying Performance Center is for everyone. If you don't need PC's features and functionality, it may appear pricey. If you are only looking for a simple cell phone, then a phone that connects to the internet and to your email and also has a touch screen may seem a little pricey. But if you need those features then you are able to see the value in those features.

I can sit here, go through each unique feature in Performance Center and explain to you the value (Not saying that it will not come in a future blog :) ). But why would you listen to me, I'm the product manager. Of course I'm going to tell you that there is a lot of value to PC. Well IDC,a premier global provider of market intelligence and advisory services, has just released an ROI case study around HP Performance Center.

A global finance company specializing in real estate finance, automotive finance,
commercial finance, insurance, and online banking was able to achieve total ROI in 5.6 months. Yes! Only 5.6 months, not 1 or 2 years. But a total return on investment in 5.6 months. If anything I think we should be charging more for Performance Center :). This company did not just begin using PC, they have been using PC for the last 4 years. And during that time they have found a cumulative benefit of $24M. I'd say that got a lot more than what they were paying for. Not only did they see a 5.6 month return on the investment but they are seeing a 44% reduction in errors and a 33% reduction in downtime in production.

What gave them these fantastic numbers?

Increased Flexibility
By moving to Performance Center they were able to schedule their tests. Before PC they had controllers sitting around being idle while other controllers where in high demand based on the scenarios that were on them. But once they were able to start to schedule their tests, they began performing impromptu load tests and concurrent load tests. They started to see that they were able to run more tests with fewer controllers.

Increased Efficiency
While they are able to increase their testing output, they didn't increase their testing resources.
Their testers/engineers were able to more through PC than what they could do with any other tool.

Central Team
With the central team they were able to increase their knowledge and best practices around performance testing. By doing this along with performing more test, they were able to reduce their error rate by 44% and their production downtime by 33%.

So you get what you pay for. Put in the time and money. Get a good enterprise performance testing product. Invest in a strong central testing team. You will get more out, than what you put in. In the case of this customer they got $21M out over 4 years.

Also invest in good shoes. Cheap shoes will only cause you headaches and embarrassment (Not saying that it happened to me).

More About Performance Centers of Excellence

Earlier I wrote about the benefits of creating a performance CoE. I explained by making a move to a CoE you can increase your quality, increase your testing throughput, and decrease your overall spend. I know that this sounds too good to be true. But I'm telling you it's not. Don't just take my word on it. Theresa Lanowitz of voke, ran her own Market Snapshot on performance CoEs. Listen to her on May 19, 2009. She will go over all of her findings on CoEs. And since I'm telling you about it, the findings are good. :). After you listen to this presentation, come back here and I'll talk about the steps you can take to begin building a CoE. voke Shares Research Findings

» ROI and performance goals with a performance testing shared service Dear HP Valued Customer, In today’s economic climate organizations are seeking ways to optimize resources and results. Part of the move towards optimization is the evolution and transformation of the application lifecycle, particularly the components that drive efficiency and quality. To achieve business optimization through the application lifecycle, organizations are building centers of excellence (COE). A COE focused on performance is a strategic way to deliver customer satisfaction, high quality and resource savings. Join Theresa Lanowitz, founder of voke Inc., as she presents the findings of a recent Market Snapshot on Performance Centers of Excellence. In this presentation, we will discuss: • Building a performance COE • Achieving performance COE ROI – qualitative and quantitative • Gaining organizational maturity through a performance COE • Realizing the benefits, results, and strategic value of a performance COE Attendees will receive a copy of the voke Market SnapshotTM Report: Performance Center of Excellence. Register Now » Speakers: • Theresa Lanowitz, CEO & Founder – voke, Inc. • Priya Kothari, Product Manager, Performance Testing Solutions – HP Software + Solutions PS: HP Software Universe 2009 registration is now open! Interested in learning the very latest about IT strategy, applications and operations? Then seize this opportunity to join us June 16-18, 2009 at The Venetian Resort Hotel Casino in Las Vegas, Nevada. » Find out more.

voke Shares Research Findings DATE: Tuesday, May 19, 2009 START TIME: 10:00 am PT / 1:00 pm ET DURATION: 60 minutes with Q&A » Register Now Learn More » Cut costs (not corners) from your application lifecycle » Performance Center blog » Download a trial version of HP LoadRunner 9.5

Finding a Needle in a Haystack

The job of finding the proverbial needle in a haystack has always been a challenge. Digging through a haystack and struggling to find a needle is hard, if not near impossible. Plus you don't even know if there is a needle in there! After searching for while, I know (or at least I hope) that you'll be asking for some tools to help you out.

The first tool that I can think of is a metal detector. This makes sense. Use the metal detector to discover if there is a needle in the stack at all. If there isn't, then you don't have to waste your time looking for it. Yay!!! At least, now I know that I can quickly check stacks and only search through stacks that actually have a needle.

Is that the best or only tool that you can use? If you can cut the search time down more, wouldn't that good? Sure, unless you really like digging through that hay. What if you had a strong magnet? That would rock!!! First, use the metal detector to make sure that there is a needle; then bring in the strong magnet and voila! Out comes the needle, and you're done! No more searching required, and your job just got a whole lot easier.

As you may have already guessed, I'm not here to discuss haystacks. They are fun and all but not really the point. Let's tie this back to performance testing. That's more fun than haystacks anyway.

In the beginning, people tried to performance test manually with lots of people attacking an application at the same time. This was found to be time consuming, not repeatable, and not that beneficial, like digging through that hackstack by hand.

Then came automated load testing with monitoring. Now, there's a repeatable process and monitors to help discover if there were problems. This is a tremendous time saver and helps to ensure the quality of apps going to production--the metal detector, if you will.

Most of you know and love the metal detector, as well you should. But it is about time to be introduced to the strong magnet of performance testing. Say hello to Diagnostics (Diag). Some call it deep-dive; others call it code level profiling. I call it the time saver. Diagnostics will pinpoint (or should I say needle point :-) ) problems down to the method or sql statement level. That is huge! Now you have much more information to take back to developers. No longer just transaction level information, now you can now show them the method or sql statement that is having the problem within the transaction. This slashes the time it takes to fix problems. Diagnostics has agents on the application under test (AUT) machines. It then hooks the J2EE and/or .NET code. This is how it can show where there are bottlenecks. Highlighting slow methods and sql statements are good, but being able to tie them back to a transaction is even better.

From personal experience, I can tell you that Diagnostics works extremely well. Back at good old Mercury Interactive, we had a product that was having some performance issues. Our R&D had been spending a few weeks looking for the problem(s). Finally, I approached them and asked if they had tried our Diagnostics on it. They of course said no; otherwise this would be a counterproductive story. After I explained to them in detail, the importance and beauty of Diagnostics, they set it up. Within the first hour of using Diagnostics, they found multiple problems. The next day, all of them were fixed. R&D went from spending weeks looking for the needles that they couldn't find to finding them within an hour with the magnet I gave them. And now they use always use Diagnostics as part of their testing process.

I've heard people complain that it takes too much time to learn and set up. First off, it isn't that complicated to set up. Secondly, yes it is something new to learn but it's not too difficult. Once you learn it, however, it is knowledge that you can use over and over again. It took time to learn how to performance test to begin with. Record, parameterize, correlate, and analyze all took time to learn. This is just one more tool (or magnet) in your tool belt. It will save you and the developers tremendous amounts of time.

Isn't Diagnostics something that IT uses in production? Yes it is! Both sides (testing and production) can gain valuable information from using Diag. And both for the same reason. Find the problem and fix it fast. Testers can always run the test again to see if they can reproduce a problem. But, in production, once the issue occurs, they want to be able grab as much information as possible so that they can prevent it from happening again. After production gets this information, they can pass it back to testing to have testing reproduce the issue. If performance group is using the same Diag as production, then it is easier to compare results. A dream of production and testing working together in harmony, but I digress.

I have said for years that if performance testers are not utilizing diagnostics, then they are doing a disservice to themselves and to the task. Stop digging through the haystack. Pick up that magnet and start pulling out those needles!

Click here to learn more

Cloud Assure

HP Unveils “Cloud Assure” to Drive Business Adoption of Cloud Services
Also expands HP Software-as-a-Service partner program to help resellers better service customers

PALO ALTO, Calif., March 31, 2009 – HP today announced HP Cloud Assure, a new Software-as-a-Service (SaaS) offering designed to help businesses safely and effectively adopt cloud-based services.
HP also introduced updates to its HP SaaS reseller program that allow partners to provide added services and value to customers.
Cloud computing is a term used to describe services that can be delivered and used over the Internet through an “as-needed, pay-per-use” business model.
The promise of cloud computing is appealing because it can reduce business costs and provide greater flexibility and scalability of services throughout the enterprise. However, IT organizations unable to ensure the security, performance and availability of the cloud services they provide or consume may be putting their businesses at risk. This uncertainty can be an impediment to wider adoption of cloud services by enterprises.
“Whether providing or consuming cloud services, enterprises should be as ready to manage the risks as they are to reap the significant rewards of the cloud model,” said Frank Gens, senior vice president and chief analyst, IDC. “HP’s new offering is aimed at addressing what IDC has identified as the top three concerns of cloud computing among enterprises – security, performance and availability.”

HP Cloud Assure consists of HP services and software, including HP Application Security Center, HP Performance Center and HP Business Availability Center, and is delivered to customers via HP SaaS. HP also provides customers with a team of expert engineers that performs security scans, executes performance tests and deploys availability monitoring.
HP Cloud Assure helps customers validate:
· Security – by scanning networks, operating systems, middleware layers and web applications. It also performs automated penetration testing to identify potential vulnerabilities. This provides customers with an accurate security-risk picture of cloud services to ensure that provider and consumer data are safe from unauthorized access.
· Performance – by making sure cloud services meet end-user bandwidth and connectivity requirements and provide insight into end-user experiences. This helps validate that service-level agreements are being met and can improve service quality, end-user satisfaction and loyalty with the cloud service.
· Availability – by monitoring cloud-based applications to isolate potential problems and identify root causes with end-user environments and business processes and to analyze performance issues. This allows for increased visibility, service uptime and performance.

“HP helps Akamai validate to our enterprise prospects and customers the performance and availability improvements we provide in the cloud,” said Willie M. Tejada, vice president, Application and Site Acceleration for Akamai Technologies, a Massachusetts-based service provider for accelerating content and applications online. “HP SaaS confirms the benefits of our dynamic Web acceleration services and will be important to us as we continue to introduce additional functionality that helps businesses effectively optimize the cloud.”
HP Cloud Assure provides control over the three types of cloud service environments:
· For Infrastructure as a Service, it helps ensure sufficient bandwidth ability and validates appropriate levels of network, operating system and middleware security to prevent intrusion and denial-of-service attacks.
· For Platform as a Service, it helps ensure customers who build applications using a cloud platform are able to test and verify that they have securely and effectively built applications that can scale and meet the business needs.
· For Software as a Service, it monitors end-user service levels on the cloud applications, loads tests from a business process perspective and tests for security penetration.

“There is no question that cloud computing is providing a new set of opportunities for businesses, but it presents new risks as well,” said Scott Kupor, vice president, Software as a Service, HP. “With over nine years of SaaS experience and a leading portfolio of solutions in security, performance and availability, HP is uniquely positioned to help assure our customers can leverage the promise of cloud while removing risk from the equation.”

HP Cloud Assure is available today. More information about the offering is available at http://www.hp.com/go/cloudassure.
HP Software-as-a-Service partner program updates
Building on a reseller option introduced to partners last year, the expanded HP SaaS partner program now includes a partner-led delivery option.
The new option allows partners to provide specialized services, based on their areas of expertise, on top of the HP SaaS portfolio. The option helps partners:
· deliver services more quickly by leveraging the already-deployed HP SaaS solutions;
· focus on delivering high-value, high-margin consulting services for customers; and
· strengthen partnerships with customers through ongoing service.

“Our customers’ reduced IT resources have necessitated them to do more with less, but especially now, there is more strain to accelerate business value,” said Jeff Jamieson, co-founder and vice president of sales at Whitlock Infrastructure Solutions. “HP SaaS Partner-Led Delivery allows partners to leverage areas of expertise to deliver value-added consulting services.”
The partner-led delivery option is available currently for HP SaaS for Business Availability Center, HP SaaS for Performance Center and HP SaaS for Quality Center. Expansion to include the rest of the SaaS Business Technology Optimization software portfolio is under development.
Register to attend a live webcast
HP Cloud Assure Offering & HP SaaS Partner-Led Delivery Option
Tuesday, March 31, 2009 at 8 a.m. to 10 a.m. PT
About HP
HP, the world’s largest technology company, simplifies the technology experience for consumers and businesses with a portfolio that spans printing, personal computing, software, services and IT infrastructure. More information about HP (NYSE: HPQ) is available at http://www.hp.com/.
Note to editors: More news from HP, including links to RSS feeds, is available at http://www.hp.com/hpinfo/newsroom/.

This news release contains forward-looking statements that involve risks, uncertainties and assumptions. If such risks or uncertainties materialize or such assumptions prove incorrect, the results of HP and its consolidated subsidiaries could differ materially from those expressed or implied by such forward-looking statements and assumptions. All statements other than statements of historical fact are statements that could be deemed forward-looking statements, including but not limited to statements of the plans, strategies and objectives of management for future operations; any statements concerning expected development, performance or market share relating to products and services; any statements regarding anticipated operational and financial results; any statements of expectation or belief; and any statements of assumptions underlying any of the foregoing. Risks, uncertainties and assumptions include macroeconomic and geopolitical trends and events; the execution and performance of contracts by HP and its customers, suppliers and partners; the achievement of expected operational and financial results; and other risks that are described in HP’s Quarterly Report on Form 10-Q for the fiscal quarter ended January 31, 2009 and HP’s other filings with the Securities and Exchange Commission, including but not limited to HP’s Annual Report on Form 10-K for the fiscal year ended October 31, 2008. HP assumes no obligation and does not intend to update these forward-looking statements.


© 2009 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. The only warranties for HP products and services are set forth in the express warranty statements accompanying such products and services. Nothing herein should be construed as constituting an additional warranty. HP shall not be liable for technical or editorial errors or omissions contained herein.

News releaseEditorial contacts: Scott Pace, HP+1 650 534 7439 scott.pace@hp.com Erin MuhlhanBurson-Marsteller for HP+1 312 596 3529 erin.muhlhan@bm.com HP Media Hotline+1 866 266 7272 pr@hp.comwww.hp.com/go/newsroomHewlett-Packard Company3000 Hanover StreetPalo Alto, CA 94304www.hp.com

Application Lifecycle Management Webcast

Next week Paul Ashwood and Bradd Hipps will be giving a webcast over on theserverside.com, presenting a whitepaper and case study Application Lifecycle Management (ALM). There will be some interesting details on how testing can be integrated into the lifecycle for application development and delivery to production. If your company or testing organization includes XXXXXXXXXXXXXXXXXX as part of the SDLC or Lifecycle processes, this might be very useful information and I'm sure Brad would love to hear some questions from you.

Here's the official abstract:

"Effective companies are riding the latest waves in application modernization. These waves touch nearly all of IT from technology and staffing to application architectures and release strategies.

HP Application Lifecycle Management (ALM) solutions help your IT organization make the most of these trends and avoid being swamped in the process. ALM from HP is an integrated suite of leading solutions that enables your IT leaders to answer comprehensively the key questions business stakeholders have regarding application modernization.

You are cordially invited to view HPs unique perspective on what ALM is and how our solution renders better business outcomes. All attendees will receive our new white paper, “Redefining the application lifecycle: Looking beyond the application to align with business goals.”

Click here to register

PC 9.5 Webinare March 16, 2009 with Nationwide

Webinar with Nationwide

HP Software recently released the 9.5 versions of HP LoadRunner software and HP Performance Center software. With these new releases, HP is addressing today's top of mind application challenges around rapid technology change and adoption of new processes. In this challenging economic environment, companies have to be as lean and agile as possible.



Please join HP and Nationwide for an informative Webinar where you will hear the highlights of the latest release as well as get a preview of an actual implementation of HP Performance Center 9.5 from John Seling of Nationwide.



You will hear:


-What new capabilities are included in the 9.5 release

-How Nationwide leveraged the new features in 9.5 to shorten their test timeframes and make their tests more realistic

-More about the new integration with Shunra Software


Join us to find out how HP's performance validation solutions can enable you to achieve legendary QA projects. All attendees will receive our new white paper, "Innovations in enterprise-scale requirements management, quality planning, and performance testing".


Register Now »


DATE: March 17, 2008

TIME: 10:00 a.m. PT / 1:00 p.m. ET

SPEAKERS:
John Seling, Performance & Data Engineering Manager, Nationwide

Priya Kothari, Product Marketing Manager (LoadRunner & Performance Center), HP Software


DURATION: 60 minutes with Q&A

REGISTER: Click here to register for the webinar

Offshoring / Outsourcing Performance Testing

There are 2 main reasons why companies utilize offshoring (outsourcing) for performance testing.

The main reason is cost savings. Obviously companies will try to choose locations where there is a lower cost of doing business. The 2nd reason is to be able to ramp up new testers quickly. If there is a greater demand for testing than what the current set of testers can handle then offshoring or outsourcing can be utilized to quickly gain more testers to help with the excess demand.

In an ideal world all performance testers are the same. If you can find cheaper testers elsewhere, then you will get immediate cost savings. But as we know we do not live in an ideal world. There are different levels of knowledge, skill and motivation. We have seen time and time again offshoring fail because companies do not have that correct expectations, they do not set up the proper training, and they do not have the correct set of tools.

You cannot assume (we all know what that does) that just contracting with a secondary company to provide all or partial performance testing will automatically start showing benefits.

There is no reason why offshoring cannot be a successful venture for companies. They must research the offshoring options and find ones that have a good fit with skill sets, low "turn over", and a proven record.

Once an outsourcing company has been chosen then there has to be training. They must understand how your company is expecting the testing to be performed. They must know what types of tests you want them to do (stress,load,failover, etc...), the kind of reports that you want and the SLAs that you expect them to achieve.

After you have chosen the team, provided the appropriate training and expectations, what is left? What tools are they going to be using? The same set of tools that you used when the entire team was internal? At first this seems like the correct response. If it worked internally, why wouldn't it work for an outsourcer? Let's explore this for moment.

First let's just talk licenses. How is the outsourcing group going to gain licenses. Do they have their own licenses that they can use? Most do not and they rely on the company to provide that. So do you transfer the licenses that you have internally to the outsourcer? Do you want to keep some of the licenses in house so that you can perform tests internally when it is needed? More than likely you will be keeping at least some of your performance testing licenses in-house. So that means that you will have to buy more licenses for the outsourced team. Can your current testing tool help with this?

What about testing machines? Do you need to get more controllers and load generators? Can the outsourced team utilize the machines that you currently have? Can your current testing tool help with this?

What about management? How do you know that the outsourced team is doing what they are supposed to do? How do you know if the tests that they are creating are correct? How do you know if they are testing enough? In short how do you know that they are doing a good job? Lack of proper management and oversight is one of the biggest reasons why offshoring fails. Can your current testing tool help with this?

What if you would like "follow the sun testing" or better collaboration with testing. Let's say that you have an important project that needs to get tested quickly. And the only way to get this done is to keep handing off the test to different testers around the world. So when one location is done for the day, a new tester can pick up where the last left off and continue with the testing. This becomes a real possibility with offshoring. A test can begin in-house and then shift to an outsourcer off-hours, thus decreasing the time it takes to get the results to the line of business. Can your current testing tool help with this?

HP Performance Center (PC) is the enterprise performance testing platform that can help you with your offshoring/outsourcing needs. Let's start from the top. PC has shared licenses. Anyone around the world, that is given the proper permission, can access the PC web interface and run tests. There is no need for more licences unless there is a demand for more concurrent testing. And if your demand for more simultaneous tests is growing, then you are doing something right.

Now let's move on to machines. With Performance Center all the machines (controllers and load generators) are centrally managed. There is no need to have LGs placed throughout the world. Testers, worldwide, have access to tests and machines through PC. Again the only time that more machines are needed is if the demand increases. No need to buy more machines just because you have outsourced the job.

Performance Center was created for performance testing management. From a single location you can see what projects and tests have been created, how many tests have been executed and who ran them. There is no need to have scripts and reports emailed or copied. All testing assets are stored centrally and accessible through PC's web interface.

Not only can you view the projects, scripts, and results, you can also manage the testing environment itself. You can run reports to see what the demand for your testing environment is an then plan for increases accordingly.

How about "follow the sun" testing? With HP Performance Center anyone with proper access can take over testing. Since all scripts, scenarios, and profiles are completely separated from controllers can stored centrally, it is easy for a new tester to pick up where a previous tester left off. There is no need to upload scripts and scenarios to a separate location, or remember to email them to the next tester. It is all available 24x7 through Performance Center.

Collaboration on testing becomes much easier in PC than with almost any other tool. If you need different people at different locations to all watch a test as it is running, PC can accommodate that. Just log on to the running test and choose the graphs that you are interested in. Now all viewers are watching the test with the information that they are interested in, all through one tool.

HP Performance Center is your best performance testing platform choice when it comes to offshoring and outsourcing.

So after you pick the correct outsourcing company, and properly train them, make sure that you use HP Performance Center to ensure the highest cost savings and highest quality.