Storm Forecast Blog

On APIs, Load Testing, Performance and StormForger.


Performance Testing – Pre and Post Cloud?

Cloud

The last part of our blog posting series "Why Load and Performance Testing in the Cloud?" will give an answer to the question "what is the difference in all this to pre-cloud times?". The article is based upon a talk I gave at the AWS PopUp Loft in Berlin.

Difference to pre-cloud?

After my overview on the types of testing I'd like to raise the question: Is this actually any different from what it used to be pre-cloud?

I would clearly have to answer: Yes and no. The testing needs and methods have not changed a lot. The needs have maybe increased a bit because our environments are tending to be more complex overall.

The main difference is though that multiple, scalable, performance testing environments used to be very expensive and very hard to manage. Not many could actually do that for real projects.

This is where the automation capability of the cloud can really make a difference. If you have your entire infrastructure, services, servers and code automated, spinning up testing environments suddenly becomes viable. You can create entire infrastructures and environments fully automated, run a series of performance tests, gather all relevant data and shut it down again within a couple of hours and for very little money.

One important thing to keep in mind, though, when aiming for such a testing environment: Do not forget about state. With state I am referring to databases, caches that are warm, etc. Having a reproducible environment, including test data is one of the bigger challenges. Frankly, I haven't seen a good and sound solution for that yet.

Conclusion

In a nut shell, you can state:

  • Scaling resources is not scaling application
  • Understanding is very important to make your system efficient and scalable
  • Complexity is still around and will become apparent again in non-trivial ways when it comes to performance
  • You should utilize the cloud to make testing simpler

Infrastructure and software architecture in general is nowadays ever changing and evolving. Getting solid data on how your system is performing is not only relevant for your application in particular but an important piece when it comes to do Continuous Architecture. Your architecture is ever evolving to run tests and validate it!

Want to know more? Switch to the previous articles:

Slides

Create repeatable Performance & Load Tests easily using our JavaScript DSL to verify your HTTP APIs. StormForger enables you to do continuous performance testing to make the web a faster place. Learn more...

definition.setTarget("API.EXAMPLE.COM");

definition.setArrivalPhases([
  {
    duration: 5 * 60,  // 5min in seconds
    rate: 1.0,         // clients per seconds to launch
  },
]);

definition.session("hello world", function(session) {
  session.get("/", { tag: "root" });
});
:production