Performance Testing Tutorial – starting point

I’ve noticed that the subject of performance testing is still a bit of unknown area for most Test Engineers. We tend to focus mainly on functional aspects of our testing, leaving performance, scaling and tuning to developers hands. Isn’t stability a substantial part of software quality? Especially in times of distributed computing, when we’re scaling applications independently and rely on integrations through HTTP protocol. Another aspect is an ability to scale our systems up. In order to be able to handle traffic growth, we have to be aware of the bandwidth limitations.

There’re few well known tools among engineers, such as JMeter, Gatling, Tsung, etc. Although these tools are relatively simple to use, what’s often confusing is analysing and taking conclusions from test results. During interviews for Test Engineer role I often meet candidates claiming to be experienced in field of performance testing, but they’re lacking the knowledge of any performance-related metric or elementary concepts. Since the main purpose of load and performance testing is not the toolset itself, but the knowledge you’re getting from it – the aim of this article is to gather core aspects of this area.

Continue reading

Protractor Tutorial: Jasmine test logging

Proper test logging is a crucial part of our framework setup. As long as your plan is green then everything is good, but have you ever tried to analyse test failures on CI server with no logs or test traces? Good practice is to log every important step of your tests, so that one could read our logs and understand test logic.

In previous tutorial we went through setting up new Protractor project from scratch. If you run your test, you would noticed that Jasmine’s default test output to console is rather poor. In this post we would configure more verbose and user friendly console logging from our tests.

Continue reading

Protractor Tutorial: example project setup

Web development is easily one of the fastest changing field of software engineering. When comes to web application’s functional testing, there is one king – Selenium Webdriver. It has been in use for about a decade and it’s still greatly popular. With demand for fast and reliable single page app’s development framework, Angular’s become first choice for majority of new web projects. Although Selenium Webdriver is still great tool for testing Angular applications, it falls short in some framework-specific aspects.

Developers and community behind Angular framework quickly realised that and came out with their own implementation of Selenium Webdriver – Protractor, an end-to-end testing framework, written on top of WebdriverJS. Protractor runs test agains your application in browser, simulating real user.

protractor-logo-large

Continue reading

REST-Assured 3.0

I started using REST-Assured framework around version 1.5 and since then it’s my first-choice REST client for test automation projects. Back then it was very straight forward, but still way better than it’s available, verbose equivalents. It’s major pros are ease of use – you basically add one static import and you’re ready to go, and BDD convention, which improves readability a lot. But when you dive deeper into REST-Assured framework, you’ll find many handy features, like object serialization, built-in assertions, response manipulation, etc. I already write two post about REST-Assured (framework overview, and more advanced), which become two most popular articles on my blog. Since then the framework gained a lot of popularity and it’s still being developed.

We have version 3.x now, with some great announcements. If you don’t want to study release notes (which I highly recommend!), take a look at this post, with overview of the major changes and new features.

Continue reading

Wiremock stateful behaviour

In distributed systems architecture, testing becomes much more complex than in monolith architecture. Although only end-to-end testing gives us full confidence of system-wide implementations, we should also test services in isolation. The easiest way to obtain isolation in microservices architecture is to stub external applications.

The problem is that mocks are usually stateless – they simulate application logic with fixed actions, without any conditional behaviour. If we want to test cases like HTTP latency or data redundancy validations, where external service represents different behaviour among repeated calls, simple mocks can be insufficient.

Continue reading

Selenium Grid with Docker: custom nodes

In my last post I wrote about creating Selenium Grid with use of Docker. I’ve received some questions about customising node’s containers – by default, Docker containers for Selenium Grid nodes run only one instance of browser per node. It’s important to understand that running Grid on Docker is slightly different approach than running it alone – instead of building huge Grid with multiple nodes and dozens of browsers, you run few smaller, independent machines. If one machine is down – you throw it away and build another one. Great use case of Selenium Grid with Docker is to build Grid’s machines as a self service for teams in your company, or to build it automatically before automated tests trigger, and destroy it after.

Nevertheless, it is possible to tweak default node’s conteiners, so they would contain more browser instances. In this post we’ll create container with custom Selenium Grid’s node configuration – our container would provide more than one instance of browser. Remember, that if you have an account on Docker Hub, you can host one container for free, so after going through this tutorial you can commit your own container and host it, or just create your own, custom Grid on Docker, tailored to your needs.

containers

Continue reading

Building microservices testing framework

RESTful services and microservice architecture in general are big trends in the industry right now. I’ve noticed it also from this blog’s traffic analytics, where topics related with REST testing get the highest interest. Therefore I’ve decided to create some kind of example framework for RESTful APIs functional testing. Project is based on REST-Assured as a services client, Spock as a testing framework, Groovy and Gradle. You can clone full repository from my github. Tests are run against Wiremock API described in this post. Please consider this project as a kind of bootstrap, since it’s an example, not full-blown test framework. Ok, so as Linus said – talk is cheap, show me the code! (PS: and yes, I know that guy on the picture isn’t Linus Torvalds 😉 )

linus

Continue reading

Selenium Grid with Docker

Selenium webdriver on it’s own, or with it’s implementation, like Geb is arguably the most popular solution for testing web-based applications. Besides all it’s greatness, it has some flaws. Selenium tests are slow, and it’s cost of maintenance is big. The answer for the first issue is distributed testing with Selenium Grid, which I described previously.

From the DevOps perspective though, setting Selenium Grid configuration like that is highly over-expensive and non-scalable. The answer for this can be Docker with it’s docker-compose tool. In this post we will try to create vm provisioned by docker-compose and set up scale Selenium Grid. All of this will be run with one command.

docker

Continue reading

Performance Testing – Vegeta Attack!

Performance testing is crucial field of modern software development. Taking into account that in today’s world majority of app’s communication is web-based, it turns out to be even more important.  However, it still enjoys less interest than automated functional testing, and publications on load testing subject usually focus on mature and complex tools like JMeter or Gatling. In this post I’d like to introduce command line usage of a super simple and lightweight tool for performance testing of HTTP services, which is Vegeta.

vegeta-min

Continue reading

Jenkins Beginners Tutorial

Software Development domain is well known for buzz-words. With the growth of agile movement, Continuous Delivery (CD) and Continuous Integration (CI) terms become highly popular. One of the best tool supporting the processes of CD and CI is Jenkins.

Jenkins is an continuous delivery and continuous integration server application. Typical use-cases of Jenkins are building your application from version control system, running acceptance tests or deploying application on dev/test/prod environments. Jenkins is free and cross-platform, and it’s web application written in Java, so it comes as a .war file.

In this article, we will perform basic and most common configuration of Jenkins job. Our scenario is to checkout project from git repository and run tests. Last step would be to generate test report.

logo

Continue reading