Rerun Flaky Tests – Spock Retry

One question I get asked a lot is how you can automatically rerun your test on failure. This is a typical case for heavy, functional test scenarios, which are often flaky. While test flakiness and its management is crucial and extensive matter itself, in this post I want to give a shout to the extremely simple yet useful library: Spock-Retry. It introduce possibility to create retry policies for Spock tests, without any additional custom-rules implementation – just one annotation.

If you are not a fan of Spock testing framework and you prefer JUnit – stay tuned! I will post analogous bit about rerunning JUnit tests soon.

Continue reading

Generating Test Data – jFairy

Test data has been always an issue. If you are running selenium or backend automated tests based on user-related scenarios, in order to make your tests more efficient you need to provide unique and realistic user test data. There’re many ways to deal with this: dumping samples of production databases, writing your own data generators or using the very same data in every test run and cleaning database afterwards.

In this post I want to write about small and handy Java library for generating fake test data – jFairy. Since the library is super-simple to use, this post is just the shout-out for the nice tool I’ve been using in many different automation projects and I hope I’ll put a spotlight on it for my readers.

Continue reading

Testing Asynchronous APIs: Awaitility tutorial

Despite the growing popularity of test automation, most of it is still likely to be done on the frontend side of application. While GUI is a single layer that puts all the pieces together, focusing your automation efforts on the backend side requires dealing with distributed calls, concurrency, handling their diversity and integration.

Backend test automation is especially popular in the microservices architecture, with testing REST API’s. I’ve noticed that dealing with asynchronous events is particularly considered as challenging. In this article I want to cover basic usage of Awaitility – simple java library for testing asynchronous events. All the code examples are written in groovy and our REST client is Rest-Assured.

Continue reading

REST API mocking with Wiremock

Probably every developer or tester have used mocks at least once in their daily professional work. Functionality mocking is an excellent way to improve development process of integrated systems production, or testing heavy dependent application functionalities. With the growth of popularity of REST webservices, API mocking is becoming hot topic.

In this article I would like to introduce a simple getting-started tutorial of setting basic standalone REST API mock server with Wiremock on your local machine. Wiremock is a simple library written in Java for mocking web services.

Continue reading

Performance Testing Tutorial – starting point

I’ve noticed that the subject of performance testing is still a bit of unknown area for most Test Engineers. We tend to focus mainly on functional aspects of our testing, leaving performance, scaling and tuning to developers hands. Isn’t stability a substantial part of software quality? Especially in times of distributed computing, when we’re scaling applications independently and rely on integrations through HTTP protocol. Another aspect is an ability to scale our systems up. In order to be able to handle traffic growth, we have to be aware of the bandwidth limitations.

There’re few well known tools among engineers, such as JMeter, Gatling, Tsung, etc. Although these tools are relatively simple to use, what’s often confusing is analysing and taking conclusions from test results. During interviews for Test Engineer role I often meet candidates claiming to be experienced in field of performance testing, but they’re lacking the knowledge of any performance-related metric or elementary concepts. Since the main purpose of load and performance testing is not the toolset itself, but the knowledge you’re getting from it – the aim of this article is to gather core aspects of this area.

Continue reading

Protractor Tutorial: Jasmine test logging

Proper test logging is a crucial part of our framework setup. As long as your plan is green then everything is good, but have you ever tried to analyse test failures on CI server with no logs or test traces? Good practice is to log every important step of your tests, so that one could read our logs and understand test logic.

In previous tutorial we went through setting up new Protractor project from scratch. If you run your test, you would noticed that Jasmine’s default test output to console is rather poor. In this post we would configure more verbose and user friendly console logging from our tests.

Continue reading

Protractor Tutorial: example project setup

Web development is easily one of the fastest changing field of software engineering. When comes to web application’s functional testing, there is one king – Selenium Webdriver. It has been in use for about a decade and it’s still greatly popular. With demand for fast and reliable single page app’s development framework, Angular’s become first choice for majority of new web projects. Although Selenium Webdriver is still great tool for testing Angular applications, it falls short in some framework-specific aspects.

Developers and community behind Angular framework quickly realised that and came out with their own implementation of Selenium Webdriver – Protractor, an end-to-end testing framework, written on top of WebdriverJS. Protractor runs test agains your application in browser, simulating real user.

protractor-logo-large

Continue reading

REST-Assured 3.0

I started using REST-Assured framework around version 1.5 and since then it’s my first-choice REST client for test automation projects. Back then it was very straight forward, but still way better than it’s available, verbose equivalents. It’s major pros are ease of use – you basically add one static import and you’re ready to go, and BDD convention, which improves readability a lot. But when you dive deeper into REST-Assured framework, you’ll find many handy features, like object serialization, built-in assertions, response manipulation, etc. I already write two post about REST-Assured (framework overview, and more advanced), which become two most popular articles on my blog. Since then the framework gained a lot of popularity and it’s still being developed.

We have version 3.x now, with some great announcements. If you don’t want to study release notes (which I highly recommend!), take a look at this post, with overview of the major changes and new features.

Continue reading