Rerun Flaky Tests – Spock Retry

One question I get asked a lot is how you can automatically rerun your test on failure. This is a typical case for heavy, functional test scenarios, which are often flaky. While test flakiness and its management is crucial and extensive matter itself, in this post I want to give a shout to the extremely simple yet useful library: Spock-Retry. It introduce possibility to create retry policies for Spock tests, without any additional custom-rules implementation – just one annotation.

If you are not a fan of Spock testing framework and you prefer JUnit – stay tuned! I will post analogous bit about rerunning JUnit tests soon.

Continue reading

Generating Test Data – jFairy

Test data has been always an issue. If you are running selenium or backend automated tests based on user-related scenarios, in order to make your tests more efficient you need to provide unique and realistic user test data. There’re many ways to deal with this: dumping samples of production databases, writing your own data generators or using the very same data in every test run and cleaning database afterwards.

In this post I want to write about small and handy Java library for generating fake test data – jFairy. Since the library is super-simple to use, this post is just the shout-out for the nice tool I’ve been using in many different automation projects and I hope I’ll put a spotlight on it for my readers.

Continue reading

Protractor Tutorial: Jasmine test logging

Proper test logging is a crucial part of our framework setup. As long as your plan is green then everything is good, but have you ever tried to analyse test failures on CI server with no logs or test traces? Good practice is to log every important step of your tests, so that one could read our logs and understand test logic.

In previous tutorial we went through setting up new Protractor project from scratch. If you run your test, you would noticed that Jasmine’s default test output to console is rather poor. In this post we would configure more verbose and user friendly console logging from our tests.

Continue reading

Protractor Tutorial: example project setup

Web development is easily one of the fastest changing field of software engineering. When comes to web application’s functional testing, there is one king – Selenium Webdriver. It has been in use for about a decade and it’s still greatly popular. With demand for fast and reliable single page app’s development framework, Angular’s become first choice for majority of new web projects. Although Selenium Webdriver is still great tool for testing Angular applications, it falls short in some framework-specific aspects.

Developers and community behind Angular framework quickly realised that and came out with their own implementation of Selenium Webdriver – Protractor, an end-to-end testing framework, written on top of WebdriverJS. Protractor runs test agains your application in browser, simulating real user.

protractor-logo-large

Continue reading

Selenium Grid with Docker: custom nodes

In my last post I wrote about creating Selenium Grid with use of Docker. I’ve received some questions about customising node’s containers – by default, Docker containers for Selenium Grid nodes run only one instance of browser per node. It’s important to understand that running Grid on Docker is slightly different approach than running it alone – instead of building huge Grid with multiple nodes and dozens of browsers, you run few smaller, independent machines. If one machine is down – you throw it away and build another one. Great use case of Selenium Grid with Docker is to build Grid’s machines as a self service for teams in your company, or to build it automatically before automated tests trigger, and destroy it after.

Nevertheless, it is possible to tweak default node’s conteiners, so they would contain more browser instances. In this post we’ll create container with custom Selenium Grid’s node configuration – our container would provide more than one instance of browser. Remember, that if you have an account on Docker Hub, you can host one container for free, so after going through this tutorial you can commit your own container and host it, or just create your own, custom Grid on Docker, tailored to your needs.

containers

Continue reading

Selenium Grid with Docker

Selenium webdriver on it’s own, or with it’s implementation, like Geb is arguably the most popular solution for testing web-based applications. Besides all it’s greatness, it has some flaws. Selenium tests are slow, and it’s cost of maintenance is big. The answer for the first issue is distributed testing with Selenium Grid, which I described previously.

From the DevOps perspective though, setting Selenium Grid configuration like that is highly over-expensive and non-scalable. The answer for this can be Docker with it’s docker-compose tool. In this post we will try to create vm provisioned by docker-compose and set up scale Selenium Grid. All of this will be run with one command.

docker

Continue reading

Selenium: Design Patterns

In a previous article I described a set of best practices for designing automated tests. So as I said, in this post I would like to expand our toolbox of two design patterns, that work great with selenium tests. In this post we will dig into Page Object pattern and combine it with Page Factory. As always, whole code for project is available on my github.

Continue reading

Test Automation: Good, Bad and Ugly

The modern approach to software quality and software development life cycle requires that business guys, developers and testers understand that the long manual test phase, although often still necessary, must be reduced to a minimum and replaced by test automation. Working in continuous delivery and continuous integration environment requires us to create automated tests that run on demand, checking our application integration and it’s core functionality correctness. However, there are still many problems with designing and writing automated tests, resulting in their costly maintenance or abandonment in favor of a return to manual processes.

Continue reading

Selenium Grid

Selenium WebDriver is a great and powerful tool when it comes to web test automation. The pros would be appreciated by everybody who test web based applications. However, along with the enlargement of our test base, there are some limitations, and use of single WebDriver implementation on the local machine becomes insufficient. The answer to this problem can be Selenium Grid tool, which distribute tests execution to remote machines.
hubnode

Continue reading