Selenium Grid parallel automation testing for Varnish Administration Console

User acceptance testing is the last phase of the software testing process which evaluates the system functionality with the business requirements and assess whether it is acceptable for delivery. Acceptance tests should be automated so they can be run often. This blog aims to explain how the automated acceptance tests process for Varnish Administration Console (VAC) was formed step-by-step from scratch. We did not have automated testing framework at the beginning and testing was carried out manually based on some test cases. For future development it became crucial for us to write maintainable automated tests for our systems.

Why Selenium Grid?

There are various automation tools out there for automating web applications and selecting the best automation testing framework was challenging in the beginning. Acceptance tests framework are being developed with Selenium Grid. It was chosen because it can be used to distribute tests across different machines against different browsers in parallel. Selenium Grid runs on multiple operating systems, multiple browsers and languages and, it’s free!

How does it work at Varnish Software?

The problem I initially saw was little documentation on how to set up grid and how to make an automation test framework from scratch. For running Selenium WebDriver tests in parallel, we need to set up the Selenium Grid Server as a hub. The selenium-server-standalone package includes the Hub, WebDriver, and legacy RC needed to run the grid. The Hub is the central point that will receive all the test request and distribute them the right nodes. Nodes are the Selenium instances that are attached to the Hub which will execute the tests. 

More information for our technical crowd

The next step was to run Selenium and browsers inside Vagrant box so any developer could check out the tests and be able to run them without any browser setup and configuration. It also enables them to run the tests on a clean VM each time so the Vagrant file can be used purely to fire up the grid hub and node. This means the Selenium infrastructure is decoupled from the project.

The Vagrant file installs default jdk as Selenium runs on java and it starts up the Selenium hub and node. The hub will automatically start-up using port 4444 by default. It also set up Firefox and Chrome headless on a precise64 box by installing Xvfb in order to fake a display and let Selenium and the browser think they are running on a machine with a display. After Vagrant is up selenium you can simply point your test to the port forwarded default hub location.

The final steps

Finally, we needed to integrate the Selenium tests to gradle to compile and run them in commandline. The build.gradle file can cater to different browsers, specific browser versions (if needed) and different OS and platforms. Afterwards, we run the tests by passing the parameters to gradle arguments, the test result folder is created and test reports are provided in html and xml.