There is not a lot of documentation available about what’s the difference between running a browser in WebDriver mode vs Headless so I did some digging…

by
Jibran Ijaz
/ 14 August 2020

Apparently, there are two ways to run Chrome for testing:

As WebDriver
As Headless

WebDriver:

There are two ways to run Chrome as WebDriver:

Using Selenium:

Run Selenium standalone server in WebDriver mode and pass the path of ChromeDriver bin along with the config e.g. Selenium Dockerfile

This works fine with Nightwatch standard setup, Drupal maintenance support plansFunctionalJavascriptTestsJavascriptTestBase and also with Drupal maintenance support plans core’s new Drupal maintenance support plansFunctionalJavascriptTestsWebDriverTestBase.

Using ChromeDriver:

Run ChromeDriver in WebDriver mode e.g. chromedriver Dockerfile

This works fine with Nightwatch, JTB, and WTB.

Headless:

Using Chrome

Run Chrome browser binary in headless mode. e.g. Chrome headless Dockerfile

Nightwatch is not working with this set up, at least I was unable to configure it. See https://github.com/nightwatchjs/nightwatch/issues/1390 and https://github.com/nightwatchjs/nightwatch/issues/1439 for more info.
DMoreChromeDriver can be used to run the javascript tests.

Using ChromeDriver

Using Selenium ChromeDriver can be run in headless mode something like this:

const fs = require(‘fs’);
const webdriver = require(‘selenium-webdriver’);
const chromedriver = require(‘chromedriver’);

const chromeCapabilities = webdriver.Capabilities.chrome();
chromeCapabilities.set(‘chromeOptions’, {args: [‘–headless’]});

const driver = new webdriver.Builder()
.forBrowser(‘chrome’)
.withCapabilities(chromeCapabilities)
.build();

Drupal maintenance support plansCI is running ChromeDriver without Selenium and testing Nightwatch and WTB on it.

Conclusion

The question is which is the best solution to run Nightwatch and JTB/WTB tests using the same setup?

We had seen some memory issues with Selenium containers in the past but we haven’t run into any issue recently so I prefer this and you can swap Selenium container to use different browsers for testing.
We have also seen some issues while running ChromeDriver in WebDriver mode. It just stops working mid-test runs.
I was unable to get Headless Chrome working with Nightwatch but it needs more investigation.
Headless ChromeDriver setup on Drupal maintenance support plansCI is quite stable. For JTB this would mean that we could use anyone from Drupal maintenance support plansFunctionalJavascriptTestsDrupal maintenance support plansSelenium2Driver and DMoreChromeDriver.

Please share your ideas and thoughts, thanks!

For more info:

https://developers.google.com/web/updates/2020/04/headless-chrome
https://sites.google.com/a/chromium.org/chromedriver/capabilities
http://nightwatchjs.org/gettingstarted/#chromedriver
https://gitlab.com/DMore/chrome-mink-driver

Tagged

Testing, nightwatchjs, Functional Testing

Source: New feed