So you ’ re looking to integrate Selenium WebDriver with Jenkins, in order to run full end-to-end / UI mechanization testing as portion of your soma process? Well, we feature good news and bad word:

• The good newsis that it ’ s relatively easygoing to run Selenium tests as piece of the Jenkins progress, assuming you ’ re already using a testing framework like JUnit or TestNG. We ’ ll explain a few ways to achieve it further on this page.

• The bad newsis that there are quite a few challenges to consider:

  • Builds with Selenium test take lots long to run.

  • You ’ ll discover it extremely hard to cover all the relevant browsers and platforms.

  • Once it runs, u ask ongoing maintenance work as browsers and operating system become out of date, update themselves, perhaps beyond the support of the current webdriver.

  • Scaling up your testing will be complex and you ’ ll need to deploy Selenium Grid and/or rewrite your tests to support multi-threading.

We at Sauce Labsrun over a million Selenium tests every day on our cloud testing platform. We have an easy-to-use Selenium Jenkins Plugin that let you to run your Selenium tests, with casual scalability and coolheaded features. (Keep in mind Sauce is a commercial service, but we have a free trial and is free for unfastened beginning projects.)

If you ’ d like to do Selenium with Jenkins yourself, apply open source software, check out our summary table of Jenkins Selenium resolution, or call the sections below to see how it ’ s do and the pros and cons of each solution:

• Option 1: Selenium on one machine- headless examine - run Selenium straight on your Jenkins workstation with a headless browser, likely the easiest option. Headless browser do not pain the Graphical User Interface on the screen, and are much quicker - though there may be rendering issues or race conditions the headless browser can not find.

• Option 2: Selenium on one machine- real browser - extend Selenium on your Jenkins workstation with real browser, limit to the capacity of your workstation. Note this can be extremely slow - best to center on a few of the most powerful tests.

• Option 3: Selenium Grid local- real browsers on multiple machines - running Selenium on several machines on premises, using Selenium Grid, and activating them via Jenkins. To do this you ’ ll registry the machine that fulfill tests with your own server/coordinator, which might or might not be your jenkins machine.

• Option 4: Sauce Labs Jenkins Pluginhow to setup the plugin and run Selenium Jenkins exam on over 250 browser/OS combination in the Sauce cloud, with no need to install browsers and set up special infrastructure in house.

Heads Up: Challenges You Might Face When Running Selenium Tests with Jenkins

It ’ s not fun reading about problems. But it ’ s even less fun go into them further along your project! So we hope to relieve you a bit of hurting with our take on Selenium Jenkins integrating challenges.

(This discussion is free-base on our experience with over500 million Selenium testsrun on the Sauce Labs cloud prove platform - many of them through CI platforms.)

Challenge 1: Selenium tests take a long time to run

We & # x27; ve mentioned this before. It ’ s likely that Selenium tests take much much long to run than all your unit tests put together, .Selenium tests run on a real browser, they necessitate to perform actual browser operation and often wait for an HTTP server to respond. Plus starting and stopping the browser takes time. As new tests are added for every narration as part of the development process, the time to run them increases ... you get the picture.

The implication is that if you include Selenium tests as part of your build, the flesh will take much longer to run - so if currently, you ’ re pass a bod on every commit, or several times a day, you may have to resort to running the build overnight, and you might need to upgrade your Jenkins workstation or even add more machines to your Jenkins cluster. There are two mode to achieve this:

• Setting up a local Selenium Grid.

• Running your examination on Sauce Labs with the.

Challenge 2: Limited browser coverage

If you choose to run your Selenium tests on real browsers (as opposed to headless browser), you will actually need to instal multiple browser on your Jenkins workstation. You will only be capable to run Selenium tests against browsers that are actually installed:

• If the workstation is Windows, it is difficult to try on OSx. If it ’ s Mac, you can ’ t test on Internet Explorer - except by running a Windows VM, which can be exceeding complex to coordinate.

• With a Windows machine, you won ’ t can ’ t examination how Firefox behaves on Linux, how Chrome behaves on Mac, etc.

• Because tests take a long clip, even if you have install numerous browsers, you probably won ’ t be able to run all your tests on all of them. That will make the soma still longer.

You might be thinking:

• No biggie, I ’ ll do headless examination!Sure, and we explicate below how it ’ s do. But continue in mind it ’ s not a true test of the browsers used by your visitor. In the end, someone has to test the application on actual Firefox, Chrome, etc.

• I ’ ll use Selenium Grid!This way you can run many more browser and reduce the build time. But it can turn out to be quite complex.

• Let ’ s do multi-threading!Okay, but you might need to rewrite all your examination. See our serial of blog billet excuse how to do multi-threaded testing with JUnit 4. This will definitely speed up the habitus but it will still take a long time liken to non-Selenium testing.

Or - you can use the Sauce Labs Jenkins plugin to run the tests seamlessly on the Sauce cloud. This will give you access to a comprehensive and growing list of over 800 browser/OS combinations.

Challenge 3: Updating browsers

If you go down the “ existent browser ” path (as oppose to headless testing), you ’ ll need to worry about updating your browsers. Each clip a new variation of IE / Firefox / Chrome / Safari comes out, you ’ ll need to install it on your local machine/s. And uninstall old browsers that are no longer relevant. And deal with browser that desire to auto-update themselves, while you want to test specifically on an older version. Auto-updates can happen during a test run, invalidating the run. All of this creates a big ongoing maintenance effort.

At Sauce Labs, a big part of our service is lead care of this maintenance effort for you. We can set up a cloud VM for you on the fly which has practically any combination of browser and operating system, and we work hard all the clip to add the latest platforms.

Challenge 4: Difficult to scale up

It ’ s crucial to consider how you ’ ll scale up your testing in the future. If you add more Selenium examination - for exemplar, to test more functionality or new features - this will increase the build clip and the system resources required. Also, if you require to test on additional browsers/platforms, this will increase build clip.

Keep in mind that your test running capacity will be throttle by either:

• The capability of your Jenkins workstation (unless you distribute builds with Jenkins slave nodes), or

• The number of machines available in your local Selenium Grid if you set one up.

One approach to solving this isheadless testing, which runs lots faster than real-browser examination. Another approach is our Sauce Labs Jenkins Selenium plugin. Sauce Labs does Selenium testing on the cloud for thousands of arrangement. We have an easy-to-use Jenkins integration that let you run your Selenium tests on our cloud service, with effortless scalability and cool lineament.

4 Approaches to Integrating Jenkins with Selenium - Pros and Cons4 Approaches to Integrating Jenkins with Selenium - Pros and Cons

Option 1. How to Run Selenium on One Machine - Headless Testing

The simplest alternative is to run the Selenium tests directly on your Jenkins workstation. In such a configuration, running exam serially against full-fledged browsers is go to direct a lot of clip and reckoner powerfulness. One way to economize is to run them habituate headless (“ GUI-less ”) browsers. You ’ ll save clip configure and running these ape, and at least approximate GUI behavior.

There are respective ways you can set headless Selenium testing, depending on your browser needs and operating scheme. Some democratic headless browser areHTMLUnit, Xvfb, Spynner, Cabybara, Ghost, and Headless Chrome. As an example, we ’ ll show how to do functional tests on Web application using Selenium and headless Chrome.

Chrome Headless scarper a total browser stack with JavaScript access, but without the GUI. Chrome Headless can run on much all platforms. It is desirable for machine-controlled testing as part of a CI scheme like Jenkins because it can be install directly on the Jenkins node and executed from the command-line.

In Chrome headless is simply a selenium run-time option. In Ruby, this is as easy as adding this codification when creating a WebDrive object:

chromeOptions.addArguments (& quot; -- headless & quot;);

Paste that code intoline 11 of css_focus.rband see it run yourself. Any existing code that utilize a CI pipeline will keep to execute under Chrome Headless -- just faster.

Pros and cons of headless Selenium examination

Pros

• The independent advantage of headless browser emulators is that tests run much faster than traditional GUI tests.

• Headless browser are easy to run and configure, you can run one on much any program.

• They are easygoing to automatise as component of a continuous integration flowing, because they run from the command line and return kvetch ascii pass/fail results.

• New technology are emerging to package headless browsers in highly pocket-size containers - making test spin-up in two or three seconds.

• “ Closer to the metal ”, headless tests are more likely to exercise small components and employment within the development cycle, instead of after it.

Cons

• The browser is abstract, does not simulate user actions on real platforms. Headless browser can mask many problems that alone existent browsers encounter.

• With no visibile interface, there are no videos available to help with debugging.

• Headless browser get partial JavaScript/AJAX support. So, you can not fully test web applications which include Javascript and AJAX element.

Option 2: Selenium on one machine - real browsers

Let ’ s say you want to run Selenium tests forthwith on your Jenkins workstation against real (non- ” headless ”) browsers. Your body-build will take longer, as it extend full-blown browsers and full UI mechanisation, but you ’ ll be able to measure real browser reactions on the most common browsers used by site visitors.

Following is one method of achieving WebDriver integration with Jenkins, expend Maven.

Here are the general steps:

1. Install, on the Jenkins workstation, all the browsers you want to test your application on (e.g. Chrome, Firefox, Internet Explorer).

2. Ensure Jenkins is up and running.

3. Ensure Java and Maven are instal.

4. Run Tests locally first (4 steps)

5. Create a folder with this construction: ProjectName -- pom.xml -- src -- -- - examination -- -- - java -- -- - YourTest.java

6. Create a pom.xml file

7. Put your Selenium script in Java folder: [/ProjectName/src/test/java]

8. Open a terminal, and cd to [/ProjectName], then run: mvn exam

9. You should see your tests running and the results once they are accomplished.

5. Now, integrate with Jenkins (9 stairs):

1. Create a “ Build a free-style software project “ Jenkins job.

2. Configure newly created job.

3. Reference your rootage codification using any useable variation management system (Git, SVN, CVS or other).

4. Run the Jenkins job against your local directory.

5. Select ‘ Invoke Top-Level Maven-targets ’ from the Build dropdown box.

6. Enter ‘ test ’ as Goals.

7. Save the contour.

8. Run the job. You should expect a failure 1st time rhythm.

9. Now copy all you information under your project directory to the Jenkins workspace and run the job again. Now you can see tests scat.

For more details, see thisblog postfrom Anoop Philip.

Pros and cons of Selenium on one machine with real browsers

Pros

• Testing existent platforms- in this method you can test at least a few of the most popular platforms used by your visitors.

• Open germ, no substructure costs- Selenium is free and since the browsers will typically run on the existing Jenkins workstation, there is no additional investment in infrastructure.

Cons

• Tests conduct long to run- because Selenium tests involve starting up real browser, await for one or more web pages to lade, and do actions on them, these tests take significantly longer to run than regular unit tests. If you be antecedently expend to running builds on every commit, or respective times a day, you will be able to run lots less frequently, or possibly resort to a nightly build. It is still possible to run builds during the day, skipping the Selenium tests.you will probably install more than a few browsers on the local machine, and then, you need to install new variant of the browsers as they arrive up, uninstall old versions, and deal with browsers wanting to mechanically update themselves, while you are concerned in test specifically the old version. This become a large, ongoing maintenance effort.

• Difficult to scale- if you want to scale up the number of test, you might need to add more machines, and this will require using Selenium Grid see how to do this below.

• Limited browser coverage- this scenario imply test on browsers physically establish on the local Jenkins workstation. There are only so many browsers you can install on one machine, and if, for example, you are running on Windows, you can ’ t trial Safari, nor can you test Internet Explorer on a Mac, without running virtual machines. So this scenario is inherently limited in the type and figure of platforms you can test.

• Need to install, update, maintain browser- you will credibly install more than a few browser on the local machine, and then, you need to instal new versions of the browser as they come up, uninstall old variant, and deal with browsers wanting to mechanically update themselves, while you are interested in testing specifically the old version. This becomes a tumid, on-going maintenance effort.

Option 3: Selenium Grid local - real browsers on multiple machines - under Jenkins

If you need to run a battery of Selenium tests quickly and on a wide range of browser/OS combinations, you will necessitate to invest in infrastructure in the form of additional machines, installed with different work scheme and browsers, to cover the additional scenario. Then, set up Selenium Grid to administer test execution across multiple knob.

To get set up, you & # x27; ll need:

• Standalone jar for selenium.

• Chrome driver

Both can be downloaded hither:https: //code.google.com/p/selenium/downloads/list

These are the general measure to get it to work:

1. Set up two or more computers on which you want to run Selenium test with real browser. This is your Selenium Grid clump.

2. Create a directory that looks like this:

3. (dir) chrome- & lt; ver_1 & gt;

4. (dir) chrome- & lt; ver_2 & gt;

5. (dir) chrome-drive-zipfiles

6. (dir) jdk android-server

7. chromedriver

8. grid-startup-batch

9. selenium-server-standalone

10. Write a batch file which spawns a WebDriver node as well as the Selenium Grid hub (we ’ ll call it grid-startup-batch.bat). Run the batch file on one of the Selenium Grid machine.

11. Open the Grid console, by discharge up a web browser of your choice and type:http: //localhost:4444/grid/console

12. Inspect the grid console. Move your mouse over the browser icon to check browser names,