We ’ ve continue a lot of ground in the past few blog posts, including how to contribute to the project, and some details of what you can expect as a Selenium exploiter. But there ’ s more to Selenium than just the APIs you use to indite your examination, and one of the big lineament we ’ ve not covered yet is the refreshed Selenium Grid: a mechanism that allow you to lot your tests across a fleet of machine.

Before going further, it ’ s invariably nice to acknowledge where we ’ ve come from, not simply because it ’ s interesting, but also because it helps excuse the “ why ” of the refreshed design.

Way rearward in the mist of time (2008), Jennifer Bevin and Jason Huggins worked on a scheme called the Selenium Farm at Google. This was a fleet of machines sit in a closet somewhere that let you to run the original Selenium protocol. This was, of trend, at Google scale, so there was more than one closet:)

This allowed people at Google to distribute their examination and scale out beyond their individual machines. This was such a great thought that when Jennifer mouth about the Farm at a Selenium meetup, Philippe Haringou (so at ThoughtWorks) adjudicate to write an Open Source implementation of the same thing, which he called “ Selenium Grid. ”

Selenium Grid was a lovely piece of technology, but it had one drawback: it only utter the original Selenium RC protocol. That was fine, but WebDriver spoke a different wire protocol, which was called the JSON Wire Protocol, and folks wanted to be able to use both Selenium RC and WebDriver at the same time.

This is where Francois Reynaud enters the picture. He was working at eBay, account to Michael Palotas, and he had written something like the original Selenium Grid, but which also work with the JSON Wire Protocol. They very kindly contributed that work to the Selenium project, and it formed the footing of the Selenium Grid 2. We decide at the time that the Selenium standalone server would effectively be a “ grid of one ”: it would have everything you want in it to set up a Selenium Grid, as well as to work as a single standalone server. Merging the code and getting it stable lead a while, but thanks to the efforts of Francois, Kristian Rosenvold, and many others, we conflate Grid 2 into the main Selenium project and shipped Selenium 2 in 2011.

Even if 2011 doesn ’ t feel that long ago, the modernistic cosmos has changed rather importantly. In 2011, we didn & # x27; t have Docker. We didn & # x27; t hold Kubernetes, we didn & # x27; t actually have AWS. So, the Selenium Grid didn & # x27; t know that those things were coming in, and wasn & # x27; t written to take advantage of them. Fortunately, wedidhave practical machine back then, and the Grid 2 had been designed to be able to indorse them.

This was the sweetener that an excellent project call Zalenium. Developed by Diego Molina, Zalenium added a really nice UI and support for Docker and Kubernetes—all built on top of the Grid 2. This has allowed the Selenium Grid to uphold to be relevant and useful to this day, which is an astonishing achievement.

But, like I state, it took us time to steady Grid 2: about six months of hard work, mostly led by Kristian. That ’ s because while Grid 2 was sophisticated, the code was hard to say and hard to maintain—and rattling few people could do that. Bad, the merging of the Grid 2 and the original Selenium server was pretty unrefined; effectively there were two separate servers ship in the same binary. This led to problems occur in the Grid but not when run in standalone mode, and vice versa.

Pro tip: Tools like SUSA can handle this autonomously — upload your app and get results without writing a single test script.

With Selenium 4 we ’ ve decided to sting the bullet and reference these three concern. Firstly, we want something that was easy to work on and maintain. Secondly, we want to merge the servers into a individual unit. Thirdly, we need something that will take reward of the reality of modern substructure we experience available to us now, not precisely in the descriptor of Docker and Kubernetes, but also to be capable to use issue technologies such as distributed tracing.

To do this, we looked at the functionality the Grid provided, and to model each piece as a component that we can either run “ in memory ” (countenance us to get a single standalone server), or in a more distributed way, allowing the “ hub and node ” architecture that we ’ re conversant with from the original Selenium Grid, to a fully distributed plan.

The first component is the “ router ”. It serve as the introduction point to the Grid. You can expose it to the internet and it directs postulation to the Grid. It ’ s designed to be homeless, and so you can add more to Grid as you happen you need to.

When the router realize a new session asking, it grade it on a “ session queue. ” The session queue is say by a component name the “ distributor, ” which maintains a framework of all the place in the Grid where a session could be run: we call them “ slot ”. The slot are host by portion phone “ nodes, ” and each node can have one or more slots. When the distributer pulls a new session request from the queue, it identifies the most appropriate slot to use, and forwards the asking to the node which owns the slot. Once the node has commence the session, the allocator puts the session ID and the URL of the Node that is running the test into the “ session map ” —you can believe of the session map as be a simple map of a session ID to URL—and the new session response is post back to the waiting test.

Requests for running sessions (that is, for most webdriver calls!) are handled slightly otherwise. The router will use the session map to look up which node to forward the postulation to, forefend the need to imply the distributor entirely. This imply that you can just keep adding knob to the Grid, and there are few bottlenecks in the architecture to slow your requests down.

Conceptually, there are these five moving pieces within the Grid. But there & # x27; s actually a sixth one, which is a message bus. The five Grid components communicate internally through the message bus, but the actual components that you need to think about when you & # x27; re thinking about the Grid are the Router, Session Queue, Distributor, and Nodes.

When you run Selenium Grid 4 in “ standalone ” mode, you actually get a “ grid of one ”. We wire up all these components in a single process, but they ’ re all still there.

You can also run it in the traditional Hub and Node near that you & # x27; ve seen with Selenium Grid 2, where you fire up a Hub and a Node and it registers. If you & # x27; ve used Selenium Grid recently, that & # x27; s probably the architecture that you & # x27; re conversant with. In this case, most of the portion (the router, session queue, and allocator) are extend in the Hub, and the Nodes run the sessions themselves.

What ’ s new with Grid 4 is that you can go into a fully deal mode if you desire to. Typically, you would need to use something like Kubernetes for this, and some of the key components are designed to store data expend a database or Redis for best reliability and scalability.

Something to note is that when go a distributed Grid, it become really difficult to figure out what is happening, particularly when something goes wrong. To assuage that problem, we have adopted Open Telemetry to wreak observability into the Grid. What do we mean by observability? It simply means that we want to be able to see everything that happens.

Finally, we ’ d like to expose information about the running Grid in meaningful and useful ways. The original Grid support both JMX (a Java management API) and an HTML-based console. While nice, this didn ’ t create it easy to query specific areas of the Grid that you might be interested in that weren ’ t surface by the UI (for example, how many slots are available, or to detect out which node a special session is running on) To supply more flexibility, we ’ ve chosen to supply a GraphQL endpoint for the Grid. To ensure that GraphQL endpoint is sufficiently flexible, we ’ re construct the new Grid console using it. That should grant you to also extract useful prosody and info from the Grid for your monitoring motivation.

Those are some highlights of the new Selenium Grid. What are you the most excited about?