Software QA: Jenkins + Jenkins Slave Nodes + Selenium 2 + Browsermob Proxy


I’ve just finished a new test setup that allows me to capture network traffic in a test suite that is launched by Jenkins onto a Selenium 2 Grid. It was as painful as it sounds, but just as satisfying to complete. One of the big hurdles was the need to run the tests via Jenkins and capture the network traffic generated by the test, which occurs on an unknown Selenium node.

If the test system we use were not distributed, this would have been relatively simple. Just start up a Java instance of a Browsermob Proxy, run the test, and read the net traffic from that proxy. This works locally, even during development, but in a scenario where a Jenkins server compiles and launches the test job, the Jenkins server would become the proxy. Not a solution. Also, in this setup, the Java code (the Selenium tests in my case) launched by Jenkins doesn’t know the address of the node that the tests are running on, so a proxy cannot be launched on that node. Even if you could, you couldn’t gather the network traffic, because, again, you don’t know which node it’s on. I’ve researched this problem quite a bit and found lots of “solutions” that do not work, at least for this environment. Most looked like partial solutions at best.

Solution. Set the tests needing a proxy to run on localhost (wait, wait… I’m not crazy) Create a Jenkins slave-node, I chose a JNLP slave on a Windows PC, with a selenium server running locally (see where I’m going with “localhost”, yet?) Setup the job in Jenkins to be restricted to that node. Now, for the proxy part. Write the test code to start and use a Browsermob proxy as a Java instance, not using the ‘java -jar filename’ option at the command line. This is a nice way to go because a Java proxy instance is FAR cleaner to work with than writing to and reading from the REST interface you have to use otherwise. A nice side effect for us using this setup with a Jenkins slave-node is that other tests in the test suite will still be distributed through our Selenium Grid Hub. This means we can still migrate to a slave-node model at any time, but only if and when we decide to do so.

Now when the tests are launched, they should run on the Jenkins slave-node, and that means the Java test code will run on the same server as the tests themselves (remember the crazy part where I said the tests should be set to run on localhost? They are local to the slave-node, now) Viola, the test code and proxy, and the Selenium browser session are running on the same machine, so can both contact the proxy using “localhost” as the server address.

If you find this article helpful, but could use a bit more detail on part of it, please leave a comment/question. I’d be glad to expand on this where I can.

Robert Arles


  • Mark Brewster

    Interesting approach Mr. Arles, we’re discussing how to integrate BMP in Selenium!

    There’s a lot going on here…

  • robert

    Fun stuff! I hope you come up with a working setup. It can be a bit of a pain depending on your requirements. Good luck.

  • Nilesh Kulkarni

    Could you please explain how did you manage to publish browsermob reports in HAR format on Jenkins?

  • robert

    Hi Nilesh,

    Maybe you could explain to me what you mean a bit more, but I’ll have a shot at a response for you. I was actually not publishing a report on Jenkins, but using Jenkins only as a tool to launch the tests in this case. The data from browsermob was programmatically parsed as needed. In our case, looking for calls from the browser to a specific server so we could check that the right data was being passed.

    Thanks. Robert.

  • Nilesh

    Thanks Robert. My requirement is to show HAR reports on jenkins so that we can monitor, review the reports as required.

    • robert

      I see. Sorry to say, I don’t have a solution that would display HAR reports in Jenkins. I think that would require a Jenkins plugin, and some code in your test to output a usable data format, which is often XML, as used by code coverage and similar plugins to create graphs and such.