How to measure Website's performance using Java+PhantomJS

Java+PhantomJSでサイト監視

How to measure Website's performance using Java + PhantomJS.
You can measure not only HTTP response but also more detailed information (e.g. page load time).

Introduction

There was a request to measure Website's performance, this artilce describes how to measure using Java.
* Photo : Pakutaso

Requirement

  • I want to measure not only HTTP response but also load-time of the Web page.
  • Want to take the screenshot of the Web page.
  • Can be performed as a batch process.
  • Cross-platform ( Windows/Linux )
  • Enable multithreaded execution ( not covered in this article )

Plan

  • Use Java.
    Cross platform(Win/Linux). Easy to implement multithreaded.
  • Use Navigation Timing API.
  • Use headless web browser.
    To use the Navigation Timing API, need to use a Web browser, so use a headless browser batch application.
    There are some implementations, I decided to use PhantomJS ( PhantomJS seems to be support Navigation Timing API )

Environment

OS Windows 8.1
Java 1.8
PhantomJS 2.1.1

Install PhantomJS

At first, install PhantomJS and check the operation

Installation

Download a binary package from the PhantomJS download page and install it.
If you use Windows, get the binary package (zip) file for Windows, and expand it to the appropriate directory.

Operation check

If you expand the zip file, you can find executable file (phantomjs.exe) under the bin directory. Let's check the operation by this executable file and a sample script.

* sample scripts are included under the examples directory.

  1. Run the command prompt.
  2. Go to the destination directory of the PhantomJS.
    > cd <destination directory>
    
  3. Try to run the script (loadspeed.js) to measure the page load time.
    > bin\phantomjs.exe examples\loadspeed.js <URL>
    
  4. If there is no problem, Loading time is displayed.
    Page title is IT (Innovation Technology) ...
    Loading time 1822 msec
    

*As a sample to take a screenshot, there is a responsive-screenshot.js.

*If you want to measure the load running time from Windows batch scripts/Linux shell script, easiest way is edit the loadspeed.js to change output value, and redirect the results to a file (or pipe to appropriate application).

Create Java application

Next implement Java application.

Maven ( pom.xml ) setting

In this article, use Apache Maven.

Create pom.xml.
Add selenium-java and phantomjsdriver in dependencies.

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
  <modelVersion>4.0.0</modelVersion>
  <groupId>agilegroup.co.jp</groupId>
  <artifactId>phantomjstest</artifactId>
  <version>0.0.1-SNAPSHOT</version>
  <dependencies>
  	<dependency>
  		<groupId>org.seleniumhq.selenium</groupId>
  		<artifactId>selenium-java</artifactId>
  		<version>2.53.0</version>
  	</dependency>
  	<dependency>
  		<groupId>com.github.detro.ghostdriver</groupId>
  		<artifactId>phantomjsdriver</artifactId>
  		<version>1.1.0</version>
  	</dependency>
  </dependencies>
</project>

Create Java class

Then, create Java application.

Create the following class ( PhantomJSTest ).

package jp.co.agilegroup.sample.phantomjs;

import org.openqa.selenium.JavascriptExecutor;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.phantomjs.PhantomJSDriver;
import org.openqa.selenium.remote.DesiredCapabilities;
import org.openqa.selenium.support.ui.ExpectedCondition;
import org.openqa.selenium.support.ui.WebDriverWait;

public class PhantomJSTest {

	private static final String PHANTOMJS_PATH = "C:/phantomjs.exe";

	public static void main(String[] args) {
		if (args.length < 1) {
			System.out.println("URL parameter is required.");
			System.exit(0);
		} else {
			String url = args[0];
			System.out.format("URL   : %s\n", url);
			PhantomJSTest test = new PhantomJSTest();
			test.checkSitePerformance(url);
		}
	}

	public PhantomJSTest() {
	}
	
	/**
	 * Initialize PhantomJSDriver.
	 */
	private PhantomJSDriver initDriver() {
			// set Capabilities
			DesiredCapabilities capabilities = DesiredCapabilities.phantomjs();
			capabilities.setJavascriptEnabled(true);
			
			System.setProperty("phantomjs.binary.path", PHANTOMJS_PATH);
			PhantomJSDriver driver = new PhantomJSDriver(capabilities);
			return driver;
	}

	/**
	 * Check Performance.
	 */
	private void checkSitePerformance(String url) {

		PhantomJSDriver driver = null;

		try {
			driver = initDriver();
			driver.get(url);	// access to specified URL
			waitForLoad(driver);
			// Get values of Navigation Timing
			long startTime = (Long) driver.executeScript(
					"return window.performance.timing.navigationStart");
			long loadEndTime = (Long) driver.executeScript(
					"return window.performance.timing.loadEventEnd");
			long responseEndTime = (Long) driver.executeScript(
					"return window.performance.timing.responseEnd");

			System.out.format("Response Time : %d\n", responseEndTime - startTime);
			System.out.format("PageLoad Time : %d\n", loadEndTime - startTime);
		} finally {
			if (driver != null) {
				driver.quit();
			}
		}
	}

	/**
	 * Wait for page load.
	 */
	void waitForLoad(PhantomJSDriver driver) {
	    ExpectedCondition<Boolean> pageLoadCondition = new
	        ExpectedCondition<Boolean>() {
				public Boolean apply(WebDriver driver) {
	                return ((JavascriptExecutor)driver).executeScript("return document.readyState").equals("complete");
				}
	        };
	    WebDriverWait wait = new WebDriverWait(driver, 60);
	    wait.until(pageLoadCondition);
	}
}

briefly describe the above.

main

is the static void main method in Java applications.
recieve a url parameter. if url parameter is not specified, print message and exit.

initDriver

Initialize Selenium WebDriver (PhantomJSDriver).

Set the path of executable file in System.setProperty("phantomjs.binary.path", ...).

checkSitePerformance

Measure a Website's performance.

Operation check

When you run the above code, It looks like the following.

URL   : http://www.agilegroup.co.jp/
6 09, 2016 1:26:15 午後 org.openqa.selenium.phantomjs.PhantomJSDriverService <init>
情報: executable: C:\Software\Network\phantomjs-2.1.1-windows\phantomjs-2.1.1-windows\bin\phantomjs.exe
6 09, 2016 1:26:15 午後 org.openqa.selenium.phantomjs.PhantomJSDriverService <init>
情報: port: 39065
6 09, 2016 1:26:15 午後 org.openqa.selenium.phantomjs.PhantomJSDriverService <init>
情報: arguments: [--webdriver=39065, --webdriver-logfile=C:\Eclipse\workspace\phantomjstest\phantomjsdriver.log]
6 09, 2016 1:26:15 午後 org.openqa.selenium.phantomjs.PhantomJSDriverService <init>
情報: environment: {}
[INFO  - 2016-06-09T04:26:17.750Z] GhostDriver - Main - running on port 39065
[INFO  - 2016-06-09T04:26:17.942Z] Session [4f72b410-2dfa-11e6-a691-810a6a8304e6] - page.settings - {"XSSAuditingEnabled":false,"javascriptCanCloseWindows":true,"javascriptCanOpenWindows":true,"javascriptEnabled":true,"loadImages":true,"localToRemoteUrlAccessEnabled":false,"userAgent":"Mozilla/5.0 (Windows NT 6.2; WOW64) AppleWebKit/538.1 (KHTML, like Gecko) PhantomJS/2.1.1 Safari/538.1","webSecurityEnabled":true}
[INFO  - 2016-06-09T04:26:17.943Z] Session [4f72b410-2dfa-11e6-a691-810a6a8304e6] - page.customHeaders:  - {}
[INFO  - 2016-06-09T04:26:17.943Z] Session [4f72b410-2dfa-11e6-a691-810a6a8304e6] - Session.negotiatedCapabilities - {"browserName":"phantomjs","version":"2.1.1","driverName":"ghostdriver","driverVersion":"1.2.0","platform":"windows-8.1-32bit","javascriptEnabled":true,"takesScreenshot":true,"handlesAlerts":false,"databaseEnabled":false,"locationContextEnabled":false,"applicationCacheEnabled":false,"browserConnectionEnabled":false,"cssSelectorsEnabled":true,"webStorageEnabled":false,"rotatable":false,"acceptSslCerts":false,"nativeEvents":true,"proxy":{"proxyType":"direct"}}
[INFO  - 2016-06-09T04:26:17.944Z] SessionManagerReqHand - _postNewSessionCommand - New Session Created: 4f72b410-2dfa-11e6-a691-810a6a8304e6
Response Time : 84
PageLoad Time : 2372
[INFO  - 2016-06-09T04:26:21.387Z] ShutdownReqHand - _handle - About to shutdown

Take ScreenShot

leave also wrote code to take a screenshot.

add the following code after the "waitForLoad(driver);" of "checkSitePerformance" method.
"driver.getScreenShotAs" method saves screenshot as temporary file ( and deleted on termination ), so copy it.

			File tmpFile = driver.getScreenshotAs(OutputType.FILE);
			File scnFile = new File("C:/tmp/scr.png");
			try {
				Files.copy(tmpFile.toPath(), scnFile.toPath(), StandardCopyOption.REPLACE_EXISTING);
			} catch (IOException e) {
				e.printStackTrace();
			}

Summary

HAR Viewer
Zoom
HAR Viewer

I can measure Website's Performance using Java + PhantomJS.

If you want to get more detailed information, you can get HAR ( HTTP ARchive ) data using netsniff.js ( sample script ).

You can parse / visualize HAR with the following website.