Behat And Selenium In Vagrant

Even with testing frameworks like PHPUnit or PHPSpec, I've still felt like something was missing. Unit tests made me sleep a little better at night, knowing my objects were interacting the way they should be...but my tests still didn't give me the dreams I had hoped for. I felt that even though I could unit test the balls off my application, I didn't have anything in place to actually test what users would experience. Does the app actually function in the browser? Do all the forms work as expected? Do all the redirects work and go to the right places? Will the user see certain validation messages in different scenarios? Enter Behat.

Behat is a testing framework that allows you to test what your end users will experience. It's the perfect compliment to PHPUnit and/or PHPSpec. I'll be honest, Behat made me nervous. There are quite a few different components that go into this testing framework: Gherkin, Mink, Goutte, Selenium2, and getting it to work with a framework. It's pretty daunting, but so worth it. If you don't know anything about Behat I suggest watching Knp University's video. It's a great overview of what Behat can do for you.

This is not a Behat tutorial

This post isn't to sell you on the idea of Behat or even a tutorial about how to use it. There are several other tutorials and videos about that. This post is about getting it up and running with Symfony in a Vagrant box. Even though I'm using Symfony, I'm fairly certain the steps can be modified slightly to accommodate any framework.

Prerequisites

I'm using a super simple Vagrant box that I've manually installed apache, php, composer, postgres, and Symfony on. Here's my Vagrantfile. I've also added a line to my mac's /etc/hosts file to point my project's url to my Vagrant box's ip.

Install via composer

First thing we need to do is to pull in the framework and some extensions via composer.

{
    "require": {
        "other-dependencies": "*",
        "behat/symfony2-extension": "*",
        "behat/mink-goutte-driver": "*",
        "behat/mink-selenium2-driver": "1.2.*@dev",
        "behat/mink-extension": "*",
        "behat/mink-browserkit-driver": "*"
    }
}

Install the dependencies, composer update.

Create a "test" entry point

Next, we need to create an entry point to ensure all our requests into the application are in a "test" environment. Let's create the entry point.

<?php // MySymfonyProject/Web/app_test.php

use Symfony\Component\HttpFoundation\Request;
use Symfony\Component\Debug\Debug;

$loader = require_once __DIR__.'/../app/bootstrap.php.cache';
Debug::enable();

require_once __DIR__.'/../app/AppKernel.php';

$kernel = new AppKernel('test', true);
$kernel->loadClassCache();
$request = Request::createFromGlobals();
$response = $kernel->handle($request);
$response->send();
$kernel->terminate($request, $response);

Now we can visit our app in the browser in the "test" environment by going to something like, http://myproject.dev/app_test.php/some_route. You should also create a test database to use with this new environment and configure Symfony to use it during testing.

Create the behat.yml file

It's time to create Behat's config file. Create the following file in the root of your app.

# MySymfonyProject/behat.yml
default:
    extensions:
        Behat\MinkExtension\Extension:
            default_session: symfony2
            goutte: ~
            base_url: 'http://myproject.dev/app_test.php'

        Behat\Symfony2Extension\Extension:
            mink_driver: true
            kernel:
                env: test

We've configured Behat to use the Goutte driver. Goutte is a web scraper library created by the creator of Symfony. It allows you to visit urls, click links, fill in forms, and more, all with php.

Init a bundle

Assuming you already have a bundle you're ready to test, you can initialize it for Behat testing with the following command, bin/behat --init @YourBundleName. This will create a Features directory in your bundle as well as a Features/Context/FeatureContext.php file which holds the context class. You'll want to change the FeatureContext class to extend Behat\MinkExtension\Context\MinkContext.

Now, you can start putting your .feature files in this directory. You can test the features in the bundle with the following command, bin/behat @YourBundleName.

What about javascript?

Everything we've done so far works great for pages that don't require javascript. Goutte is great for this but it has no means to test pages with javascript. To test pages with javascript you actually need a browser and something to use the browser to do your testing. This is where Selenium comes into play. In conjunction with Behat, Selenium can actually pop open a browser and commandeer it for performing your tests. It's pretty remarkable. That assumes that you've got everything installed on your host machine though because that's where your browser lives. We're using Vagrant though and Vagrant doesn't really have any sort of display for a browser. Obviously we can't use Selenium inside a Vagrant box...right? WRONG!

It's about to get exciting.

Set up the VM to run Selenium

Before we do any of this, let's update the list of available packages, sudo apt-get update.

Let's create a directory especially for Selenium, mkdir /var/selenium. cd into that directory and download Selenium with, wget http://selenium-release.storage.googleapis.com/2.40/selenium-server-standalone-2.40.0.jar.

We've got Selenium now, awesome! Except it's a .jar and we don't have java installed, not awesome. Oh well, let's get some java! sudo apt-get install openjdk-7-jre-headless.

We need a browser now and believe it or not, we can install Firefox in our VM. Do so with, sudo apt-get install firefox.

Because we're testing browser interaction and browsers require a display, we need some way to accommodate that in our VM. We can use Xvfb for this. Xvfb is a display server that performs graphical operations in memory, all without actually displaying anything. Perfect for our VM. Install it with, sudo apt-get install xvfb.

Phew...we're done installing stuff. Woo hoo!

Reconfigure

Now that we have all that pieces in place, we need to reconfigure Behat to take advantage of our new goodies. Update your behat.yml file like below:

# MySymfonyProject/behat.yml
default:
    extensions:
        Behat\MinkExtension\Extension:
            default_session: symfony2
            goutte: ~
            base_url: 'http://myproject.dev/app_test.php'
            javascript_session: selenium2
            browser_name: firefox
            selenium2: ~

        Behat\Symfony2Extension\Extension:
            mink_driver: true
            kernel:
                env: test

Create a test feature

To actually see if this works, create a route to a controller action that uses this template. All the page does is displays a button and uses jQuery to attach a click handler that when clicked, populates a div with "It works!".

Let's create a feature to test this.

# MySymfonyProject/src/YourProject/YourBundle/Features/selenium_test.feature
Feature: Selenium works
  In order to see if selenium works
  As a visitor
  I need to be able to use javascript

  @javascript
  Scenario: Testing selenium
    Given I am on "/your/route"
    And I press "Button"
    Then I should see "It works!"

Note the @javascript annotation. This, along with our new behat configuration, tells behat to use Selenium for this scenario. All scenarios without this annotation will continue to use Goutte.

Run it!

Before we run Behat, we need to start up Selenium. You can do that with DISPLAY=:1 xvfb-run java -jar /var/selenium/selenium-server-standalone-2.40.0.jar. I suggest creating an alias for this in a .bash_aliases file. You should eventually see something like below.

Mar 18, 2014 3:52:04 AM org.openqa.grid.selenium.GridLauncher main
INFO: Launching a standalone server
03:52:04.213 INFO - Java: Oracle Corporation 24.45-b08
03:52:04.214 INFO - OS: Linux 3.2.0-23-generic amd64
03:52:04.231 INFO - v2.40.0, with Core v2.40.0. Built from revision fbe29a9
03:52:04.356 INFO - Default driver org.openqa.selenium.ie.InternetExplorerDriver registration is skipped: registration capabilities Capabilities [{platform=WINDOWS, ensureCleanSession=true, browserName=internet explorer, version=}] does not match with current platform: LINUX
03:52:04.419 INFO - RemoteWebDriver instances should connect to: http://127.0.0.1:4444/wd/hub
03:52:04.421 INFO - Version Jetty/5.1.x
03:52:04.422 INFO - Started HttpContext[/selenium-server/driver,/selenium-server/driver]
03:52:04.423 INFO - Started HttpContext[/selenium-server,/selenium-server]
03:52:04.424 INFO - Started HttpContext[/,/]
03:52:27.029 INFO - Started org.openqa.jetty.jetty.servlet.ServletHandler@67db296e
03:52:27.030 INFO - Started HttpContext[/wd,/wd]
03:52:27.038 INFO - Started SocketListener on 0.0.0.0:4444
03:52:27.044 INFO - Started org.openqa.jetty.jetty.Server@709dd800

This takes a few seconds, so wait until you see that last line before moving on.

Now that Selenium is running, open up a new console tab in iTerm or PHPStorm and ssh back into your vagrant box to the root of your Symfony project.

What are you waiting for!? Run Behat! bin/behat @YourBundleName

Your tests should pass. Pat yourself on the back!

This is a super simple example, purely for testing whether or not all the pieces are working together. However, I'm sure you can dream up some scenarios where you need to be able to dynamically add some fields to your forms and test the submission, validation, creation, updating, etc. That's now possible with this setup.

Start testing what your users are actually going to use on your development/testing VM that uses the same stack that your production code will be running on. Now you can sleep even better at night! Enjoy!

Tips

You can kill selenium with Control + C in its terminal tab on a mac.

I highly recommend you peruse Sylius's Behat context classes and features. I've found them immensely useful to learn how to test javascript and use Symfony's services behind the scenes.