Acceptance Testing I - Hello World

Our first Acceptance test

In the previous article, we looked briefly at Functional testing. In this one we'll get our first look at Acceptance testing.

MODX logo

This article assumes that you've installed and configured Composer, Codeception and PhpUnit.

Acceptance Testing

As we said in previous articles, the Codeception Acceptance tester is like a robot that pretends to be you, navigates through pages, testing as it goes. In this article, we'll create a simple acceptance test opens a web page and that loads a form-based web page, fills out a form, submits it, and checks the result.

Getting Ready for Acceptance Testing

There are several steps to perform before we can run an Acceptance test in Codeception. First, we need to install Codeception's WebDriver module. WebDriver is a W3C protocol for automatic operation of a browser. Install the module with this command in the web root of your project:

composer require --dev codeception/module-webdriver

Now that we have the WebDriver module installed, we need a server to do the work and a driver for each individual browser we want to test with.

There are a number of servers you can use to run acceptance tests. The gold standard for automated testing is the Selenium Server. The Selenium folks developed the original WedDriver software and Selenium is the most reliable platform for automated testing. It's also the one recommended by Codeception.

There are some other server choices and configuration options described in the Codeception Docs. I've played around with most of them, but compared to the Selenium driver, they are less capable, less reliable, and their configuration often requires a lot of trial and error. Most of them don't support all of the WebDriver Protocol, and some of them don't support actions that involve JavaScript. These articles assume that you're using the Selenium Server.

Installing the Selenium Server

In order to run, the Selenium Server requires Java to be installed. Download the appropriate installer for your platform directly from Oracle at the Java Downloads page. (If you use PhpStorm, you already have Java installed.)

You'll either need to put the Java executable somewhere in your current path, or modify the path to include its location. After finishing the installation, type java at the command line from somewhere in your testing project. If you see a list of Java commands, you're in business.

Now, you need to install the Selenium Server.

You can download the latest stable version of the Selenium Standalone Server at the Selenium Download page. The file you download ends in .jar, which means that it's an executable Java program.

The Selenium Server must be launched before executing any Acceptance tests, but there's a way to launch it automatically in your acceptance.yml file. We'll see how to do that in just a bit. The Selenium executable can be in your path, but it's easy to give the full path to the executable Selenium file (or create an alias for it).

Browser Driver

For Acceptance tests, I use the ChromeDriver with the Chrome browser. You're welcome to try any of the other browser drivers listed in the Codeception Acceptance Test docs, but I've had the best luck with Chrome. The last time I checked, I couldn't get the Firefox driver (GeckoDriver) to work, though that was some time ago.

To get ChromeDriver, go to the ChromeDriver Home Page. Near the top, you'll see a download link for the latest stable version. Click on it and download the appropriate version for your platform. Unzip it to get the ChromeDriver executable. It can go anywhere, but make a note of the path to it. You'll need it for the acceptance.yml file.


The configuration file for your Acceptance tests is the acceptance.suite.yml file in the _build/tests/ directory. Create it if it's not there already. Here's the full content of that file on my system:

# Codeception Test Suite Configuration
# Suite for acceptance tests.
# Perform tests in browser using the WebDriver or PhpBrowser.
# If you need both WebDriver and PHPBrowser tests - create a separate suite.

actor: AcceptanceTester
        - \Helper\Acceptance
        - WebDriver
        - Asserts
            url: 'http://localhost/test/'
            browser: 'chrome'
            # browser: 'firefox'
            window_size: 'maximize'
            wait: 5
            # window_size: false
               # chromeOptions:
                    #args: ["--headless", "--disable-gpu", "--disable-extensions"]

        - Codeception\Extension\RunProcess:
            - java -jar c:/Users/BobRay/Downloads/Selenium/selenium-server-standalone-3.9.0.jar
            - C:/Users/BobRay/Downloads/selenium-drivers/chromedriver.exe
            # -  C:/Users/BobRay/Downloads/selenium-drivers/geckodriver.exe

The lines that start with # are commented out. You'll have to correct the paths in the extensions: section and refer to your own version of the Selenium server.

If you're using Firefox, you need to uncomment the two firefox lines and comment out the two chrome lines.

The extensions: code will launch both the Selenium Driver and your browser driver before executing any of your Acceptance tests.

The url: line under the WebDriver section should point to the web root of your MODX install. Whenever you have an amOnPage() call, it will be prefixed with that path.

Important: Note that amOnPage() is not really a test. It's a navigation command. Whenever Codeception sees this, it sends a new request for that page. It it's the page you're already on, the page will reload. Occasionally, you need that to make a test work. For example, if you're trying to switch tabs on a page with multiple tabs, I've found that a page reload is sometimes necessary to make the tab switch work.

The wait: 5 in the same section tells Codeception/WebDriver to wait for 5 seconds before failing on a test that needs to see something on the page. You can experiment with shorter durations. The more complex the page you're going to, the longer the wait needs to be to make the tests reliable. I assume that if the element or text is found in less than 5 seconds, it stops waiting, but I've never found a reliable way to test this.

You'll notice the commented out "headless" stuff at the end of the webdriver: configuration section. Running headless means there's no browser display. Acceptance tests can be fairly slow, so I expected that running headless would speed things up. It took me three days to get the headless configuration right, and then found that it was actually slightly slower. This might have changed since I tested it, so feel free to try it out.

Creating Our Test

Create the test file with this command in the _build directory:

codecept generate:cest acceptance T11_HelloWorld

Notice that we've used generate:cest. The cest format is what we'll be using for our Acceptance tests. The resulting file will be T11_HelloWorldCest.php.

Here's the code for our test (also available at GitHub, here). Replace everything in the file with this code:


class T11_HelloWorldCest
     * @var \AcceptanceTester
    public $I;

    protected function _before()

    protected function _after()

    // tests
    public function testSomeFeature(\AcceptanceTester $I)
        $I->see('Hello World');

The $I->wait(3) introduces a 3-second delay so that you have time to see the text on the page. This is often helpful if a test is not working because it lets you get a better look at what's happening. Any unnecessary waits can be removed once a test is working properly.

Running the Test

Try running the T11_HelloWorldCest.php file with this command in the _build directory:

codecept run acceptance T11_HelloWorldCest

You should see a browser window open, then after a delay, a failure (along with some text from your MODX site's home page). Codeception didn't see the text "Hello World" because we haven't created that page yet, but if you saw the browser window, your configuration is correct.

If the browser window doesn't show up, double-check the paths and URL in the acceptance.suite.yml file and make sure the Java executable is in your path and specified correctly .

Hello World

Create a page at the root of your MODX install called helloworld.html with this code:

<p>Hello World</p>

Run the test again, you should see this:

OK (1 test, 1 assertion)

If you do, you've just performed your first successful acceptance test.

More Info

We'll be seeing a lot more techniques for Acceptance testing with Codeception in upcoming articles, but you might want to take a look at the Codeception docs.

There's more information on performing Codeception Acceptance tests on this page of the Codeception docs.

You can find a lot of information on configuring and using WebDriver on this page of the Codeception docs.

Coming Up

In the next article, we'll take a look at acceptance testing in MODX.

For more information on how to use MODX to create a web site, see my web site Bob's Guides, or better yet, buy my book: MODX: The Official Guide.

Looking for high-quality, MODX-friendly hosting? As of May 2016, Bob's Guides is hosted at A2 hosting. (More information in the box below.)

      Previous Article << —— >>

Comments (0)

Please login to comment.