Acceptance Testing XVII - Codeception Environments

Using Codeception environments to alter the way tests are run


In the previous article, we created the final version of our ResourceProtection test. It used the page and step objects we created earlier to call Codeception's commands to navigate through the MODX Manager and perform the tests. In this one, we'll use that same test, but run it in multiple browsers.


MODX logo

This article assumes that you've installed and configured Composer, Codeception, PhpUnit, MODX, Java, WebDriver, and ChromeDriver as described in earlier articles, and that you've created the acceptance test support files from those articles.


Environments

Environments in Codeception allow you to run your tests under different conditions. The most common use of Environments is to run the same test in multiple browsers. We'll be doing that in this article.

In order to tell Codeception which environment you want to use, you just add this option at the end of the command line: --env environment_name. So if you wanted to run a test with Chrome, you'd do something like this:

    codecept run acceptance SomeTestName --env chrome
    

To do the same thing in PhpStorm, you'd edit the Run/Debug configuration for your test and put --env chrome in the "Test Runner options input."

Of course, in order for Codeception to know what to do with that --env chrome command, you need to create the environment code first. There are two ways to create multiple environments in Codeception.

The first method is to create a section in the suite .yml file (e.g. acceptance.suite.yml) for each environment. That's what we'll do in this article. The environment-specific code can go in any of the suite .yml files, but it's most common in acceptance tests, so we'll use the example of tests/acceptance.suite.yml.

The second method is to create a separate .yml file for each environment. We'll do that in the next article.

In both cases, the environment-specific code overrides the code that comes before it, but only when the code in the environment section matches the code it's overriding. You can think of it like a PHP array, where assigning a new value to an array member overrides any value set earlier.

When you put the environment code in your acceptance.suite.yml file, it should go at the bottom, so it can override the previous code. When the code is in separate files, it will be loaded just after the acceptance.suite.yml file.

Let's look at an example showing how the environment code works. Imagine that you have this in part of your acceptance.suite.ymlfile:

modules:
    enabled:
        - \Helper\Acceptance
        - WebDriver
        - Asserts

    config:
        WebDriver:
            url: 'http://localhost/test/'
            browser: 'chrome'
            window_size: 'maximize'

Suppose that you add this code to the bottom of the acceptance.suite.php file and are running with the environment set to firefox (--env chrome):

env:
    firefox:
        modules:
            config:
                WebDriver:
                    url: 'http://localhost/test/'
                    path: ''
                    browser: 'firefox'
                    window_size: 'maximize'
                    wait: 5
        extensions:
            enabled:
                - Codeception\Extension\RunProcess:
                    - echo "Launching Selenium Server in acceptance.suite.php"
                    - java -jar c:/Users/BobRay/Downloads/Selenium/selenium-server-standalone-3.141.59.jar
                    - echo "Launching geckodriver in acceptance.suite.php"
                    - C:/Users/BobRay/Downloads/selenium-drivers/geckodriver.exe

    chrome:
        modules:
            config:
                WebDriver:
                    url: 'http://localhost/test/'
                    browser: 'chrome'
                    window_size: 'maximize'
                    wait: 5
        extensions:
            enabled:
                - Codeception\Extension\RunProcess:
                    - echo "Launch Selenium server in acceptance.suite.yml"
                    - java -jar c:/Users/BobRay/Downloads/Selenium/selenium-server-standalone-3.141.59.jar
                    - echo "Launch Chrome driver in acceptance.suite.yml"
                    - C:/Users/BobRay/Downloads/selenium-drivers/chromedriver.exe

The code above will change the browser used by WebDriver to firefox and load the geckodriver driver for Firefox (overriding the browser set in the code above it). It will also set the wait variable (not set above) to 5 seconds. The URL is unchanged because we didn't override it in the lower section file.

Notice the "root" variable from the earlier code (the one at the left margin — modules:). The settings we want to change are nested under it. It's indented two levels in the env: section, not at the left margin. Notice, too, that other than that change, we duplicate the indentation levels used above exactly (more on this in a bit).

If no environment is set, all the code under the env: label is ignored. If --env firefox is set, only the firefox section under env: is applied (after the code above the env: label is processed. If --env chrome is set, the chrome section is applied.


acceptance.suite.yml File

Before going any further, we need to update our acceptance.suite.php file to include an env: section. Note that after this update, all our earlier acceptance tests will still run. Since they don't specify an environment, the environments section of the code will be ignored unless you specify an environment that's listed in the file).


The Code

Here's the entire content of our updated acceptance.suite.yml file (also available at GitHub here). Replace the entire content of the acceptance.suite.yml file with this code:

The last two environments listed (modx2 and modx3 will be used in, not the next article (which is about using separate .yml files), but in the one after that, where we'll make our test run in MODX 3.

# Codeception Test Suite Configuration
#
# Suite for acceptance tests.

actor: AcceptanceTester

bootstrap: _bootstrap.php

modules:
    enabled:
        - \Helper\Acceptance
        - WebDriver
        - Asserts

    config:
        WebDriver:
           url: 'http://localhost/test/'
           browser: 'chrome'
           window_size: 'maximize'
           wait: 5

extensions:
    enabled:
        - Codeception\Extension\RunProcess:
            - echo "Launch Selenium in acceptance.suite.php"
            - java -jar c:/Users/BobRay/Downloads/Selenium/selenium-server-standalone-3.141.59.jar
            - echo "Launch Chrome driver in acceptance.suite.php"
            - C:/Users/BobRay/Downloads/selenium-drivers/chromedriver.exe

################################################################
# The code below will be ignored if no Environment is specified
################################################################

env:
    firefox:
        modules:
            config:
                WebDriver:
                    url: 'http://localhost/test/'
                    path: ''
                    browser: 'firefox'
                    window_size: 'maximize'
                    wait: 5
        extensions:
            enabled:
                - Codeception\Extension\RunProcess:
                    - echo "Launching Selenium Server in acceptance.suite.php"
                    - java -jar c:/Users/BobRay/Downloads/Selenium/selenium-server-standalone-3.141.59.jar
                    - echo "Launching geckodriver in acceptance.suite.php"
                    - C:/Users/BobRay/Downloads/selenium-drivers/geckodriver.exe

    chrome:
        modules:
            config:
                WebDriver:
                    url: 'http://localhost/test/'
                    browser: 'chrome'
                    window_size: 'maximize'
                    wait: 5
        extensions:
            enabled:
                - Codeception\Extension\RunProcess:
                    - echo "Launch Selenium server in acceptance.suite.yml"
                    - java -jar c:/Users/BobRay/Downloads/Selenium/selenium-server-standalone-3.141.59.jar
                    - echo "Launch Chrome driver in acceptance.suite.yml"
                    - C:/Users/BobRay/Downloads/selenium-drivers/chromedriver.exe

    modx2:
        bootstrap: _bootstrap2.php

        modules:
            config:
                WebDriver:
                    url: 'http://localhost/test/'

    modx3:
        bootstrap: _bootstrap3.php

        modules:
            config:
                WebDriver:
                    url: 'http://localhost/modx3/'

As you can see, the part before the big comment is pretty much the same as our old acceptance.suite.yml code. Below it are the sections that will be read and used if you specify one or more of the environments.

The environment sections will override the browser and the browser driver that were set earlier in the file. The Firefox section also adds the path: '' setting, which is required for Firefox, but breaks Chrome.

Notice that in the env: section the code is indented by two more levels than the matching code above. Everything indented under the each specified environment will be moved to the left when it's applied.

For example, with this code:

env:
    firefox:
        modules:

The modules: line will be moved to the root (all the way left), as it is in the code it's overriding.

Be careful to use env: here, not envs: which would seem more natural, but will create all kinds of weirdness when your tests run. That mistake cost me two days of head-banging frustration. It's a logical mistake, not only for grammatical reasons, but the path: command that specifies the directory for environment files uses /envs/.

Also, the overriding code must match any earlier code exactly. So when you modify the paths and URLs for your own setup, be sure to modify them the same way in both the earlier code and in the env: section, otherwise, Selenium or the browser driver may be launched twice. The echo statements in the code will tell you if that's happening.


Trying it Out

Once you've updated the acceptance.suite.yml file, you should be able to run the latest test in either Chrome or Firefox by using one of these commands in the _build directory (or with same environment specification in the Test Runner options section of the Run/Debug configuration in PhpStorm):

codecept run acceptance T17_ResourceProtectionFinalCest --env chrome
codecept run acceptance T17_ResourceProtectionFinalCest --env firefox

In fact, you should be able to run all of the acceptance tests in either browser like this:

codecept run acceptance --env chrome
codecept run acceptance --env firefox

Whats more, you can run a test, or all tests, in *both* browsers in a single run by adding --env chrome --env firefox to the end of the command line.

MODX 3

You probably noticed the modx3 environment in the code. If so, you might be tempted to try a run specifying that environment, but doing so could have unpredictable results since we haven't updated the test itself for MODX 3. We'll do that in the article after the next one.


Coming Up

In the next articles, we'll look at running our test in multiple browsers using Codeception environments stored in separate files, rather than in the acceptance.suite.yml file.



For more information on how to use MODX to create a web site, see my web site Bob's Guides, or better yet, buy my book: MODX: The Official Guide.

Looking for high-quality, MODX-friendly hosting? As of May 2016, Bob's Guides is hosted at A2 hosting. (More information in the box below.)



Comments (0)


Please login to comment.

  (Login)