Integration Testing I - Data Providers

Performing integration testing, with a more complex data provider


In the previous article in this series, we looked at how to use Codeception and PhpUnit to generate a code coverage report. We also saw how to interpret the report and navigate through it. Then we looked at using the report to improve the code coverage. In this one, we'll discuss integration testing.


MODX logo

This article assumes that you've installed and configured Codeception and PhpUnit and created the classes and unit tests described in the previous articles. The test files and classes are available at GitHub here.


Integration Testing

Unit testing tests the smallest units of your code that are testable. The units may work perfectly and pass all tests, but still not work properly with each other. Integration testing tests the interaction between different parts of your code. In many cases, integration testing is unit testing without mocks and stubs.

In several earlier articles, we used stubbed versions of our Validator class and used them to test our User class. For an integration test, let's use the actual Validator class to do much the same thing. We'll use our User4.class.php file again as the test class.

Remember that the User4 class expects to be passed a Validator object in its constructor. We'll simply pass a real instance of the Validator class rather than a stubbed or mocked version.

In the /build directory, issue this command:

codecept generate:test integration T6_User

The command above will create a _build/tests/integration directory if it doesn't already exist, and a file in that directory called T6_UserTest.php.


T6_UserTest.php File

Here's the code for the T6_UserTest.php file. You can paste it in (replacing everything that's there). It's also available at GitHub here.

<?php 
class T6_UserTest extends \Codeception\Test\Unit
{
    /**
     * @var \UnitTester
     */
    protected $tester;
    /** @var Validator $validator */
    protected $validator;
    protected $fields = array(
        'username' => 'BobRay',
        'email' => 'bobray@hotmail.com',
        'phone' => '218-456-1234'
    );

    protected function _before()
    {
        require_once 'C:\xampp\htdocs\addons\assets\mycomponents\testingproject\core\model\validator.class.php';
        require_once 'C:\xampp\htdocs\addons\assets\mycomponents\testingproject\core\model\user4.class.php';
        $this->validator = new Validator();
    }

    protected function _after()
    {
    }

    // tests
    public function testWorking()
    {
        assertTrue(true);
    }

    /** @throws Exception */
    public function testConstructorWithParams() {
        $validator = $this->validator;
        $user = new User4($validator, $this->fields);
        assertInstanceOf('User4', $user);
        assertEquals('BobRay', $user->get('username'));
        assertEquals('bobray@hotmail.com', $user->get('email'));
        assertEquals('218-456-1234', $user->get('phone'));
        assertFalse($user->hasErrors());
        $result = $user->get('unknown');
        assertTrue($user->hasErrors());
        assertNull($result);
        $errors = $user->getErrors();
        assertEquals(1, count ($errors));
        assertContains('Unknown Field: ' . 'unknown', $errors, print_r($errors, true));
        $user->clearErrors();
        assertFalse($user->hasErrors());
    }

    /** @throws Exception */
    public function testConstructorNoParams() {
        $validator = $this->make(Validator::class, $this->fields);;
        $user = new User4($validator);
        assertinstanceof('User4', $user);
        assertEquals('', $user->get('username'));
        assertEquals('', $user->get('email'));
        assertEquals('', $user->get('phone'));
        assertFalse($user->hasErrors());
    }


    /**
     * @dataProvider errorDataProvider
     * @param string $username
     * @param string $email
     * @param string $phone
     * @param bool $expectErrors -- are errors expected?
     * @param int $count -- number of expected errors
     */

      public function testErrors($username, $email, $phone, $expectErrors, $count) {
          $fields = array(
             'username' => $username,
             'email' => $email,
             'phone' => $phone,
          );
          $user = new User4($this->validator, $fields);
          assertInstanceOf('User4', $user);
          $errors = $user->getErrors();
          if ($expectErrors) {
              assertTrue($user->hasErrors(), print_r($errors, true));
          } else {
              assertFalse($user->hasErrors(), print_r($errors, true));
          }
          assertEquals($count, count($errors));

          if (! $this->validator->validateUsername($username)) {
              assertContains('Invalid username', $errors, print_r($errors, true));
          }
          if (!$this->validator->validateEmail($email)) {
              assertContains('Invalid email', $errors, print_r($errors, true));
          }
          if (!$this->validator->validateUsername($phone)) {
              assertContains('Invalid phone', $errors, print_r($errors, true));
          }
      }

      public function testSave() {
          $user = new User4($this->validator);
          assertTrue($user->save());
      }
    /**
     * @dataProvider errorDataProvider
     * @param string $username
     * @param string $email
     * @param string $phone
     * @param bool $expectErrors -- are errors expected?
     * @param int $count -- number of expected errors
     */

      public function testUnknownField($username, $email, $phone, $expectErrors, $count) {
          $count++; /* add one for unknown field */
          $fields = array(
              'username' => $username,
              'email' => $email,
              'phone' => $phone,
              'unknown' => 'unknownField',
          );
          $user = new User4($this->validator, $fields);
          assertInstanceOf('User4', $user);
          $errors = $user->getErrors();
          assertNotEmpty($errors);
          assertEquals($count, count($errors));
          assertContains('Unknown field', $errors);
      }


    public function ErrorDataProvider() {
        $validUsername = 'BobRay';
        $invalidUsernameTooShort = 'Bo';
        $invalidUsernameTooLong =
            'asdasdadasdasdasdadasdssss';
        $validEmail = 'bobray@hotmail.com';
        $invalidEmail = 'bobrayhotmail.com';
        $validPhone = '218-356-2345';
        $invalidPhone = '218x123x2345';

        /* Array: username, email, phone, expectErrors, numErrors */
        return array(
            /* All valid */
            array($validUsername,$validEmail,$validPhone, false, 0),

            /* All invalid */
            array($invalidUsernameTooLong, $invalidEmail, $invalidPhone, true, 3),
            array($invalidUsernameTooShort, $invalidEmail, $invalidPhone, true, 3),

            /* Invalid username only */
            array($invalidUsernameTooShort, $validEmail, $validPhone, true, 1),
            array($invalidUsernameTooLong, $validEmail, $validPhone, true, 1),

            /* Invalid Email only */
            array($validUsername, $invalidEmail, $validPhone, true, 1),

            /* Invalid Phone only */
            array($validUsername, $validEmail, $invalidPhone, true, 1),

            /* Invalid username and invalid email */
            array($invalidUsernameTooLong, $invalidEmail, $validPhone, true, 2),
            array($invalidUsernameTooShort, $invalidEmail, $validPhone, true, 2),

            /* Invalid username and invalid phone */
            array($invalidUsernameTooLong, $validEmail, $invalidPhone, true, 2),
            array($invalidUsernameTooShort, $validEmail, $invalidPhone, true, 2),

            /* Invalid email and phone */
            array($validUsername, $invalidEmail, $invalidPhone, true, 2),
        );
    }

    public function save() {
        return true;
    }
}

Changes

We took out the two use statements at the top because we're not stubbing or mocking anything. We pass an actual instance of the Validator class to the User4 constructor.

In order to improve the code coverage report, we've added a test of the save() method. We've also added test of the get() method with an unknown field.

The biggest change, though, is in the new data provider code. In the previous test, we had lengthy test code in one method to test all the possible combinations of good and bad User fields. In this one, we've moved the fields themselves to the ErrorDataProvider() method at the end of the code. We use the same data provider for two of our test methods.


Data Provider Review

We saw how data providers work in a previous article, but this one is a little more complex, so let's review it.

Data providers work the same in Codeception and PhpUnit. An annotation is used to specify the data provider method to be used by a given test method:

/** @dataProvider errorDataProvider */
public function testErrors($username, $email, $phone, $expectErrors, $count) {...}

The lines above basically say to get the array returned by the errorDataProvider() function, and run the testErrors() method once for each row in the array, using the fields in that row to provide the parameters for the testErrors() method.

For example, here are a few rows of the array returned by errorDataProvider():

/* Invalid username and invalid email */
array($invalidUsernameTooLong, $invalidEmail, $validPhone, true, 2),
array($invalidUsernameTooShort, $invalidEmail, $validPhone, true, 2),

The first three values in each row are (in order) the username, email, and phone. Each of those three is either valid or invalid. The next value tells whether creating the user object should result in an error. The final value is the number of errors expected. Since both the username, and the email are invalid, we expect exactly 2 errors for each row.

Of course the process doesn't know which values are which. For each pass through the array returned by the data provider, it takes the first value in the row as the first parameter for the test method, the second value as the second parameter, and so on. You can have as many columns in the array as you like, as long as they are matched by parameters of the test function.


The testErrors() Method

Let's take a closer look at the testErrors() test.

This method will run once for each row of the array returned from the data provider. The parameters will be set from the values in that row of the array.

public function testErrors($username, $email, $phone, $expectErrors, $count) {
    $fields = array(
     'username' => $username,
     'email' => $email,
     'phone' => $phone,
    );
    $user = new User4($this->validator, $fields);
    assertInstanceOf('User4', $user);
    $errors = $user->getErrors();
    if ($expectErrors) {
      assertTrue($user->hasErrors(), print_r($errors, true));
    } else {
      assertFalse($user->hasErrors(), print_r($errors, true));
    }
    assertEquals($count, count($errors));

    if (! $this->validator->validateUsername($username)) {
      assertContains('Invalid username', $errors, print_r($errors, true));
    }
    if (!$this->validator->validateEmail($email)) {
      assertContains('Invalid email', $errors, print_r($errors, true));
    }
    if (!$this->validator->validateUsername($phone)) {
      assertContains('Invalid phone', $errors, print_r($errors, true));
    }
}

In the code above, first we set up the $fields array using the first three parameters. Then we create the $user object using those fields.

If the $expectErrors parameter is true, we're expecting errors, so calling $user->hasErrors() should return true. If we're not expecting errors, it should return false.

Next, we verify that the number of errors returned by $user->getErrors is equal to the $count parameter

Finally, we check each of the three user fields directly with our Validator class. It they're invalid, we check to see if the proper error message has been set in the $errors array.

Now, let's look at the testUnknownField() method. Notice that it uses the same data provider as the testErrors() method we looked at above. Since we already have a test set that includes all the possible valid and invalid combinations of the three User class fields, we can use it again here just by adding an unknown field to the array of fields sent to the class constructor.

public function testUnknownField($username, $email, $phone, $expectErrors, $count) {
    $count++; /* add one for unknown field */
    $fields = array(
      'username' => $username,
      'email' => $email,
      'phone' => $phone,
      'unknown' => 'unknownField',
    );
    $user = new User4($this->validator, $fields);
    assertInstanceOf('User4', $user);
    $errors = $user->getErrors();
    assertNotEmpty($errors);
    assertEquals($count, count($errors));
    assertContains('Unknown field', $errors);
}

In the first line, the code above will be run once for each row in the array returned by the data provider. For each row, we set up the array of three fields from the first three parameters, then add an unknown field to them at the end of the $fields array.

We bump up the $count parameter by one because we're adding one more error with the unknown field.

We create the user, make sure there are errors (since every instance has an unknown field, there will always be errors). We make sure the number of errors equals $count + 1 and that the 'Unknown Field' message is in the error array.

We're not using the $expectErrors parameter here at all because we know there will always be errors, but it still has to be there to preserve the match with the array members coming from the data provider.

Notice that we're using assertContains() with a string and an array here rather than two strings, which is more common. When using assertContains() with an array, the array must always be the second parameter, or it will throw an error. In this case, the assertion tests whether the string in the first parameter is equal to any value in the array. The match must be perfect, and it's case-sensitive.


Coming Up

In the next article, we'll take a look at setting up Codeception to do integration testing in the MODX CMS.



For more information on how to use MODX to create a web site, see my web site Bob's Guides, or better yet, buy my book: MODX: The Official Guide.

Looking for high-quality, MODX-friendly hosting? As of May 2016, Bob's Guides is hosted at A2 hosting. (More information in the box below.)



Comments (0)


Please login to comment.

  (Login)