I opted to use one of the available MailChimp bundles to aid integration and with a bit of tweaking this worked great. The bundle comes with a stub connection which is able to collate requests sent and make them available for inspection. I wanted to be able to utilise this neatly inside a set of tests, and potentially also make it available in the dev environment. A bit of skimming the Symfony2 cookbook led me to the entry on Symfony2 data collectors.

The concept is simple – a data collector is registered and is able to, well, collect data! For example, the number of queries during a request, or the number of emails sent. This can then be accessed via the profiler in the relevant environment (dev, test etc). In my case I wanted to collect and subsequently inspect all requests sent to MailChimp.

First step – create me a data collector:

<?php
 
namespace Jirafe\Bundle\MailChimpBundle\DataCollector;
 
use Symfony\Component\HttpKernel\DataCollector\DataCollector;
use Symfony\Component\HttpFoundation\Request;
use Symfony\Component\HttpFoundation\Response;
use Jirafe\Bundle\MailChimpBundle\Connection\ConnectionInterface;
 
class MailChimpDataCollector extends DataCollector
{
    private $connection;
 
    public function __construct(ConnectionInterface $connection)
    {
        $this->connection = $connection;
    }
 
    public function collect(Request $request, Response $response, \Exception $exception = null)
    {
        $this->data = array(
            'requests'  => $this->connection->getRequests(),
        );
    }
 
    public function getRequestCount()
    {
        return count($this->data['requests']);
    }
 
    public function getRequests()
    {
        return $this->data['requests'];
    }
 
    public function getName()
    {
        return 'mail_chimp';
    }
}

The collect() method is the interesting part here – this is called whenever data collection is required – usually once per request. We collect the requests from the connection supplied to the constructor.

Next step, register this during the bundle’s load() method:

// Add connection to data collector
$definition = $container->getDefinition('mail_chimp.data_collector');
$definition->addArgument(new Reference(sprintf('mail_chimp.connection.%s', $config['connection'])));

The above uses the connection type specified in the bundle’s configuration option and previously registered as a service in the same method. If you’re creating your own custom collector, this is just the normal parameters you’d pass in to your service when it’s instantiated.

Finally, I added a method and subsequent implementation to the interface and related connection classes within the bundle, so that MailChimp request objects were collected in an array, and made available to the data collector upon request. This is the getRequests() method listed above in the data collector; called in the collect() method. Quick ‘n’ dirty here, since there would only ever be a single MailChimp request (possibly 2) made in any one request.

Once all this was in place, I could then test correctly. Note that I’d rather have done the other way round (test and then develop) but since this was experimentation, I figured it was OK ;-) In my Behat scenarios, I added the getSymfonyProfiler() method detailed in a Behat/Mink cookbook entry, and proceeded to write my step (Behat 1.x-style):

$steps->Then('/^my email address is unsubscribed from the relevant MailChimp list$/', function($world) {
 
    $profiler = $world->getSymfonyProfiler();
    if (!$profiler)
    {
      throw new \RuntimeException("Symfony2 profiler not present for tests, wrong driver?");
    }
    $mcDataCollector = $profiler->getCollector("mail_chimp");
    $listID = $world->container->getParameter("mailchimp_list_id");
 
    // Check the requests made to make sure we have an unsubscribe
    assertEquals(1, $mcDataCollector->getRequestCount());
 
    $requests = $mcDataCollector->getRequests();
    $req1 = $requests[0];
    assertEquals("listUnsubscribe", $req1->getMethod());
 
    $params = $req1->getParams();
    assertEquals($listID, $params["id"]);
 
    // Fire the redirect on the client
    $world->getSession()->getDriver()->getClient()->followRedirect();
});

The final line here is as a result of disabling redirects in a previous step since the data collector operates on a per-request basis. If you’re subscribing eg via a form POST, then you’ll need to disable redirects prior to submitting the form, otherwise your data collector will be empty when you check it. Disable them and then redirect after you’ve carried out your checks.

The implementation for the above is up on our Github MailChimpBundle fork here; I’m not particularly fond of having to hack about the connection classes, and ideally it would be nice to layer the data collector on top without having to adjust existing code. That said, it does the job fine here, and it’s good to use even more features of Symfony2 ;-)

Here is a quick code sample which shows how to test emails that are sent using Silex and Swiftmailer.

I would recommend using this example app to get Silex and Swiftmailer setup if you haven’t got a working setup.

The aim in the code is to stop sending of real email in the test environment and log the messages so we can test them.

Here is a quick gist of the Test class, testing for number of emails sent and the first emails subject.

I have copied out the Symfony2 MessageLogger.php class from the SwiftmailerBundle, which I’ve included in the gist for your convenience.

The beauty of BDD is that it’s written in plain everyday language, and this can be directly translated to code snippets. Therefore the client is happy working “in plain english”, and the developer can crack on with writing the actual code to test these parts.

We’ve chosen to use Symfony2 for the project (more on that in a separate post) and for the BDD aspect we’ve gone with Behat (coupled with Mink); both of which have bundles available for Symfony2. Essentially, BDD boils down to Features, consisting of a number of Scenarios, which in turn consist of a number of Steps. Here’s an example:

Feature: Login to the admin area
	As an administrator
	I want to be able to log into an admin area
	So I can administer MyApp

	Scenario: Joe Bloggs logs into the admin area
	  Given I visit the admin area
	  When I am shown the login form
	  And I click login with my correct credentials
	  Then I should be on /admin/home

This could translate to code such as the following:

$steps->Given('/^I visit the admin area$/', function($world) {
 
    $world->getSession()->visit($world->getPathTo("/admin"));
});
 
$steps->When('/^I am shown the login form$/', function($world) {
 
    $doc = $world->getSession()->getPage();
 
    assertTrue($doc->hasField("_username"));
    assertTrue($doc->hasField("_password"));
});
 
$steps->And('/^I click login with my correct credentials$/', function($world) {
 
    $doc = $world->getSession()->getPage();
 
    $doc->fillField("_username", "joe");
    $doc->fillField("_password", "bloggs");
    $doc->clickButton("Log in");
});
 
$steps->Then('/^(?:|I )should be on (?P<page>.+)$/', function($world, $page) {
 
    assertEquals(
        parse_url($world->getPathTo($page), PHP_URL_PATH),
        parse_url($world->getSession()->getCurrentUrl(), PHP_URL_PATH)
    );
});

The beauty of Behat is that the step definitions are reusable, meaning that if you stick to a standard way of describing your steps, the amount of code you need to write is vastly reduced, compared with normal functional TDD. Steps that aren’t implemented yet are given default code as per:

$steps->Given('/^I visit the admin area$/', function($world) {
    throw new \Behat\Behat\Exception\PendingException();
});

which will translate into the output shown as a step needing to be implemented. Behat also takes note of the Pending exception being thrown and automatically skips any steps following this in the scenario, notifying you of this at the end of the test run.

Behat also comes with a number of standard step definitions built-in, although these are somewhat buried in the code and not in the docs :-( So for example the following steps are part of the included ones:

I go to /my/url:
 
$steps->When('/^(?:|I )go to (?P<page>.+)$/', function($world, $page) {
    $world->getSession()->visit($world->getPathTo($page));
});

and

I fill in "username" with "foo":
 
$steps->When('/^(?:|I )fill in "(?P<field>[^"]*)" with "(?P<value>[^"]*)"$/', function($world, $field, $value) {
    $world->getSession()->getPage()->fillField($field, $value);
});

which translates as even less code to write. Bonus!

Tips ‘n’ tricks

The Mink (browser emulator abstraction layer) setup comes with a Symfony2 driver which is great for just testing normal usage. In our project however we’re integrating Facebook authentication using the FOS FacebookBundle, and we needed a way to test that the user was being bounced to Facebook in order to log in. The solution for this was simple – switch to using Goutte as the driver, as per the BehatBundle docs, and voila, you can test the current URL contains eg facebook.com or whatever. The actual testing of the authentication itself I’ll write up in a separate blog post as it’s a bit epic for here.

The documentation mentions using a separate front controller for testing, with a separate environment configuration. This is definitely recommended, particularly if you’re using Goutte. If you need services to be available, for example your entity manager, pop something like this in your env.php file:

$kernel = new AppKernel('test', true);
$kernel->loadClassCache();
$service = $kernel->getContainer()->get("your.service.here");

You can also use the env.php file for setting up fixtures that need to be reset for each Scenario that you create, eg pre-defined user states and so on.

Caveats

Unless you’re using Sahi, you obviously can’t test things such as Javascript popups effectively. I’m in two minds as to whether this should be handled by a new type of Exception which can be thrown e.g. a TestInRealBrowserException(). That way it could be caught and notified to the user running the tests, or flagged as part of an automated build process for manual testing.

Overall, getting into BDD has been a fun and rewarding experience. At times the lack of docs make it somewhat frustrating, but this is easily solved by a) digging into the code and b) probably me getting off my ass and submitting some docs back. The ease of which the steps can be written though is a massive time saver when writing your tests, coupled with the reusability of the code. Big thumbs-up from me!

(crossposted from Sebastian’s personal blog)

One of our clients needs to send emails with UTF-8 subject lines. Swift Mailer, which Symfony uses, makes this easy to do, but writing functional tests for it is another story. The problem is that, if non-ASCII characters are present in the subject line, Swift Mailer encodes them as required by RFC 2047 – for example, a string like “Transformación” turns into “=?utf-8?Q?Transformaci=C3=B3n?=”. How do we know that this is the right subject line?

There’s no immediately-available “convert this into a RFC 2047-compliant subject line” function, but we can get around this by making use of Swift_Message->toString and some choice preg functions.

First, create a dummy message containing the appropriate header. Swift_Message->toString() will contain our encoded header:

Next, we have to extract our encoded header – I’ve used preg_match() and a fairly simple regex.

Finally, now that the encoded header has been extracted, we need only wrap it in preg_quote() (because checkHeader() uses regex syntax) and test away.

Voila – now we can be confident that our test script can deal with Unicode email subjects.

So we decided it was time to perhaps investigate alternatives.  The obvious choice for Symfony would be to use Sismo, but sadly it hasn’t been released yet… Fabien, any clues as to a release date? ;-) It looks at the moment as if we’re stuck with a Java-based solution, so we decided to look at Hudson as an alternative.

Setting up Hudson was straightforward using Nicolas Perriault’s blog post on exactly what we were trying to achieve, linking in with our Subversion repository.  The configuration of the projects took a bit of faffing – for some reason, Hudson liked to check out the whole repository rather than just eg the trunk folder  as specified, so this screwed up some of the paths.  A bit of trial and error later and we were up and running with a build every 30mins if there are new commits, as this pretty graph shows… (using the jUnit XML output available in Symfony 1.4).  Total number of tests in blue, failures in red.

One thing to bear in mind if you’re getting test failures randomly, or perhaps always after a certain developer checks in (and it’s not a result of broken code) is to ensure that the permissions are correct for Symfony, as SVN likes to store permissions where it can.  The fix for this, apart from committing new permissions, is to add a build step in Hudson that carries out ./symfony project:permissions (Symfony 1.3+) shortly before the build.  This seems to solve it nicely, and explained the small red peaks in the graph above.

If you focus on Quality, through effective refactoring and test coverage, speed will be maintained in the long term. When the code base gets quite large, speed of change can be difficult when there is poor quality code and little test coverage to support the code base.

You would be fooling yourself if speed was more important.
[Adrian Richte]

This has got me thinking about Test Driven Development (TDD) all over again.

Writing tests upfront for projects always  seems like a massive over head, but as any project gets larger, and more functionality is added, without any tests you just wont know what effect those changes are having on the rest of the system.  To run tests and see them fail means you can deal with an unexpected corner case before a client discovers it, and you know they will.  This will save time in the long run, and will also leave you with happy clients.