I opted to use one of the available MailChimp bundles to aid integration and with a bit of tweaking this worked great. The bundle comes with a stub connection which is able to collate requests sent and make them available for inspection. I wanted to be able to utilise this neatly inside a set of tests, and potentially also make it available in the dev environment. A bit of skimming the Symfony2 cookbook led me to the entry on Symfony2 data collectors.

The concept is simple – a data collector is registered and is able to, well, collect data! For example, the number of queries during a request, or the number of emails sent. This can then be accessed via the profiler in the relevant environment (dev, test etc). In my case I wanted to collect and subsequently inspect all requests sent to MailChimp.

First step – create me a data collector:

<?php
 
namespace Jirafe\Bundle\MailChimpBundle\DataCollector;
 
use Symfony\Component\HttpKernel\DataCollector\DataCollector;
use Symfony\Component\HttpFoundation\Request;
use Symfony\Component\HttpFoundation\Response;
use Jirafe\Bundle\MailChimpBundle\Connection\ConnectionInterface;
 
class MailChimpDataCollector extends DataCollector
{
    private $connection;
 
    public function __construct(ConnectionInterface $connection)
    {
        $this->connection = $connection;
    }
 
    public function collect(Request $request, Response $response, \Exception $exception = null)
    {
        $this->data = array(
            'requests'  => $this->connection->getRequests(),
        );
    }
 
    public function getRequestCount()
    {
        return count($this->data['requests']);
    }
 
    public function getRequests()
    {
        return $this->data['requests'];
    }
 
    public function getName()
    {
        return 'mail_chimp';
    }
}

The collect() method is the interesting part here – this is called whenever data collection is required – usually once per request. We collect the requests from the connection supplied to the constructor.

Next step, register this during the bundle’s load() method:

// Add connection to data collector
$definition = $container->getDefinition('mail_chimp.data_collector');
$definition->addArgument(new Reference(sprintf('mail_chimp.connection.%s', $config['connection'])));

The above uses the connection type specified in the bundle’s configuration option and previously registered as a service in the same method. If you’re creating your own custom collector, this is just the normal parameters you’d pass in to your service when it’s instantiated.

Finally, I added a method and subsequent implementation to the interface and related connection classes within the bundle, so that MailChimp request objects were collected in an array, and made available to the data collector upon request. This is the getRequests() method listed above in the data collector; called in the collect() method. Quick ‘n’ dirty here, since there would only ever be a single MailChimp request (possibly 2) made in any one request.

Once all this was in place, I could then test correctly. Note that I’d rather have done the other way round (test and then develop) but since this was experimentation, I figured it was OK ;-) In my Behat scenarios, I added the getSymfonyProfiler() method detailed in a Behat/Mink cookbook entry, and proceeded to write my step (Behat 1.x-style):

$steps->Then('/^my email address is unsubscribed from the relevant MailChimp list$/', function($world) {
 
    $profiler = $world->getSymfonyProfiler();
    if (!$profiler)
    {
      throw new \RuntimeException("Symfony2 profiler not present for tests, wrong driver?");
    }
    $mcDataCollector = $profiler->getCollector("mail_chimp");
    $listID = $world->container->getParameter("mailchimp_list_id");
 
    // Check the requests made to make sure we have an unsubscribe
    assertEquals(1, $mcDataCollector->getRequestCount());
 
    $requests = $mcDataCollector->getRequests();
    $req1 = $requests[0];
    assertEquals("listUnsubscribe", $req1->getMethod());
 
    $params = $req1->getParams();
    assertEquals($listID, $params["id"]);
 
    // Fire the redirect on the client
    $world->getSession()->getDriver()->getClient()->followRedirect();
});

The final line here is as a result of disabling redirects in a previous step since the data collector operates on a per-request basis. If you’re subscribing eg via a form POST, then you’ll need to disable redirects prior to submitting the form, otherwise your data collector will be empty when you check it. Disable them and then redirect after you’ve carried out your checks.

The implementation for the above is up on our Github MailChimpBundle fork here; I’m not particularly fond of having to hack about the connection classes, and ideally it would be nice to layer the data collector on top without having to adjust existing code. That said, it does the job fine here, and it’s good to use even more features of Symfony2 ;-)

Yesterday I was privileged to be involved in a user testing and feedback session that they were running down in Brighton. Aarron Walter and Stephanie Troeth from MailChimp came down, and ran an hour’s session with me where we went through various aspects of the MailChimp interface, features that were available (including ones I wasn’t aware of), and various usability points were addressed. I felt it was incredibly beneficial to both sides – MailChimp, so that they can understand what works and what doesn’t for users, and from my perspective seeing different features and having the opportunity to raise problems I had with usability. Too many times I’ve used a product which feels unfriendly to use, that feels like it was designed for hardcore non-typical users of the product, or sometimes just “designed by developers”. And we know how that turns out… So it was great to be involved in contributing to any potential user interface changes; it definitely makes you feel more valued as a customer.

Aside from the obvious part of testing a user interface, it was good during the session to learn about different features that people maybe aren’t aware of – there’s a large number of different integrations available in the admin area – everything from Google Analytics (tracks MailChimp campaigns being sent out against page views, e-commerce goals etc, all inside your admin area) to Freshbooks (import contacts etc). I also discovered the MailChimp integrations directory – very cool. One very good point Aarron also raised was how I found out about new features – for example, just clicking around in the admin area through to email and Twitter updates from MailChimp etc. There’s no point having new features available if you don’t shout about them…!

Overall – more than happy to carry on recommending them where suitable – a company that listens to its users AND provides a great service too. And maybe the most interesting fact of the session? Learning Freddie the chimp’s full name…. not sure how common knowledge that is, so maybe I’ll keep that a secret for now ;-)

The beauty of BDD is that it’s written in plain everyday language, and this can be directly translated to code snippets. Therefore the client is happy working “in plain english”, and the developer can crack on with writing the actual code to test these parts.

We’ve chosen to use Symfony2 for the project (more on that in a separate post) and for the BDD aspect we’ve gone with Behat (coupled with Mink); both of which have bundles available for Symfony2. Essentially, BDD boils down to Features, consisting of a number of Scenarios, which in turn consist of a number of Steps. Here’s an example:

Feature: Login to the admin area
	As an administrator
	I want to be able to log into an admin area
	So I can administer MyApp

	Scenario: Joe Bloggs logs into the admin area
	  Given I visit the admin area
	  When I am shown the login form
	  And I click login with my correct credentials
	  Then I should be on /admin/home

This could translate to code such as the following:

$steps->Given('/^I visit the admin area$/', function($world) {
 
    $world->getSession()->visit($world->getPathTo("/admin"));
});
 
$steps->When('/^I am shown the login form$/', function($world) {
 
    $doc = $world->getSession()->getPage();
 
    assertTrue($doc->hasField("_username"));
    assertTrue($doc->hasField("_password"));
});
 
$steps->And('/^I click login with my correct credentials$/', function($world) {
 
    $doc = $world->getSession()->getPage();
 
    $doc->fillField("_username", "joe");
    $doc->fillField("_password", "bloggs");
    $doc->clickButton("Log in");
});
 
$steps->Then('/^(?:|I )should be on (?P<page>.+)$/', function($world, $page) {
 
    assertEquals(
        parse_url($world->getPathTo($page), PHP_URL_PATH),
        parse_url($world->getSession()->getCurrentUrl(), PHP_URL_PATH)
    );
});

The beauty of Behat is that the step definitions are reusable, meaning that if you stick to a standard way of describing your steps, the amount of code you need to write is vastly reduced, compared with normal functional TDD. Steps that aren’t implemented yet are given default code as per:

$steps->Given('/^I visit the admin area$/', function($world) {
    throw new \Behat\Behat\Exception\PendingException();
});

which will translate into the output shown as a step needing to be implemented. Behat also takes note of the Pending exception being thrown and automatically skips any steps following this in the scenario, notifying you of this at the end of the test run.

Behat also comes with a number of standard step definitions built-in, although these are somewhat buried in the code and not in the docs :-( So for example the following steps are part of the included ones:

I go to /my/url:
 
$steps->When('/^(?:|I )go to (?P<page>.+)$/', function($world, $page) {
    $world->getSession()->visit($world->getPathTo($page));
});

and

I fill in "username" with "foo":
 
$steps->When('/^(?:|I )fill in "(?P<field>[^"]*)" with "(?P<value>[^"]*)"$/', function($world, $field, $value) {
    $world->getSession()->getPage()->fillField($field, $value);
});

which translates as even less code to write. Bonus!

Tips ‘n’ tricks

The Mink (browser emulator abstraction layer) setup comes with a Symfony2 driver which is great for just testing normal usage. In our project however we’re integrating Facebook authentication using the FOS FacebookBundle, and we needed a way to test that the user was being bounced to Facebook in order to log in. The solution for this was simple – switch to using Goutte as the driver, as per the BehatBundle docs, and voila, you can test the current URL contains eg facebook.com or whatever. The actual testing of the authentication itself I’ll write up in a separate blog post as it’s a bit epic for here.

The documentation mentions using a separate front controller for testing, with a separate environment configuration. This is definitely recommended, particularly if you’re using Goutte. If you need services to be available, for example your entity manager, pop something like this in your env.php file:

$kernel = new AppKernel('test', true);
$kernel->loadClassCache();
$service = $kernel->getContainer()->get("your.service.here");

You can also use the env.php file for setting up fixtures that need to be reset for each Scenario that you create, eg pre-defined user states and so on.

Caveats

Unless you’re using Sahi, you obviously can’t test things such as Javascript popups effectively. I’m in two minds as to whether this should be handled by a new type of Exception which can be thrown e.g. a TestInRealBrowserException(). That way it could be caught and notified to the user running the tests, or flagged as part of an automated build process for manual testing.

Overall, getting into BDD has been a fun and rewarding experience. At times the lack of docs make it somewhat frustrating, but this is easily solved by a) digging into the code and b) probably me getting off my ass and submitting some docs back. The ease of which the steps can be written though is a massive time saver when writing your tests, coupled with the reusability of the code. Big thumbs-up from me!

First of all, my stored functions are in separate .sql files in my plugin’s data/sql/ directory in the interests of following convention within the project’s directory structure.

Secondly, in myPlugin/config/MyPluginConfiguration.class.php file, I have the following for the initialize() method.:

public function initialize()
{
  sfConfig::set("zz_myplugin_dir", $this->getRootDir());
  $this->dispatcher->connect("command.post_command", array($this, 'initializeStoredFuncs'));
}

We listen for the command.post_command event which we can then check to see if it’s a Doctrine insert-sql task being run. We also check to see if the function already exists before attempting to create it – if not, then carry on.

public static function initializeStoredFuncs(sfEvent $event)
{
  static $doneConfig = false;
 
  if ($doneConfig)
  {
    // Already done it.
    return;
  }
 
  if (!($event->getSubject() instanceof sfDoctrineInsertSqlTask))
  {
    // Not inserting any SQL. No point continuing
    return;
  }
 
  // Get database connection
  $con = Doctrine_Manager::connection();
 
  // Configure function 1
  self::addStoredRoutine($con, "myFuncName1", "myFuncName1.sql");
 
  // Configure function 2
  self::addStoredRoutine($con, "someOtherFunc", "someOtherFunc.sql");
 
  // Set config completed flag for later
  $doneConfig = true;
}
 
/**
* Adds a stored procedure into the database
*
* @param string $con Connection to use
* @param string $routineName Routine name
* @param string $routineFile Filename inside our [plugin dir]/data/sql/
*/
protected static function addStoredRoutine($con, $routineName, $routineFile)
{
  $options = $con->getOptions();
  $dsn = $options["dsn"];
  preg_match("/dbname=([^;]+)[;]?/", $dsn, $matches);
  $dbname = $matches[1];
 
  $sql = "SELECT COUNT(*) AS c FROM information_schema.routines WHERE routine_name = '{$routineName}' AND routine_schema = '{$dbname}'";
  try
  {
    $dbh = $con->getDbh();
  }
  catch (Exception $e)
  {
    // No database available - can't carry on
    return;
  }
 
  $resultSet = $dbh->query($sql);
  $results = $resultSet->fetch();
  if ($results["c"] == 0)
  {
    // Create the function
    $sqlRoutine = file_get_contents(sfConfig::get("zz_myplugin_dir") . "/data/sql/" . $routineFile);
    $con->getDbh()->prepare($sqlRoutine)->execute();
  }
}

I’m not a massive fan of the way the database name has been retrieved but it works for now :-) I’m pretty sure there’s a neater way of accomplishing the above too, but this is working consistently and allows the stored functions to become part of a plugin, so that if the plugin is removed at a later date, then any subsequent database builds will not have unnecessary routines hanging around.

id: 1
name: Joe Bloggs
profile_picture: ffba81d3ac4ef1dcba70.txt

Great, if you’re not concerned about the appearance of the filename after uploading. If however, you want to preserve the filename, then you can create a generateFilename() method in your form object and Symfony will use that. But, what if you want to store more than the filename in your database, for example part of a path including multiple levels deep in directories? Or you want to do this each time, but don’t want to have to recreate/copy the same method multiple times? Or both?! Enter the use of sfValidatedFile.

In my experimentation, I wanted a custom file upload path, something like this:

• /home/rich/myproject/web/uploads/entries/4cad77fd2da01/1/filename.txt
• /home/rich/myproject/web/uploads/entries/4cad77fd2da01/2/otherfile.jpg

but I wanted to be able to store everything from 4cad… onwards as the filename, as everything up to ‘entries’ was a Symfony configuration value.

So, in your form configure() method, do something like the following:

// Generate your unique path here
$path = sfConfig::get("sf_my_upload_path") . DIRECTORY_SEPARATOR . uniqid();

$this->validatorSchema["myFileUpload"] = new sfValidatorFile(array("path" => $thisPath, "validated_file_class" => "myValidatedFile");


This tells Symfony to use a custom path to save, but also to use the myValidatedFile class to handle the processing of the uploaded file – move it to the correct place and so on. The myValidatedFile class is what we’re going to use to accomplish the above challenges.

Next, create the file lib/vendor/myValidatedFile.class.php:


class myValidatedFile extends sfValidatedFile
{
  public function generateFilename()
  {
    return $this->path . DIRECTORY_SEPARATOR . $this->getOriginalName();
  }

  public function save($file = null, $fileMode = 0666, $create = true, $dirMode = 0777)
  {
    $result = parent::save($file, $fileMode, $create, $dirMode);

    // Alter the savedName to reflect what we're after
    $this->savedName = str_replace(sfConfig::get("sf_my_upload_path") . DIRECTORY_SEPARATOR, "", $this->savedName);

    return null === $this->path ? $this->savedName : str_replace($this->path.DIRECTORY_SEPARATOR, '', $this->savedName);
  }
}


Simples. Let’s take each method in turn.

The first method simply returns the full path and original filename. Symfony will use the directory of this when deciding where to save the file, and it will use the leaf part to determine the file’s name. Part 1 accomplished!

The second method calls the original sfValidatedFile save() method. We’re not concerned with the mechanics of saving the file, so we’ll just do as normal (we’ve already set the path above). What we are concerned with is creating the correct value to be saved into the database or whatever you do with the field value eventually.

The key parts are the final 2 lines. Essentially here, we remove the part of the file’s path and filename that we have stored in the configuration value. This means that what is left will be (in the example above), our uniqid() value and the filename. This return value is what is saved/used elsewhere.

And then, your database entry will look like this:

id: 1
name: Joe Bloggs
profile_picture: 4cad77fd2da01/2/joes_picture.jpg


All you need to do now is set the “validated_file_class” option value on your file upload widget to myValidatedFile whenever you need this functionality, and job done!

I took a mini-break to New York last week, and had the chance to pop into the Apple store on 5th Avenue (I love how awesome that sounds). Obviously then I had to try out the iPad first-hand. It was clearly the most popular item in-store as the gorgeous MacBook Pros and large iMac demonstration areas were practically empty, with large crowds around the iPad areas.

It’s a nice device, if a little large for my liking, but very responsive, particularly with regards to the keyboard. Positioning the iPad horizontally on the provided slanted stand allowed for typing on the on-screen keyboard at the same speed as you would a normal computer keyboard, and was extremely nice. I was personally quite sceptical back in January when the product was announced, but I did find myself experiencing iPad-envy after playing with the device; travelling back to our hotel on the subway, I noticed about 4 people using them for various activities including reading books and browsing the web. Suffice to say I restrained myself and didn’t purchase one sadly!

It’ll be interesting to see the UK release; by then hopefully all the little niggles will have been ironed out and we’ll have a solid product. Watch this space :-)

Oh, and I also came across a stack of paper versions of The Onion hidden in a subway corridor. Very bizarre!

As part of one of our current Symfony projects in work, a colleague and I decided that rather than reinventing the wheel everytime we used a new admin-generated module in the application with regards to overriding partials and actions in pretty much the same way each time, it would be productive and easier in the long run to create an admin generator theme. The documentation on the Symfony site for this is sparse to say the least, and when I get some time, I’ll look at contributing back to the Symfony docs a condensed version of this blog post so that it’s a bit more straightforward for the next person. You have the ability to customise pretty much every part of the admin generator including batch actions, form actions, filters, table headers and so on, so it’s worth the work in the long run if you’re looking for quick admin-generated module deployments in line with your site’s template.

OK, so before I start, be warned this is nice and long (and hopefully explanatory)…! I’ll also say that clearing your cache upon making changes to partials is vital. I repeat, VITAL! :-)

Getting started

The first step I carried out, which is mentioned on the Symfony site, was to copy all the files from the default theme into your project. In our case, we’re using Doctrine as the ORM, so the files were found in lib/vendor/symfony/plugins/sfDoctrinePlugin/data/generator/sfDoctrineModule/admin, and they were copied to data/generator/sfDoctrineModule/ourTheme. This directory contains 3 subdirectories – we’ll need all 3. The names should be self-explanatory – “parts” consists of all the parts needed to dynamically generate the code, “skeleton” consists of skeleton files that form the basis of the final module (generator.yml etc), and “template” consists of the template files that use the files in “parts“. Simple!

That’s the basic step out of the way. If you want to check that everything is being correctly read here, simply alter an existing generator.yml file as follows:

theme: ourTheme

Clear your cache, and reload your page. You shouldn’t see any change – this is because the files are exactly the same as the default theme. Now comes the fun part!

For the following paths, I’ll assume the prefix of data/generator/sfDoctrineModule/ourTheme/ to avoid typing it out each time.

Adding configuration options

The first change I wanted to carry out was to add some extra configuration options via the generator.yml file. One major gripe I have with the Symfony admin generator is that once you’ve added a new object, or edited one, the default destination is the edit page, rather than the list page. Personally I see returning to the list as expected behaviour, and every project I use the admin generator in, I end up overriding the processForm() action and making this change. I wanted to be able to specify this destination via a configuration option on the ‘new’ and ‘edit’ contexts, with a ‘return_to’ option. Carrying out this change involves overriding some other classes in Symfony, most notably the sfModelGeneratorConfiguration class. I copied this class from the Symfony core to lib/generator/myModelGeneratorConfiguration.class.php.

Once you’ve done this, alter parts/configuration.php so that the generated class extends from myModelGeneratorConfiguration instead of sfModelGeneratorConfiguration.

Open up this class, and you’ll see a lot of abstract methods defined near the top. These methods are created during the generation process, and are located in the parts/fieldsConfiguration.php file. You’ll see that it consists of what looks like PHP within PHP – this is a common theme throughout the development of an admin generator theme, and can get a tad confusing if you forget what context you’re in when you’re writing code! You can use the example methods to build your own. In my case, I added the following:

public function getNewReturnTo()
{
return '<?php echo $this->escapeString(isset($this->config['new']['return_to']) ? $this->config['new']['return_to'] : 'edit') ?>';
<?php unset($this->config['new']['return_to']) ?>
}

public function getEditReturnTo()
{
return '<?php echo $this->escapeString(isset($this->config['edit']['return_to']) ? $this->config['edit']['return_to'] : 'edit') ?>';
<php unset($this-&gtconfig['edit']['return_to']) ?>
}

These methods both return a configuration value if it exists in the YAML file, or the defaults of ‘new’ and ‘edit’ respectively if no configuration value is specified.

In order to make this option available in the configuration, the myModelGeneratorConfiguration->compile() method should be altered to allow this option:

// ...
'new' => array(
'fields' => array(),
'title' => $this->getNewTitle(),
'actions' => $this->getNewActions() ? $this->getNewActions() : $this->getFormActions(),
'return_to' => $this->getNewReturnTo()
),
// …


and the same for the edit option. The final change is to actually action this change in the processForm() method, conveniently found in parts/processFormAction.php. This file (along with all the other *Action.php files) are included in the generated sfActions class specific to the module you’re creating. I removed the final redirect line which redirected back to the ‘edit’ URL, and replaced it with the following changes:

// ...
if ($form->isValid())
{
$thisContext = $form->getObject()->isNew() ? 'new' : 'edit';
$notice = $form->getObject()->isNew() ? 'The item was created successfully.' : 'The item was updated successfully.';

// ...
$this->redirect('@<?php echo $this->getUrlForAction('new') ?>');
}
else
{
$route = '<?php echo $this->getSingularName(); ?>';
$returnTo = strtolower($this->configuration->getValue($thisContext . '.return_to'));
if ($returnTo != 'list')
{
$route .= '_' . $returnTo;
}

$returnArray = array('sf_route' => $route);
if ($returnTo != 'list')
{
$returnArray['sf_subject'] = $<?php echo $this->getSingularName(); ?>;
}
$this->redirect($returnArray);


That’s it! Add the configuration option into your generator.yml as follows:
new:
return_to: list


Clear your cache, reload the page and test it out. After creating a new object, you should be returned to the list. This is restricted to eg my_model_ACTION-type routes, but you can easily enhance the above code to redirect to any other route, check for an “@” symbol if needs be and so on.

New “contexts”

I wanted to implement an Ajax edit form on a “view” page for my model, so that the user wasn’t taken away. The templates I was working from had this as a pop-over box, but instead of redirecting the user afterwards, I wanted the edit and update to be seamless. In terms of the form, the ajax form was similar (but slightly different) to the normal admin-generated edit form, so I wanted the ability to be able to configure the form separately from the edit page.

I experimented with configuring it as part of my ‘view’ context but ran into problems with nested array configurations, so I decided to abstract it out and create a new “context” with which I could configure items in the same way as a normal context (‘new’, ‘edit’, ‘list’ and so on). I termed this ‘ajaxedit’; NB I’m calling them ‘contexts’ here as I’m sure I read that’s what they were called somewhere! Correct me if not ;-)

The creation of this is very similar to the above changes for adding a configuration value. In myModelGeneratorConfiguration.class.php, add the following in to the configuration array in compile():

'ajaxedit' => array(
'title' => $this->getAjaxeditTitle(),
'actions' => $this->getAjaxeditActions() ? $this->getAjaxeditActions() : $this->getFormActions(),
'fields' => array()


I added this after the ‘edit’ one currently there. If you proceed down through this method, I also duplicated the ‘edit’ line in the first foreach() loop, where the fields are configured using sfModelGeneratorConfigurationField, and altered ‘edit’ to ‘ajaxedit’ where appropriate. Same in the ‘virtual fields’ configuration, the form actions, the field configuration (duplicated from ‘list field configuration’), and the $this->parseVariables() lines.

You should also add ‘ajaxedit’ to the credentials array near the end of the compile() method, so that you can control access to the popup/context with credentials, and again into the getConfig() method at the end of the class.

You then need to create the relevant abstract methods as before, so that the context can be correctly configured. Again, I used the ‘edit’ context as an example to base my changes and additions on.

The next step is to configure the templates and partials so that they recognise this. I used the default _form.php partial so that I could re-use code where possible, but added a ‘context’ variable to the partial when calling it, eg. in my view page:


[?php include_partial('<?php echo $this->getModuleName() ?>/form', array('<?php echo $this->getSingularName() ?>' => $<?php echo $this->getSingularName() ?>, 'form' => $form, 'configuration' => $configuration, 'helper' => $helper, 'context' => 'ajaxedit')) ?]


In my form partial, I then altered the getFormFields() call to the following:


[?php foreach ($configuration->getFormFields($form, $form->isNew() ? 'new' : 'edit') as $fieldset => $fields): ?]


so that I was using the correct set of fields for the form display. Don’t forget to clear your cache after making these changes.

You can handle the ajax post however you like; I used jQuery and a callback handler to adjust details and display new values etc where required. The handling of the form data was done in the same way as a normal Symfony action; details of this (and Ajax-specific stuff) are in the main Symfony documentation.

Here’s the result:

Conclusion

The one major thing to remember is that clearing your cache is VITAL to a speedy admin generator theme development. If you’re running in dev mode via your front controller, then Symfony will only regenerate partials that are directly related to your generator.yml on the fly. It won’t recreate action templates if there is no need to from generator.yml changes. This means that you will need to manually call “./symfony cc” from the command line whenever you make changes to partials. I lost count of how many times I ran that task during my theme development :-)

I’d imagine there are nicer ways to do some of the above – I’m not particularly a fan of the overriding of classes and having them separate from the theme, although I know that Symfony’s autoloading can be pretty clever sometimes, so it would probably find the classes with a bit of tweaking, if I put them as part of the theme. It’s also a bit tricky to get your head around the whole PHP-creating-PHP idea, and remembering whereabouts you are, and what variables you have available to you. I lost a good few hours trying to use variables that were in the top-level PHP, in my generated partials, and wondering why they weren’t working.

However, the end result is definitely worth it and saves a lot of time in the long run if you’re looking to re-use templates and code across a number of admin-generated modules!

The conference kicked off with the keynote speech delivered by Josh Holmes, on “The lost art of simplicity”.  This was a great keynote – some of the feedback I’ve read since the conference describes it as “nothing new”, or “knew this already” but regardless of whether you knew it already or not, it’s all too easy to forget about making things simple.  An example that stuck in my mind from the keynote was if a client asks for a report… and they’re delivered a report system.  Error, not what the client asked – they don’t care about a system to do it, they just want to know how many pencils they have in stock etc.  This seemed to ring true with a large majority in the room! It’s something that we at White October try and build into systems/sites we produce, so that the client has as much hands-on control as they need, without fancy technology or “cool ideas” getting in the way.

After the keynote, Ed and I split off to attend talks in our areas of interest.  My first stop was Stefan Koopmanschap‘s talk on documentation.  Another area which seems to get forgotten about! Stefan gave some great examples of how to get documentation done effectively, eg by getting your users to write the user manual (NOT the developers!).  In particular, he included “obvious” documentation methods which may be forgotten about, such as commit messages, DocBlock comments in code and unit/functional tests (we all do those, right?).  These all make for a much more maintainable system.  Great talk – extremely useful.

The lunchtime talk was given by Symfony legend Fabien Potencier, on new features in PHP 5.3 and how to solve real problems using new features.  The ubiquitous “lambda functions” came up, used as part of the example of the Symfony 2 Dependency Injection container.  Thoughts are divided in the WO camp on dependency injection, but I think once we shift to Symfony 2 and start using it in practice, it will make much more sense.  We also ambushed Fabien at lunch regarding the possibility of open-sourcing/releasing Sismo (the Symfony continuous integration server) but sadly it appears that’s not meant to be!

After lunch I attended the “hidden features” talk by Johannes Schluter which was interesting, if a little disjointed.  In particular, I discovered useful applications of streams eg for processing zip files, and  the PECL ‘inclued‘ package which shows you your require/include() calls. Handy stuff.  This talk was followed by “Regex-fu”, by Juliette Folmer – lots of sweet-throwing around the room for correct answers to questions; highly entertaining! I picked up a good few tips here, and enjoyed the explanations of how regular expressions work internally in terms of optimizing them etc.  I haven’t seen a talk with quite so many random characters being drawn on the presentation whiteboard before however…

The final talk we both attended was “Best practices for web service design” by Lorna Jane Mitchell.  This was an excellent talk, littered with real-world anecdotes from Lorna, and included again some “obvious” ideas/guidelines which I’d imagine we all forget to do from time to time! Things like stacking errors, presenting a consistent interface in terms of response codes etc, keeping as small an API as is needed for operation – all useful stuff, and a good way to end the talks of the day.

After the talks finished, Facebook kindly provided a large amount of free beverages, which we all were only to happy to partake in.  We met some interesting people during the next few hours, and tried our best to recruit budding junior developers… and failed.  Seems like everyone was recruiting at this year’s conference! We then headed back up to WO HQ in Oxford.

There were a reasonable number of stands, and varied ones at that, which was good.  The O’Reilly stand seemed to get a large amount of attention; probably due to the great discount available! It was also good to see the data.gov.uk stand in attendance, if a little sparse each time I glided past…

Anyway, a massive thanks to the PHP London guys, for putting on a great conference – definitely looking forward to next year’s!

So we decided it was time to perhaps investigate alternatives.  The obvious choice for Symfony would be to use Sismo, but sadly it hasn’t been released yet… Fabien, any clues as to a release date? ;-) It looks at the moment as if we’re stuck with a Java-based solution, so we decided to look at Hudson as an alternative.

Setting up Hudson was straightforward using Nicolas Perriault’s blog post on exactly what we were trying to achieve, linking in with our Subversion repository.  The configuration of the projects took a bit of faffing – for some reason, Hudson liked to check out the whole repository rather than just eg the trunk folder  as specified, so this screwed up some of the paths.  A bit of trial and error later and we were up and running with a build every 30mins if there are new commits, as this pretty graph shows… (using the jUnit XML output available in Symfony 1.4).  Total number of tests in blue, failures in red.

One thing to bear in mind if you’re getting test failures randomly, or perhaps always after a certain developer checks in (and it’s not a result of broken code) is to ensure that the permissions are correct for Symfony, as SVN likes to store permissions where it can.  The fix for this, apart from committing new permissions, is to add a build step in Hudson that carries out ./symfony project:permissions (Symfony 1.3+) shortly before the build.  This seems to solve it nicely, and explained the small red peaks in the graph above.

The problem I discovered was that obviously Doctrine didn’t know what the conditional relationships were, since it was dependant on the type column. I researched some different approaches:

1. Doctrine’s RawSql

   This approach worked for generating the correct SQL query without a problem, but the issue then arose of how Doctrine could hydrate the result set. Answer was – it couldn’t :-) Well, I lie – I could use Doctrine::HYDRATE_SCALAR but I had issues with other related models, where I was joining to a table twice but under different aliases, which failed, since the scalar hydration would override earlier joins on the same table.

2. Column aggregation

   Column aggregation looked great initially – I could have a “TypeAEntry” and a “TypeBEntry”, and Doctrine would have handled the typing of the Entry automatically. I couldn’t however see a way to query “in reverse” – get all Entries and their associated models.

3. Write a raw SQL query, and use Doctrine’s PDO instance to retrieve the data.

Option 3) was the one I settled for in the end. It meant of course that I had to specify column names, and perform all the joins myself, but it did mean I could specify precisely what data I wanted back. This came back in the form of a normal array.  I’d tried to stay away from this option intially, but after spending too long researching the other 2 and hitting brick walls, I bit the bullet and dropped down to the raw stuff.  And hey presto!

Note – this approach will only work if you want the data to be read-only. If you want to then use Doctrine’s model goodness, you’ll need to work out how to transform that *back* into a model.