code

X Mail Headers on identi.ca

A while ago I submitted a patch for statusnet, the code that powers identi.ca and other microblogging platforms. The patch added X headers to email notification messages, to make it easier to filter messages. The headers look something like this:

X-StatusNet-MessageType: subscribe
X-StatusNet-TargetUser: skwashd
X-StatusNet-SourceUser: username
X-StatusNet-Domain: identi.ca

The patch was included in the 0.9.1 release of statusnet, and is now running on identi.ca. I think this is the highest traffic site running any of my code. I am pretty excited about it.

This is one of the reasons why I love free software cloud services. Instead of just being a passive consumer of the service, you can actively contribute to its development.

You can run it too by downloading statusnet 0.9.1. Enjoy.

Now to setup some mail filters in Zimbra ...

Check Drupal Module Status Using Bash

When you run a lot of drupal sites it can be annoying to keep track of all of the modules contained in a platform and ensure all of them are up to date. One option is to setup a dummy site setup with all the modules installed and email notifications enabled, this is OK, but then you need to make sure you enable the additional modules every time you add something to your platform.

I wanted to be able to check the status of all of the modules in a given platform using the command line. I started scratching the itch by writing a simple shell script to use the drupal updates server to check for the status of all the modules. I kept on polishing it until I was happy with it, there are some bits of which are a little bit ugly, but that is mostly due to the limitations of bash. If I had to rewrite the it I would do it in PHP or some other language which understands arrays/lists and has http client and xml libraries.

The script supports excluding modules by using a extended grep regular expression pattern and nominating a major version of drupal. When there is a version mismatch it will be shown in bolded red, while modules where the versions match will be shown in green. The script filters out all dev and alpha releases, after all the script is designed for checking production sites. Adding support for per module update servers should be pretty easy to do, but I don't have modules to test this with.

To use the script, download it, save it somewhere handy, such as

~/bin/check-module-status.sh
, make it executable (run
chmod +x ~/bin/check-module-status.sh
). Now it is ready for you to run it -
~/bin/check-module-status.sh /path/to/drupal
and wait for the output.

Group Redent Plugin for Status.net / Identi.ca

For a bit over a year now I've been using the free software based microblogging service identi.ca. Unlike twitter, the identi.ca code base is released under the AGPLv3, through the status.net project. Anyone is free to setup their own instance of status.net and can even federate with other instances, using the Open Micro Blogging standard.

Another feature which identi.ca/status.net has over twitter is support for groups, so someone can send a message to a group of people, a bit like how a mailing list works. One down side of this feature is that some people redent (like retweeting, but on identi.ca) without removing the group "bang tag" (aka !). This leads to the same message being sent to the group over an over again which gets annoying. I left !linux and have considered leaving !ubuntu, because of how often this happens.

As status.net is freely available, I decided to write a plugin which handles this problem. If someone redents a message to a group without changing the group tag to a hash tag, the plugin does it for them. It is like the status.net twitter bridge when reposting dents to twitter. If someone really wants to redent something to a group, they simply use !! instead of ! and it will be sent to the group. This is useful, if for example, a dent from someone is relevant to a group, but wasn't posted there initially, it can be redented to the group

I disagree with Bela Hausmann (aka @and3k), I think it should be enabled by default on identi.ca. I hope others consdier it to be a useful feature and ask for it to be added to identi.ca.

If you run your own instance of status.net running, http://davehall.com.au/sites/davehall.com.au/files/GroupRedentCleanerPlugin.php.txt">download the plugin, drop it in

/path/to/statusnet/plugins/GroupRedentCleanerPlugin.php

and add the following to your config.php: addPlugin('GroupRedentCleaner', array()); Have a play!

Updating all of your Drupal Sites at Once - aka Lazy Person's Aegir

Aegir is an excellent way to manage multi site drupal instances, but sometimes it can be a bit too heavy. For example if you have a handful of sites, it can be overkill to deploy aegir. If there is an urgent security fix and you have a lot of sites (I am talking 100s if not 1000s) to patch, waiting for aegir to migrate and verify all of your sites can be a little too slow.

For these situations I have a little script which I use to do the heavy lifting. I keep in ~/bin/update-all-sites and it has a single purpose, to update all of my drupal instances with a single command. Just like aegir, my script leverages drush, but unlike aegir there is no parachute, so if something breaks during the upgrade you get to keep all of the pieces. If you use this script, I would recommend always backing up all of your databases first - just in case.

I keep my "platforms" in svn, so before running the script I run a svn switch or svn update depending on how major the update is. If you are using git or bzr, you would do something similar first. If you aren't using any form of version control - I feel sorry for your clients.

So here is the code, it should be pretty self explanatory - if not ask questions via the comments.

#!/bin/sh # Update all drupal sites at once using drush - aka lazy person's aegir # # Written by Dave Hall # Copyright (c) 2009 Dave Hall Consulting http://davehall.com.au # # This program is free software; you can redistribute it and/or # modify it under the terms of the GNU General Public License # as published by the Free Software Foundation; either version 2 # of the License, or (at your option) any later version. # Alternatively you may use and/or distribute it under the terms # of the CC-BY-SA license http://creativecommons.org/licenses/by-sa/3.0/ # Change this to point to your instance of drush isn't in your path DRUSH_CMD="drush" if [ $# != 1 ]; then     SCRIPT="`basename $0`"     echo "Usage: $SCRIPT path-to-drupal-install"     exit 1; fi SITES_PATH="$1" PWD=$(pwd) cd "$SITES_PATH/sites"; for site in `find ./ -maxdepth 1 -type d | cut -d/ -f2 | egrep -v '(.git|.bzr|.svn|all|^$)'`; do     if [ -f "${site}/settings.php" ]; then         echo updating $site         $DRUSH_CMD updatedb -y -l $site     fi done # Lets go back to where we started cd "$PWD"

OK, so my script isn't any where as awesome as aegir, but if you are lazy (or in a hurry) it can come in handy. Most of the time you will probably still want to use aegir.

Notes:

Make sure you make the script executable (hint run chmod +x /path/to/update-all-sites)

If you don't have drush in your path, I would recommend you add it, but if you can't then change DRUSH_CMD="drush" to point to your instance of drush - such as DRUSH_CMD="/opt/drush/drush".

Thanks to Peter Lieverdink (aka cafuego) for suggesting the improved regex.

PHP on crack

I had a chuckle while reading Ed Finkler's PHP6 wish list. After reading the comment by Damien Seguy I almost fell off my chair.

Try running the following code under PHP

<?php ${'[email protected]#$%^&*()[]:;"<>,./?'} = "i bet this won't work!<br>\n"; ${"omg???!!! wtf???!!! :D"} = "omg it does<br>\n"; echo ${'[email protected]#$%^&*()[]:;"<>,./?'} . ${"omg???!!! wtf???!!! :D"};

It worked here on PHP 5.2.3 with ubuntu security fixes. I got

i bet this won't work!<br>
omg it does<br>

This contradicts the PHP manual, but hey it works.

hello_world.c

$ cat hello_world.c #include <stdio.h> main() { printf ("Hello World!\n"); }
$ gcc hello_world.c -o hello_world
$ chmod +x hello_world
$ ./ hello_world
Hello World!
$

I have started playing with C recently. I am still finding it annoying, yet enjoyable at the same time. The PHP developers have spoilt me (and other PHP coders). PHP gives you most of the power of C, without having to deal with annoying string handling, easy arrays, memory management and having to build to test. I am sure there is more.

Unlike OS X, I think I will persist with C.

Other than reminding myself of how to do hello world in C, why did I post this? I thought you would never ask, it is because my blog is now syndicated on Planet Linux Australia. I am pretty honoured by being added. Hopefully there are others who appreciate my rants :)

The Summer of Code Roller Coaster

I am awaiting final confirmation from google and the GNU project, but I am 99% sure now that phpGroupWare will be getting a Summer of Code (SoC) slot, and I will be mentoring a student to implement sync.

What a process it has been, and I am yet to start mentoring. I thought others might be interested in the the ups and downs involved in getting phpGroupWare a SoC place.

At the start of March, Google opened applications mentoring organisations. I made an application on behalf of phpGroupWare. All up 141 organisations were accepted, ranging from small obscure projects through to some of the stars of the FOSS community, with a mix of grass roots and commercial projects. Unfortunately phpGroupWare ended up at 142nd or lower in the rankings.

I am sure google was inundated with applications, but it would have been nice to at least receive a "thanks but no thanks email" from them. I found out by checking the listings when they were announced. It kind of reminded me of first year uni when they would post provisional marks for the semester a couple of weeks before mailing out academic transcripts. I thought our Summer of Code was over before it had even started.

Then I noticed that the GNU Project had been accepted as a mentoring organisation. As phpGroupWare is a GNU Package it was eligible for SoC slots, under the GNU banner. The GNU project received 65 valid applications, with a further 12 being deemed to be invalid. I thought phpGW had a pretty good chance of getting one of the slots.

phpGroupWare received 2 applications for worth while improvements, those being sync and redoing the installer/setup code. It was really hard choosing between the 2 applications. The rationale went something like this. Setup is the first thing a new phpgw admin will see, our current setup app has had some eye candy added by it is far from polished and needs some serious attention, yet it is not something that someone is likely to invest money in, but it is important. Sync on the other hand allows us to support mobile devices, desktop apps (including dare I say it MS Outlook) and to tick another business functionality box, yet our previous attempts at sync have failed, usually due to technical reasons. In the end a tough choice had to be made (sorry again jarg), and we went with sync as it was going to bring the greatest benefit to our current and potential users.

The GNU SoC admins asked all the mentors to rate the applications. Our top choice (sync) was rated up.

While I was away for Easter (with limited dialup speed internet) it was announced that the GNU project had been allocated 8 spots. That seemed like a reasonable number off slots given that some GNU packages, such as GNOME had been given many more in their own right. The problem was that the 8 places allocated to the GNU project had to be shared between 11 packages. This meant that some people were going to miss out. I still thought that there was good chance of phpGW getting a slot.

On Monday when I got home (with a good connection) the final 8 was proposed, to my disappointment phpGroupWare wasn't on the list, we had been ranked 10th. This ride was still far from over. A later message suggested that there had been a rethink and 2 packages had been dropped and the last 2 slots were to shared between phpGroupWare and 2 other projects. Back to a two-thirds chance. I fired off an email explaining why I thought we should get one of remaining slots.

The next morning I woke to the bad news, we had lucked out again. Another application had been found as the diamond amongst the coal and elevated to 8th spot. phpGW was now sitting in 10th spot and out of contention, or so I thought at the time.

By this stage I was proof reading a draft blog post on the whole SoC of process. I am now glad that I didn't publish it, not that I was likely to, it was more a venting. I did email one of the admins privately asking for more information on why we had been rejected.

On Wednesday morning I was checking my mail and found a new GNU SoC final list (rev 3, I counted). One student had been allocated to 2 projects and had decided to work on the other project, this freed up 1 slot, which meant phpGroupWare moved to 9th, still 1 short. The news got better, google had allocated the GNU project an additional place. We made it, finally!

I am sure Google didn't intend it to be such a tough process for mentors, but when there are so many worthwhile FOSS projects, some many enthusiastic and competent students and a limited budget it does make it difficult for Google to give everyone a go.

It has been a long and stressful process to get phpGroupWare a SoC slot this year. I hope that the stressful part of the process is now over and that Johan Gunnarsson will turn out a functional syncML interface for phpGroupWare.

If it works out well this year, it might be worth all the effort to ride the roller coaster again in 2008, as long as Google is willing to put up the cash.

Update: I have checked and phpGroupWare is in Google's accepted list.