Tell Them What You Think

TTWYT’s development blog and related musings

The recent redesign of the DoH website is, as has been noted by others, a great improvement. I was happy to discover that they do now have RSS feeds for consultations, and, more impressively, for consultation responses. This is something people have wanted for years, so it’s great to see that it has finally happened.

Unfortunately, they don’t work. The current live consultations feed, erroneously, contains no items.

I shall let them know…

The site will now display all the Ofsted consultations it can find. I hope this means that, along with the DCSF, the site will now find all government consultations relating to education.

That said, a little bird told me that schools regularly receive consultation documents, often directed at specific heads of department, with requests that they participate. I don’t know the details yet. If they are “private” consultations then I suppose that’s all well and good, but one does wonder if they might just be public ones that no one knows about!

Hopefully I’ll have some more information about that soon — watch this space!

One of the things I hope that TTWYT might accomplish is to stimulate more “blogospheric” debate when consultations are released. It would be marvellous if the publication of an important consultation led to lots of excellent blog posts followed by the submission of lots of independent* responses!

To that end, each consultation’s details page now displays a list of pages which have linked to the consultation, as reported by Technorati.
* I say independent because I am reliably informed that when a response to a consultation is submitted in the names of multiple people, it is common practice to count it as a response from one individual, rather than several. Argh!

Just did a quick spot on You & Yours about the lack of central consultation publishing, the difficulty this creates in finding interesting consultations, and the consequent narrowing of public debate that occurs.

Unfortunately, they asked me not to mention the site’s URL on the air. Oh well!

Gavin, who is responsible for this tasty little number which generates an RSS feed from the Department of Health’s consultation site, is redirecting his feed to TTWYT. His site predates mine by some years and was inspired by a post at Doctors.net bemoaning the lack of RSS feeds on the DoH website. Having been at it for a while, and having only intended for it to be a stopgap measure, he’s decided to pass the torch.

Kudos to Gavin — his site is certainly the earliest one I’ve seen that produces RSS for consultations.

A few months ago, I responded to a couple of government consultations and, in the process, discovered there was no way to search all live consultations, or to be alerted when a new one was published.

This struck me as more than a little mad: consultations are one of the government’s main methods for engaging in substantive public debate. I am not aware of any other process by which Joe Public can express their views on a particular topic, in detail, and can be reasonably certain that they will be read, and justifiably hopeful that they will be taken into account. That said, the entire process is worse than useless if no one finds out about them in time to contribute.

I thought that there must be a solution to this problem. My first port of call was a quick survey of government websites to determine which ones provided their data as RSS feeds. To my utter astonishment, only the Welsh Assembly Government did. With this came the realisation that the problem was a bigger one than a mere aggregator could solve (call me naive, but I had hope!), so I set about writing a screen-scraper for the DCMS as an experiment. I subsequently discovered that I had unwittingly picked one of the worst departments to scrape: their code is particularly unstructured. After implementing scrapers for ten or so departments, I had a look at the horrendous spaggetified ascii pudding that my code had become, and started again.

Over a couple of weeks, my little collection of scrapers grew until I had twenty or so departments. At that point, the data was only available as RSS feeds, and I had no plans to do anything more with it. After showing it to a few people, however, its potential became clear, so I started on a proper website to do all the things that people would want. In particular, Tom Steinberg’s reaction was effusive, and his enthusiasm infectious. I was also much encouraged by the numerous splendid people I met at the inaugural UKGovWeb Barcamp, who all said very useful things. Kudos in particular to MySociety for offering their hosting services.

In the weeks following Barcamp, I continued to beaver away at the scrapers and the site, until it went live on 11th Febuary, 2008. I have been very pleasantly surprised, to say the least, by the level of interest TellThemWhatYouThink has received. I have already had a few meetings with civil servants to discuss changes to their sites which would improve the quality of the data TTWYT’s scrapers can gather, and several more are planned. Lots of people have emailed me great feedback on improving the site, and I and others have lots of ideas about where it could go in the future. To all involved: many thanks.

Watch this space!