PPVote in action
Building Perfect Council Websites ’10, the fifth such annual event and the UK’s largest, brought together more than 350 participants, including local authority webmasters and web teams from all parts of the UK.
Private sector partners; health service, government and other public sector web teams; communications specialists; and others with an interest in the development of efficient, citizen-centred public services online were also in attendance.
This high profile event combines a headline conference with a unique interactive format of round-table discussion groups and pre-arranged one-to-one meetings for structured networking. The event focuses on both current best practice and predicts future trends to keep practitioners ahead of the game, from open data to mobile services.
This year I ran a “live” website review along the lines of the ‘Better connected’ survey Socitm Insight undertakes each year to assess the state of development of local authority websites and their provision of customer-oriented information and services. The survey project team, of which I’ve been a member since 2004, goes to extraordinary lengths to conduct a qualitative review of each website during the annual survey but there’s inevitably an element of subjectivity in the review – we are human after all.
Its not uncommon after the results of the survey are published each March for councils to come forward and tell us that information that we’ve indicated in the survey as ‘not found’ is actually on the website. But our response to that is simple. The information sought may well be on the given website, but our reviewer could not find it. The reviewer is not saying ‘it isn’t there’ but rather ‘I couldn’t find it’.
Our reviewers have a collective wealth of experience in local government and website management. We know how local government works, including some of the issues relating to silo working and often a lack of customer focus, and we make allowances along the way. Our reviewers try very hard to find answers to set questions and we really don’t like scoring a question ‘No’ if at all possible!
Thus when we do say ‘No’ to a question, it usually means that we’ve tried pretty hard to find an answer. The average citizen probably wouldn’t be that flexible, patient. If the Socitm reviewer can’t find a piece of information, then what chance does an ordinary citizen, unfamiliar with the ways of working in local government?
Earlier this year I thought of a novel way to help council web teams understand better how our annual website reviews are conducted. It occurred to me that, at the Perfect Council Website event in July, a member of the team could perform a live website review in front of an audience and explain as he/she progressed what they were examining, what they found and how the final assessment of the site would look if it was a genuine review.
I put my idea to the event organisers and was delighted to get a green light. I piloted the concept at the Scottish Webteams Forum in Edinburgh in May, and it was a great success – very well received, with lots of great feedback after the event.
I have to say I was a little disappointed to have less than 30 people attend my workshop this Wednesday, but as it turned out I need not have been concerned. I’d like to think that I engaged well with the ‘select’ group that were there, and that they went away at the end of hour-long session with a better understanding of what happens during our real reviews, but also a bit of an insight into how others may see their own website! If that WAS the case, then I’m a very happy bunny.
I invited a member of the group of 30 to be my guinea pig, and was delighted to have a guy called Adrian from a private sector consultancy firm step forward. He’d been working with a London borough council to improve their website, but for my purposes he was technically a genuine citizen rather than a council employee. Almost the perfect scenario!
I set Adrian to work to navigate to the homepage of the website for West Somerset Council (www.westsomersetonline.gov.uk). And then I explained to him (and the group) that he would be examining this website by posing a number of questions to test just one subject area to see how easy it would be for a citizen to obtain information or a service, and covered off a few housekeeping notes for the group about how the review would be conducted.
- I would pose a question
- Adrian would try and find the answer to the question and his use of the web browser was displayed on a large projection screen for all to see clearly.
- The group would decide whether the question was satisfactorily answered or not.
Thanks to the involvement of my new best mate Jeff from a super company called PPVote (which I thoroughly recommend to all, even if I’m not on commission!), and through the wonders of technology, Adrian’s review was going to be interactive so the group could participate rather than experience an online equivalent of death by PowerPoint.
Adrian would navigate the site as best he could, then when he’d answered the question to the best of his ability we would switch to a slide that posed the question again and gave the audience options for answering it (eg 1 for Yes, 2 for No). They were to press the appropriate number on a keypad they had been provided with on entering the room (you can watch a demo here).
I explained that when they pressed their selection nothing would happen until everyone had voted, when a score would be displayed on the screen in percentage terms. The highest percentage would be the ‘answer’ to the question and, if we were doing a Better Connected survey for real then that would have been the answer we’d record in that website’s review record. Then we’d proceed to the next question and repeat the process.
I said that what I should like them to get out of the session was that they’d leave with a greater understanding both of how our website reviews are conducted but also that they’d be scoring a genuine council website and would comprehend how we might think our website has everything in place but sometimes we may be too close to it to realise that, actually, its not as good as we might believe and that the Socitm reviewer really has tried hard to find the answers to his questions!
Housekeeping over, I posed the first question. The real McCoy survey has around 120 questions and the review can easily take about two hours, of total concentration, to complete. So obviously I had to cut it down to something manageable and relatively straightforward to examine, demonstrate and open for discussion with the group. I had elected to test for food business registration, a statutory obligation upon anyone starting up a new food business (eg a kebab takeaway), and applicable to all levels of local government except English shire counties (eg Hampshire County Council).
“Does the Business landing page signpost you to information about food premises registration? Yes or No.”
Adrian found his way to the Business landing page and struggled to find anything on that subject. The display was switched and the group cast their votes. 100% for No.
“Does the Licensing landing page signpost you to information about food premises registration? Yes or No.”
My guinea pig struggled this time, first he had to find the Licensing landing page and wasn’t having much success via navigation or search, so he tried the A-Z entry for L (Licensing) to reach the right page. But again there was nothing relating to food there. And again, the group cast 100% of votes for No.
“Does the A-Z contain an entry for register food business or similar? Yes or No.”
There was nothing for ‘register food business’ but ‘food business registration’ was found, and subsequently received a 100% vote for Yes.
At this point I detected a bit of a vibe that 100% answers throughout wasn’t exactly setting the room alight. I suspected a few thought the session was a bit of waste of time. So I reassured them that it would become more interesting, just bear with the process.
“Using the website’s own search, can I find out how to register my new food business? Yes or No.”
The website came up trumps again, the very first result was what was required. We didn’t bother voting this time, just went straight on to the next question!
“Does the website inform you that you are required by law to register your food selling business? Yes or No.”
Adrian clicked on the first search result and landed on the right page. He scanned down what is quite a lengthy, rather verbose page but couldn’t find any reference to a legal requirement to register a food business. Lots of useful information but nothing about the law. Well, that is until someone in the group called out that there had been a reference in the first paragraph. Adrian scrolled back up and we agreed that, yes, the site did inform us as the question asked. Over to the interactive scoring, this time the rating was interesting: 96% said Yes, 4% No. So we explore the result a little and it was concluded that the information was easily missed, indeed Adrian had missed it altogether and only found it when it was pointed out to him. Would a real life citizen have seen the statement?
“Am I able to register my food business online? Yes or No.”
Adrian played a stonker here. He scrolled straight down to a mid-paragraph link (appearing below the fold) that led you to an online registration form care of the UK Welcomes gateway on Business Link. Well done! Back to the group for the next vote, and unsurprisingly it was 100% for Yes.
I pointed out to the group that at the top right of this page about food business registration, there was a prominent link to a PDF registration form that could be downloaded, completed and submitted to the council for the statutory registration.
“If I can/have to download a form to register, is a signature required? Yes or No.”
This question caused a few eyebrows to be raised in query. This is investigating a level of detail that we don’t normally pursue in our reviews, but this was an important question to ask in light of the recently effected EU Services Directive. The group answered the question 100% as Yes.
The requirement for a signature on this form is not a statutory requirement and, technically, could be viewed as an unnecessary block to smooth conduct of trade in the EU. A council cannot refuse a food premises registration, it must accept it when made, and a signature is not required on the form. Further, under the Directive, the registration should be able to be completed online electronically. However, because this form required a signature, it would have to be printed out and posted to the council by snail mail. All of which would take time, completely unnecessary time.
I posed the next question of Adrian and the wider group:
“Which registration channel did you notice first? Was it downloading a form or clicking a link to the online form?”
96% noticed the PDF first.
As intimated above the online service is supposed to be available for this registration. Not only that but the online service will be more convenient for both the business owner and the council to undertake. Faster and cheaper to administer too. But the PDF was the channel that the website is giving more prominence to.
The online form link is buried in paragraph text, whereas the PDF gets a ‘panel’ all to itself on the right.
Definitely a missed opportunity, but perhaps indicative of a wider service issue. Perhaps this council is failing to maximise the potential for its customers to self-serve online, to use its e-services and thereby help the council cut its costs? If take up of these services is currently low, it may well be because they are not being adequately promoted on the website through prominent signposting as in this instance, and one can’t help wondering whether individual service delivery areas are promoting the e-service as they should.
“Are contact details provided so that I can get in touch if I wish to discuss my registration? Yes or No.”
Adrian really struggled with this one. He came to the conclusion that a reference in the middle of all the text on the page to ‘contact the Environmental Health team if required’ certainly did not amount to contact information, and the wider group decided that the cover-all ‘Contact us’ link at the top of the page did not count as contact information in the context of this question. And I agreed. The vote took place and 96% voted for No.
“Can I find out what a local resident would need to do if they wanted to register a complaint about a local takeaway business? Yes or No.”
This question was intended to test one aspect of the EU Services Directive whereby the service description is expected to include details of how to seek redress in the event of a problem, whether consumer or business. Adrian read through the text of the registration page but it was inconclusive. So he tried navigating around the Food Safety section of the website, eventually locating a page titled Food Poisoning that explained what someone would need to report such an eventuality.
The question was now put to the group, is this a Yes or a No? 100% voted No and during the subsequent discussion various views were expressed about the food poisoning page including the fact that you might want to make a complaint about somewhere even if you haven’t had food poisoning, yet this website did not tell you how to do that.
That completed the questions about specific aspects of the site content. Now the group was asked to rate a couple of aspects about the scenario just examined.
“How easy do you think it was for the reviewer to find answers to the questions posed?”
The group had a choice of four different answers for this question. They could only vote once. The choice was:
- Not found
- Very good
The highest vote was for Satisfactory with Poor second. Some discussion took place about why people felt it was satisfactory, including considering the information provided, taken in the round (considering the access via the A-Z and search, etc). But the next question proved interesting…
“Rate how well this website enabled the customer to complete the task.”
The options for answering were the same as the previous question. The vote this time was reversed. The highest vote now was Poor with Satisfactory second. If it had been a real ‘Better connected’ review, that assessment would have resulted in a points score for the whole section of 1 (Poor) – the other scores could have been 2 for Satisfactory, or 3 for Very Good.
Discussion followed about the drivers for those who had voted Satisfactory before to have now switched to Poor. Several theories were put forward. A fair amount of discussion had followed the first question and so people had had an opportunity to reflect on the overall score and adjust it down based on the evidence seen and the subsequent conversation. Another suggested that the wording of the actual question was a factor, highlighting the importance of getting the survey questions right in the first place.
The final question didn’t involve a technical wizardry at all. At the end of each section of the annual review survey there’s an opportunity for the reviewer to comment upon what he’s found during the exercise. This might be about highlighting an excellent example of the genre that other councils might wish to view, or perhaps pointing out where something didn’t work, or something else. So the final question posed of the group was hopefully going to be interesting as I wanted to stimulate further discussion about what had been found and what might be changed.
“Do you have any comments to make about what you have found?”
Adrian was given the first go here, as he had been the one using the mouse and answering the questions each time. He started off by saying that he had found the website rather verbose (pages needed to get to the point faster), but thought the search was good the way it highlighted search criteria entered. Overall he found the experience rather difficult, he said. Others joined in saying that they too thought the page about food premises registration had too much text on it, but other aspects of the site looked really good. It was fresh, contemporary in design and felt well managed but that all that’s to no avail if the service delivery aspect of the site doesn’t cut the mustard (ie prevent the customer from completing his transaction online without directly contacting the council in some way.
A couple of questions were asked about the EU Services Directive and aspects of its implementation by local authorities, focusing specifically on the area of food business registration since that was the topic of the moment. I was able to show the group the website of South Somerset Council which is, unsurprisingly, adjacent to West Somerset. South Somerset have implemented the directive well and the required information for food business registration is comprehensive and straightforward to find and use. It wasn’t perfect, but it was better than satisfactory.
In terms of look and feel, all were agreed that the West Somerset site, with a fresher, more engaging design, was much better than that of South Somerset, but that unfortunately a great design doesn’t answer questions. In short, South Somerset’s website appeared to be rather more focused on providing better information to its customers than being aesthetically pleasing.
As the session drew to a close, I summed up the experience that I’d just taken them through and said that I hoped that they now had a better understanding of what a website review involves. The general consensus appeared to be very favourable, which I was delighted (and not a little relieved) to hear!
Later that day, I discovered a number of tweets from @Speedball74, @eGovtBulletin, @Stephen_Cross and @GossInteractive which suggest that the session was indeed well received, and a delight to read at that! I shall look forward to receiving the official feedback from Headstar in due course!
In summary, West Somerset Council have covered the basics of food premises registration well, but there are a few issues which would benefit from being attended to. Every public sector web manager’s mantra should be ‘The customer is king’ or, more simply, just ‘Think customer’. We need to encourage content providers to write with the customer very much in mind, and make it is a straightforward as possible to complete a given task.
A few tweaks to the West Somerset Council website would make such a huge difference to its overall usability, but on the whole its a website that I really like and is a credit to the individual who manages it on a day to day basis.
And finally, a huge and very public ‘thank you’ to my up-for-it guinea pig on the day, Adrian. I’m afraid I don’t know his last name or company, but he was the hero of the hour!