I should preface this entry by saying that it could be a little dull if you’re not interested in radio research; that the views expressed here do not necessarily reflect my employer; and that I have no particular insider knowledge, beyond having used RAJAR for a long time.
RAJAR is the organisation that publishes radio listening figures in the UK. Figures are currently collected by carefully giving out over 130,000 diaries to people to keep for a single week over the course of a year. This is obviously a vast undertaking, but the UK has several hundred radio stations, the majority of which are local; so each locality has to have a representative sample of diaries if radio listening for those stations is to be accurately recorded.
The nature of this methodology means that listening figures are produced quarterly – and this Thursday at 7am, Q1 2008 figures will be published. Radio stations across the country are eagerly awaiting them!
Back in the mists of time – well 1998 – Kelvin MacKenzie, once editor of The Sun bought the station then known as Talk Radio. He rebranded it talkSPORT, but he had a problem. He felt that the RAJAR diary methodology discriminated against his station. In particular, he felt audiences were perhaps listening to his station but writing BBC Radio Five Live in their diaries. BBC stations on the whole have better awareness amongst the public, and that’s especially the case for national services compared to their commercial competitors who don’t have the marketing muscle of Auntie. On top of that, the diary methodology doesn’t work well for one-off events – by its very nature, it’s a 12 or 13 week average. So if you buy the rights to, say, a boxing match on a given Saturday night, you can’t tell if you achieved an increased audience as a result for that one-off event. Sports rights aren’t cheap, and aside from things like the Premier League, they don’t always run every week at the same time.
So he sought help with a research company called GfK who had a sister company that produced a special kind of watch. GfK took audio captures of all the services they were monitoring, while the watches also regularly captured the ambient sound of wherever the watch wearer was. By examining the uploaded audio the watch had captured alongside the time of capture, and comparing it to its big database of radio (and TV) audio, the idea was that the system could tell you what service, if any, was being listened to.
Kelvin’s company paid for the implementation of this technology, and he even took to publishing an alternative set of ratings. These showed far more people listening to his, as well as other services.
But in testing, the watch methodology didn’t pass muster. So while he’d undoubtedly put meters on the map, and stirred up the industry, which led to plenty of calls to adopt metering, the system he’d adopted simply wasn’t up to scratch. It also didn’t really help his cause that he showed that BBC2 was getting more viewers than ITV – unlikely given the popularity of such trifling shows as Coronation Street, X Factor/Idol or Emmerdale.
The other viable system in the marketplace was from Arbitron, a massive US research group who are responsible for American radio ratings. They had their people meter (PPM), and over the last few years it has been tested extensively in the UK.
Arbitron’s technology differed because it relied on a hidden signal embedded into stations’ output. This signal is inaudible to listeners, but the pager-style devices Arbitron manufactured were able to detect them, and thus measure what station a listener was listening to and when.
Since the start of 2007, a test has been conducted in London, initially in partnership with the TV ratings company BARB and RAJAR. UK TV ratings rely on boxes in around 5,100 panellists’ homes. The boxes also measure replayed material via video or PVR devices. But they do miss out on out-of-home viewing, e.g. watching the football in a pub. For some major games in large tournaments, that can be a substantial audience. Think of England featuring in the recent Rugby World Cup Final – a game played at 8pm in the evening.
But although theoretically with PPMs all seems fine, there are still issues that need to be addressed before metering can work, and RAJAR has now determined that they’re not easy to overcome in the short term, so they’re ending the project despite having spent £3.5m to date.
As RAJAR states, there are some key elements that PPMs seemed unable to cope with to a satisfactory extent. Foremost of these is the breakfast peak. Unlike television, which sees peak audiences in the evening, radio’s natural peak is at breakfast time. Most people live relatively busy lives and breakfast doesn’t afford much leisure time; they’re getting washed, dressed, eating breakfast, getting themselves or other family members off to work or school, and so on. So radio is the natural partner for keeping you entertained and informed. You can listen in the bedroom, the bathroom, your kitchen or living room. You may well then listen in the car on the commute to work, or on a portable radio or mobile phone on a train journey. So far, so obvious.
Yet PPMs seem to have under-reported this peak – to the point where this is no peak. In all likelihood that’s because people simply don’t take their pager devices into the bathroom with them, and then into the kitchen or living room. Thus listening at radio’s most critical time of the day is missed out by them.
On top of that there are other issues: the devices can’t cope with listening via earphones especially easily. And they’re simply not convenient to carry everywhere you go. Panellists are expected to take them everywhere, but would you carry one on an evening out, or on a shopping trip on a Saturday? If you forget your pager half-way to the station in the morning, would you return to the house to collect it, or leave it at home and not record a day’s listening? Short of implanting a chip in your head (not something the libertarian in me is desperately looking forward to), or perhaps using a mobile phone solution (since that’s a single device that you just might carry everywhere), it’s hard to see how meters will ever work in a fully satisfactory manner.
As a consequence, RAJAR has pulled out of this development for the time being, and is looking to other methodologies going forward. In the first instance, they’re examaning the use of online diary completion technologies.
Some people are going to be disappointed – not least Kelvin Mackenzie (no longer directly involved in radio) who has called it “an absolutely shocking decision.” He points to TV having used electronic boxes for years.
But Mackenzie is simply wrong, and his comparison with TV is specious. He knows that TV is viewed in a limited number of locations. And he well understands that the methodology that medium uses is also weak for smaller stations (the sample isn’t really big enough for full figures for every programme on every niche channel in a multi-channel world), and nor does it properly cope with out-of-home viewing, as mentioned above.
I have a feeling that Mackenzie’s won’t be the only dissenting voice we hear over the coming days, but until we have something that’s proven to work, the £600m commercial radio industry cannot rely on a device that doesn’t even recognise the most important listening time of the day. And it’s not just commercial radio that recognises this – Jenny Abramsky of the BBC is similarly supportive. It’s going to be very easy to give radio a kicking over this, but RAJAR is absolutely doing the right thing in launching a thorough industry wide review under Morag Blazey.
Interestingly, yesterday saw Media Guardian’s Radio Reborn conference (I didn’t go, but James’ notes seem very fine), which I believe included some Arbitron data to prove some points. Unfortunate timing for them really given yesterday was the day RAJAR made these announcements.
However it should be noted that at the same time, Canada is adopting PPMs to a greater extent.
But there are still ongoing issues in Arbitron’s home US territory, where some groups – particularly those catering to ethnic stations – are still facing difficulties with PPM technology.
There’s certainly still a watching brief over PPMs. But in the short term, they’re not an adequate replacement methodology in the UK.