Back in May, as I walked down Broadway in New York on my way to lunch with a contact at Nielsen Media Research, there was more noise than usual as I approached my destination.
When I turned the corner toward Nielsen’s office, I saw a group of placard-wielding demonstrators marching in a tight circle outside the door.
On closer examination, there were about a dozen of them, each sporting slightly different racial credentials. There were people of African, Hispanic, Asian and Indian descent, each carrying a banner demanding that Nielsen not “count us out”. A Fox Television camera captured the scene, while a beaming Reverend Al Sharpton, a leading local politician, spoke to journalists.
Welcome to the US version of the Barb Technical Advisory Group.
Perhaps we should rewind a few years to understand how things got to this point.
America has been consumed with the story of Nielsen’s battle to convert the 10 largest cities from an old-style diary measurement system to one based on the People Meter – an electronic system that measures the tuning status of the TV set (on, off, time), what channel is being tuned and who is watching.
It may seem strange in this day and age that $25bn of local television market spending is still reliant on a methodology developed in 1930 to measure radio listening. But it is.
The national market, where broadcasters such as NBC and CBS and several hundred cable networks compete for viewers and advertisers, switched to ANDREW GREEN.
If you thought the Rajar diary debate was hot, it’s worth casting an eye across the Atlantic, where placard-wielding protesters – supported by New York senator Hillary Clinton – claim a new system of compiling ratings amounts to an attack on minorities. Billetts’ Andrew Green explains How Mr Sulu got entangled in the battle of the People Meters People Meters in 1987. Like all change, it wasn’t made without a struggle, but it is now the accepted “currency” of media buying and selling in the national television arena.
Meanwhile, in 60 or so of the largest local markets (out of 210) a different method is used.
Nielsen equips panels of people with set meters to measure household tuning throughout the year. But the data produced by this system is not generally used by buyers or sellers as their currency of transaction.
Instead, 300 to 500 people in every market are asked to complete seven-day diaries of their television viewing. A new sample is asked to do the same for a second week and so on until four weeks have passed.
This occurs four times a year in periods known as the “sweeps”.
Because the timing of the sweeps is well known, television stations in local markets put on their best shows to maximise audiences. The data is used to plan for several months ahead – until the next sweep occurs.
Respondents are expected to remember everything they watch and to record it carefully in the diary, both for themselves and for other household members.
In practice, they are likely to fill in their diaries like they do their tax returns – just before they have to. This means it will probably have been several days after viewing and people are likely to remember only regularly watched programmes and those shown in primetime, penalising small stations and non-primetime programming.
Filling in a diary was probably not difficult when there were three or four channels to choose from, one set to watch and the whole family in the room at the same time. Today, the average household has 2.5 sets and can receive 141 channels.
Because viewers cannot be expected to indicate exactly which minutes they watch, advertisers have to make do with quarter-hour average audiences.
And since the diaries are filled in for only a week, statistical techniques have to be used to estimate how viewing might accumulate over longer periods such as a month or a quarter, over which campaigns are typically planned.
There are also a host of problems with viewing from the diary not matching viewing from the set meter, not picking up small programmes, and so on.
In short, diaries are an absurd way of measuring and reporting TV viewing in today’s environment and sweeps are plain stupid.
So it was a relief to media buyers – or should have been – when Nielsen announced a few years ago that it was to introduce more modern measurement techniques to local TV, starting with the 10 largest markets.
These represent about a third of the population and almost half of all local television ad spend.
Boston was first off the mark in May 2001. The People Meter system was carefully piloted alongside the diary for a year and officially launched in April 2002.
As every research company knows, when a new methodology is introduced there will be winners and losers.
People Meters – wherever they have replaced diaries – have tended to show lower audiences for the largest stations and higher audiences for smaller stations and programmes outside peak viewing hours. This results from the tendency mentioned earlier for people to recall more regular and significant viewing sessions and to forget shorter, more incidental viewing.
But to those on the losing end, this must be a serious flaw.
There must be something wrong with the sample composition. Or the response rate. Or the way Nielsen processes the data. More pilot work needs to be done – preferably in private so as not to distort the market and “confuse” buyers, and for a very long time to make sure things are properly ironed out.
But things will never be ironed out. The audience share of the largest networks had always been exaggerated because the diaries inflated it.
Late night audiences were always bigger – people just didn’t remember watching by the time they filled in their diary.
Somehow things settled down and people got used to the fact that Boston had a different currency of measurement from other markets.
Late in 2003 there was a pilot run of the People Meter in New York, followed a few months later by Los Angeles, then Chicago. The original plan was that New York would launch in April 2004, although it actually started in June. Los Angeles began in July and Chicago in August. There have been massive sample increases in these cities as well, with 800 households metered in New York compared with 540 under the diary system.
But what a fuss the doubters have made this time. The launch process has been a case study in effective public relations by one significantly disadvantaged network, a political platform for various Democratic hopefuls from Hillary Clinton downward and, in some ways, quite funny.
In Boston, opponents highlighted issues of sample quality without generating much interest outside a small group of research specialists. It didn’t work. Fox Television decided to play the race card in New York, where its viewing loss was substantial.
The People Meter sample in New York, it argued, “undercounted” people of colour.
As a result, the networks would make fewer and fewer programmes that appealed to minorities. It was, in effect, racial discrimination.
A coalition of interested parties was cobbled together under the banner of the Don’t Count Us Out Coalition. With the endorsement of Senator Clinton and the Revd Sharpton, as well as the financial support of Rupert Murdoch’s Fox Television, the movement had no trouble attracting a sympathetic press.
Because the issues are quite difficult for non-experts to grasp (or, indeed, to take any interest in) they have mostly been ignored. The Don’t Count Us Out website declares: “AfricanAmerican and Latino viewers using People Meter technology may be undercounted by as much as 25%.”
It goes on to refer to “substantial evidence” proving that Nielsen undercounts minority viewers and quotes a comparison from February 2004 between programme ratings measured by the two systems in New York. In this, the programmes ranking high with African-American viewers apparently suffered greater declines than those geared primarily to white audiences.
There is plenty more “evidence” cited, although nothing that would support the 25% claim.
A system audit carried out by the Media Ratings Committee found no such evidence, although it did uncover a small issue with Asian-American sample representation, which has been quickly corrected.
Nielsen explained clearly in a report of its own that overall viewing levels for all racial groups were unchanged and that they had in fact been watching other networks and channels.
But the damage was done.
Nielsen even went to court in Los Angeles, where a Hispanic network accused it of “engaging in unfair, unlawful and deceptive business practices by launching its Local People Meter service in Los Angeles utilising flawed sampling and weighting methodologies”. The judge disagreed and allowed Nielsen to go ahead with the launch.
Meanwhile, Nielsen has put together its own committee, known as the Task Force on TV Measurement, which will, in the autumn, pronounce on how well the various minorities are represented.
George Takei (aka Star Trek’s Lt Sulu) is one of the committee members representing AsianAmericans, alongside the president of the National PuertoRican Coalition, the executive director of the National Association for the Advancement of Colored People and the publisher of the Amsterdam News.
Nielsen is forging ahead with its launch programme, hoping the fuss will die down.
It shows how difficult it is to do anything new in the US market. It is a pity that People Meters themselves will probably be defunct a few years from now.
Because then Nielsen will have to start all over again .
Andrew Green is global research director at Billetts Connections