Defining an elusive population

What do you do when you need to define a population but the usual administrative boundaries don’t apply, and the client’s budget is limited?

What began as a conversation in a Midsomer Norton restaurant, led Cognisant Research and local Community Radio Station, Somer Valley FM, to develop a method of population analysis that could apply to any research problem where the population is difficult to define.

At its conclusion, the Cognisant study found that Somer Valley FM reached a weekly audience of around nine and a half thousand adults and was the 4th most listened to radio station in its target broadcast area, following only BBC Radio 2, Heart FM and BBC Radio 4. So how did they do it?

As with any research exercise, defining the population of interest is critical. In theory, this should be a straightforward exercise, as community radio stations typically cover a radius of up to 5km. However, the rolling countryside of the Somer Valley, which sits just to the south of Bath, belies this simple definition. Using more scientific methods that establish broadcast signal strength, the Somer Valley FM target broadcast area, where listeners could expect good reception on all radios, is shown below.

While the Radio Joint Audience Research study (RAJAR) undoubtedly leads the way in reporting radio listenership, this method isn’t available to the vast majority of community radio stations, like Somer Valley FM, because their broadcast area is too small. The Somer Valley project didn’t seek to mirror RAJAR; instead, it needed to provide a reliable measure of listenership, on a budget.

The challenge for Cognisant was to translate the broadcast area map, with its patchy coverage, into something compatible with Office of National Statistics (ONS) datasets that would allow for a reliable assessment of the resident population.

Using the ONS Neighbourhood Statistics website, Super Output Area Lower Layers were selected to match the coverage areas shown in Figure 1. This mapping exercise defined the geographical boundary of the broadcast area, using data from the from the 2011 Census to estimate a resident population of 48,819, with 39,315 individuals aged 16 or above.

Having defined the population of interest, the method of collection was the next key consideration, as this would dictate how the research subjects would be targeted. On-street interviewing was discounted as it couldn’t guarantee a good spread of respondents from across the broadcast area. While door-to-door surveying was an option, the limited budget meant that cluster sampling would have been needed to make data collection efficient, limiting the geographical coverage of the project. To maximise data collection from across the broadcast area, telephone interviews were chosen, to deliver the most efficient spread of respondents.

The telephone numbers to be called were generated by Random Digit Dialling (RDD), which produced landline numbers across target postcode sectors. The selection of postcode sectors required a further mapping exercise to assess which fell, either in their entirety or the majority, across the target broadcast area. Understanding which postcodes were within the target broadcast area meant respondents could be screened if they lived in an area that wasn’t relevant.

Using the population data obtained from the Neighbourhood Statistics website (Postcode Sector Analysis of the Target Broadcast Area), the data collected from the 500 completed telephone interviews was weighted to ensure it reflected the age and gender profile of the target population, as too much emphasis on a certain section of the population could skew the results.

While this project was primarily about measuring listenership for the local community radio station, the data mapping exercise that went on to define the target population has provided a valuable dataset in its own right, helping the station to understand better the community it serves. The mapping exercises undertaken for the ONS and RDD elements of the project, along with an assessment of fieldwork time, provides Somer Valley FM with the ability to repeat the project, which they intend to do in 2017.

This methodological approach provides valuable insight for any researcher tasked with understanding a population that is not already clearly defined by administrative boundaries.

 

Post Code Sector Classification
BA3 2 Completely in Target Area

 

BA3 3
BS39 7
BA2 0 Majority in Target Area

 

BS39 6
BA3 5