To Print: Click here or Select File/ Print from your Browser Menu.
Article printed from
Discovery Surveys, Inc. http://www.DiscoverySurveys.com
By Bruce L. Katcher, Ph.D. President, Discovery Surveys,
This article originally appeared in Folio magazine
To get the right answers, you must
ask the right questions,
at the right time, and in the right format. Here's how.
You have urgent questions that need answers:
Why are we losing circulation?
How do readers feel about our format?
What changes do our readers want?
What issues are most important to our readers?
How can we show a potential advertiser that a large segment of our readership values its products?
These are the types of questions that can be answered with readership surveys. But, to get answers that are valid, that pass the credibility test, the survey must be properly conducted. Here are some important guidelines.
Choose the right survey method. There are four basic approaches to gathering the opinions of readers: focus groups, telephone surveys, mail surveys, and in-magazine surveys. Each method is appropriate for particular purpose.
Focus groups are ideal for in-depth exploration of readers' opinions when the specific questions you want to ask are unclear. For example, the publisher of a retirement-related publication wanted a deeper understanding of how readers felt about the magazine. Several focus groups were held in which the participants read portions of the publication and were then asked their general opinions. The publishing team watched the groups from behind a one-way window and were able to gain valuable insights.
Mail surveys are most appropriate when you want close control over who receives the survey. For example, if you have advertisers who want to know more about the purchasing patterns of a particular subset of your readers, a survey mailed to only that subset should do the trick. Because mail surveys are addressed to specific individuals, they generally yield better response rates than in-magazine surveys. And you can optimize the response rate by sending a pre-survey mailing to announce the survey, and a post-survey mailing to remind participants to complete the survey.
Telephone surveys are often a good compromise between a mail survey and a focus group. Interviews can target specific individuals (whether they are current readers or not). And follow-up questions can be asked for further clarification. However, such surveys, are very labor intensive and do not produce the quantifiable data obtained with either mail or in-magazine surveys.
In-magazine surveys are an inexpensive and easy way to reach all your readers, but response rates are typically low, and the results are not statistically representative of your entire readership. Using the results for PR purposes, therefore, is usually not possible. On the other hand, this can be an ideal way to generate reader interest in a particular topic. Engineering News-Record, a national McGraw-Hill title, conducts periodic in-magazine surveys asking readers about career issues. The results are then printed in a special advertising section funded by recruiting firms. And women's magazines, fitness magazines, and health and lifestyle magazines all profit from editorial that is based on responses to in-magazine surveys on topics of high interest to their readers.
Keep it simple. Mailed surveys or in-magazine surveys should be kept to a single page. Questions should be closed-ended, since open-ended or fill-in-the-blanks questions are more difficult for readers to complete -- and cumbersome to analyze. Rating scales, checklists or true-or-false questions work best. It is also important to use the same response scale throughout the survey. It is difficult, for example, for readers to alternate between a five-point rating scale and a ranking scale. For mailed surveys, readers should be provided with a self addressed, postage-paid reply envelope.
Ask interesting questions first. Studies have shown that asking the questions that readers find the most interesting at the beginning of the survey increases the chances that readers will complete the survey and return it to you.
Don't get too personal. Respondents are often understandably unwilling to give their names, addresses, phone numbers or incomes. However, if this information is vital to the purpose of the survey, ask for it at the end of the survey. Respondents are more likely to answer these questions once they have psychologically committed to the survey.
Offer an incentive. Respondent incentives can increase response rates. There are many different approaches to incentives to respondents. One publication we work with offers a copy of the survey results as well as a chance to win $500. Sometimes we offer to contribute $25 to a well-respected charity for each completed survey we receive. Placing a $1 bill in a mail survey can also improve the response rate.
Plan the data analysis ahead of time. If, for example, you want to know how three different subsets of your readers respond to a particular set of questions, print the survey in three different colors. That way, you will be able to analyze the data for each group without compromising the confidentiality of respondents. Or, if you plan to include any open-ended questions, make certain you have the available administrative resources to categorize or type the written comments.
Just as important as knowing what to do is knowing what not to do. In terms of reader research, there are three big pitfalls to be avoided.
Respondents to in-magazine surveys are self-selecting, and as such can bias results. Although, there are some weighting strategies that can be used to correct for sampling bias, a mail survey is better suited for obtaining statistically valid results. In any event, plan ahead: Know who you want to sample -- readers, readers and non-readers, specific subsets of your readers, and so on -- and design your survey to reach them.
Unless you take some precautions, the results of in-magazine readership surveys can be altered by readers. State clearly that photocopies of survey forms will not be accepted, and that each reader may submit only one survey. Consider the following potential problem. A popular consumer-oriented magazine published an annual survey to assess the attitudes of its readers toward different insurance companies. One company was rated consistently low on claim service. The company's CEO, who was fighting mad, asked his research director how many magazines the company would have to buy to dramatically improve its claims rating. In the end, the CEO decided not to stuff the ballot box -- but he could easily have done so.
If you ask your readers for their opinions about any aspect of your magazine, take what they say very seriously. Listen to them if they want change. If you allow your survey results to waste away on a dusty shelf, you may soon find yourself and your magazine competing for space on the same spot.