No company involved with the music industry can ignore the online music market. It is a growing and highly dynamic industry segment. Sony is conscious of this fact and has developed an innovative means to determine the views of the lead users of the Internet to capture the pulses of this market
“It just happens that I’m a member of Sony’s PS3 [Playstation Web site) Voice of the Elite,” says Joseph Laszlo, a diehard netizen and gamer. Voiceof the elite (VOTE) is a Web-based survey run by Sony on the Playstation Web site. The access to the survey is restricted to select members of the PS Web site who are also serious garners
Aside from making the respondent feel a bit important, Sony’s surveys are often very interesting. They share some of their results with the respondents, which often provides new insights. Sony regularly shares a result from an earlier survey as part of a current one; this is a good practice for this kind of ongoing loyalty/market research program since everyone wants to know how his or her opinion matches that of the panel.
Several Web-based services are available for fielding online surveys. Create Survey and Zoomerang let you create and administer online surveys at their sites. The Survey System is a software package available for working with telephone, online, and printed questionnaires. It handles all phases of survey projects, from creating questionnaires through data entry, interviewing, e-mail, or Web page Internet surveys, to producing tables, graphics, and text reports. EFM Feedback . Web Online Surveys , and SurveyPro are other popular software for creating Web-based and other surveys.
Experiencing Survey Research
Use CreateSurvey and Zoomerang to create a survey questionnaire to measure students’ satisfaction with the campus newspaper. Compare the two sites in terms of (I) ease of creating a survey, (2) flexibility of asking different type of questions, and (3) overall satisfaction. 2. Visit the Web sites of any two of the following survey software companies: EFM Feedback .WebOnline Surveys. and Survey Pro If you had to recommend the purchase of survey software, which one of the two would you recommend and why? 3. To experience how a sponsor of an Internet survey can monitor results during the field portion of the project, go to , read the Lightspeed Mini-poll question, and select “View Results.” 4. Visit Greenfield Online ) and take an online survey. Not~ that-you will first have to become a member of the panel to take a survey. Write a brief report about your experience. 5. Visit comScore SurveySite and write a report on SurveySite’s methodology for e-mail surveys, pop-up surveys, and domain departure surveys
A Comparative Evaluation of Survey Methods
Table 6.2 compares the different survey methods across a variety of factors. For any particular research project, the relative importance attached to these factors will vary. These factors may be broadly classified as task, situational. and respondent factors. Task factors relate to tasks that have to be performed to collect the data and to the topic of the survey. These factors consist of diversity of questions and flexibility, use of physical stimuli, sample control, quantity of data, and response rate. The situational factors comprise control of the data-collection environment, control of field force, potential for interviewer bias, speed, and cost. The respondent factors pertain to survey respondents and include perceived anonymity, social desirability, obtaining sensitive information, low incidence rate, and respondent control. We discuss in detail the evaluation of the ditferent survey methods on each of these factors
The demand that the task to be performed places on the respondents and the data collection process influences the survey method that should be used. The nature of the task involved has an
impact on the diversity of questions and flexibility, use of physical stimuli, sample control, quantity of data, and response rate
DIVERSITY OF QUESTIONS AND FLEXIBILITY The diversity of questions that can be asked in a survey and the flexibility of data collection depend upon the degree of interaction the respondent ‘has with the interviewer and the questionnaire, as well as the ability to actually see the questions. A wide variety of questions can be asked in a personal interview because the respondents can see the questionnaire and an interviewer is present to clarify ambiguities Because the respondent and the interviewer meet face to face, the interviewer can administer complex questionnaires, explain and clarify difficult questions, and even utilize unstructured techniques. Thus, in-home, mall-intercept, and CAPI allow for high diversity. Flexibility is also high, especially in the case of=in-home and mall-intercept interviews but is decreased somewhat in CAPI due to limited interaction with the interviewer. In Internet surveys, multimedia capabilities can be utilized and so the ability to ask a diversity of questions is moderate to liigh, despite the absence of an interviewer. Moreover, a certain amount of interactivity can be built into the questionnaire enhancing the flexibility. In mail surveys, mail panels, and e-mail surveys, less diversity and flexibility are possible, as the survey is essentially self-administered by the respondent and there is no interviewer present to offer any clarifications. However, the lack of interaction with the interviewer is moderated by the.ability to see the questionnaire. Therefore, these methods receive a moderate rating. In traditional telephone interviews and CATI, the respondent cannot see the questions while answering and this limits the diversity of questions. For example, in a.telephone interview or CATI, one could
not ask-respondents ‘to rank 15 brands of auto mobiles in terms of preference. However, in CATI,- as in the case of CAPI and Internet surveys, the researcher can personalize the questionnaire and handle complex skip patterns (directions for skipping questions in the questionnaire based on the subject’s responses), thereby-providing some flexibility’ and resulting in a low to moderate rating.
An often-overlooked benefit of Internet survey research is the ease with which an Internet survey can be quickly modified. For example, early data returns may suggest additional questions that should be asked. Changing or adding questions on the fly would be nearly impossible with a mail questionnaire and difficult with personal or telephone questionnaires, but ear. be achieved in a matter of minutes with some Internet survey systems
USE OF PHYSICAL STIMULI Often it is helpful or necessary to use physical stimuli such as the product, a product prototype, commercials, or promotional displays during the interview. For the most basic example, a taste test involves tasting the product. In other cases, photographs, maps, or other audiovisual cues are helpful, In these cases, personal interviews conducted at central
locations (mall-intercept and CAPI) score high and are preferable to in-home interviews, Mail surveys and mail panels are moderate on this dimension, because sometimes it is possible to mail the facilitating aids or even product samples. Internet surveys are also moderately suitable. Because they are Web-based, the questionnaires can include multimedia elements such as prototype Web pages and advertisements. The use of physical stimuli is limited in traditional telephone interviews, CATI, and also in e-mail surveys.
Mall-intercept and CAPI allow only a moderate degree of sample control. Although the interviewer has control over which respondents to intercept, the choice is limited to mall shoppers, and frequent shoppers have a greater probability of being included. Also, potential respondents can intentionally avoid or initiate contact with the interviewer. Compared to mall-intercept, CAPI offers slightly better control, because sampling quotas can be set and respondents randomized automatically.
Moderate to high sampling control can be achieved with traditional telephone interviews and CATI. Telephones offer access to geographically dispersed respondents and hard-to-reach areas. These procedures depend upon a sampling frame-a list of population units with their telephone numbers.l? The sampling frames normally used are telephone directories, but telephone directories are limited in that (I) not everyone has a phone, (2) some people have unlisted phones, and (3) directories do not reflect new phones in service or recently disconnected phones. While the telephone has achieved an almost total penetration of households in the United States, there are some variations by region and within regions. The percentage of households with unlisted numbers is about 31 percent and varies considerably by geographical region. In large metropolitan areas, it may be as high as 60 percent. The total of unpublished numbers and new phones in service since the directory was published can account for as much as 40 percent of total telephone households in some metropolitan areas.
Mail surveys require a list of addresses of individuals or households eligible for inclusion in the sample. Mail surveys can reach geographically dispersed respondents in hard-to-reach areas.20 However, mailing lists are sometimes unavailable, outdated, or incomplete. Typically, telephone and street directories are used for a listing of the general population. Problems with these types of lists have been discussed already. Catalogs of mailing lists contain thousands of lists that can be purchased. Another factor outside the researcher’s control is whether the questionnaire answered and who answers it. Some subjects refuse to respond because of lack of interest or motivation; others cannot respond because they are illiterate. For these reasons, the degree of sample control in mail surveys is low
Mail panels, on the other hand, provide moderate to high control over the sample. They provide samples matched to U.S. Bureau of the Census statistics on key demographic variables. It is also possible to identify specific user groups within a panel and to direct the survey to households with specific characteristics. Specific members of households in the panel can be questioned. FinaIly,low-incidence groups, groups that occur infrequently in the population, can be reached with panels, but there is a question of the extent to which a panel can be considered to be representative of the entire population.
Not all populations are candidates for Internet survey research. The general consumer population is often a poor fit, because many U.S. households do not regularly use Internet services. Although the respondents can be screened to meet qualifying criteria and quotas
imposed, the ability to meet quotas is limited by the number and characteristics of respondents who visit the Web site. However, there are some exceptions to this broad statement. For example, computer products purchasers and users of Internet services are both ideal populations. Business and professional users of Internet services are also an exceIlent population to reach with Internet surveys. Over 90 percent of businesses are currently estimated to have Internet connections. It can be difficult to prevent respondents from completing the Internet survey multiple times. Thus, sample control is low to moderate for Internet surveys .targeted at the general population. E-mail surveys suffer from many of the limitations of mail surveys and thus offer low sample control
QUANTITY OF DATA In-home personal interviews allow the researcher to coIlect large amounts of data. The social relationship between the interviewer and the respondent, as well as the home environment, motivates the respondent to spend more time in the interview. Less effort is required of the respondent in a personal interview than in a telephone or mail interview. The interviewer records answers to open-ended questions and provides visual aids to help with lengthy and complex scales. Some personal interviews last for as long as 75 minutes. In contrast to in-home interviews, mall intercept and CAPI provide only moderate.ameunts of data. Because these interviews are conducted in shopping malls and other central locations, the respondents’ time is more limited. Typically, the interview time is 30 minutes or less .
Mail surveys also yield moderate amounts of data. Fairly long questionnaires can be used, since short questionnaires have not been shown to generate higher response rates than long ones, up to a certain limit. The same is true for e-mail and Internet surveys, although the Internet is a better medium in this respect. Mail panels, on the other hand, can generate large amounts of data because of the special relationship between the panel members and the sponsoring organization.
For example, the author has used the Synovate panel to administer a questionnaire that took more than an hour to complete
Internet surveys have the poorest response rates, even lower than e-mail surveys. This is due to the fact that some respondents may have access to e-rnail but not to the Web, and accessing the Web requires more effort and skill. Furthermore, respondents generally need to be connected to . the Internet while completing a Web survey; they may not be offline, as with e-rnail surveys. If the respondents are prerecruited, they have to log onto a Web site. Many are unwilling to undertake this effort.
A comprehensive, though dated, review of the literature covering 497 response rates in 93 journal articles found weighted average response rates of 81.7,72.3, and 47.3 percent for, respectively, personal, telephone, and mail surveys.i” However, response rates have decreased in recent times. The same review also found that response rates increase with:
• either prepaid or promised monetary incentives
• increase in the amount of monetary incentive
• nonmonetary premiums and rewards (pens, pencils, books)
• preliminary notification
• foot-in-the door techniques
These are multiple request strategies. The first request is relatively small, and all or most people agree to comply. The small request is followed by a larger request, called the critical request, which is actually the target behavior.
• personalization (sending letters addressed to specific individuals)
• follow-up letters
In any practical situation, the researcher has to balance the need to collect accurate and high quality data with the budget and time constraints. The situational factors that are important include control of the data collection environment, control of field force, potential for interviewer bias, speed, and cost. The first three factors have an impact on the accuracy and quality of r two dictate constraints that have to be met.
CONTROL OF THE DATA COLLECTION ENVIRONMENT The degree of control a researcher has over the environment in which the respondent answers the questionnaire is another factor that differentiates the various survey modes. Personal interviews conducted at central locations (mall intercept and CAPI) offer the greatest degree of environmental control. For example, the researcher can set up a special facility for demonstrating the product. In-home personal interviews offer moderate to high control because the interviewer is present. Traditional telephone and CATI offer moderate control. The interviewer cannot see the environment in which the interview is being conducted, but he or she can sense the background conditions and encourage the respondent to be attentive and involved. In mail surveys and panels, e-mail, and Internet surveys, the researcher has little control over the environment, as the interview is self-administered by the respondents in an environment they choose.
CONTROL OF FIELD FORCE The field force consists of interviewers and supervisors involved in data collection. Because they require no such personnel, mail surveys, mail panels, e-mail, and Internet surveys eliminate field force problems. Traditional telephone interviews, CAT!, mall intercept, and CAPI all offer moderate degrees of control because the interviews are conducted at a central location, making supervision relatively simple. In-home personal interviews are problematic in this respect. Since many interviewers work in many different locations, continual
supervision is impractical.P
POTENTIAL FOR INTERVIEWER BIAS An interviewer can bias the results of a survey by the manner in which he or she (1) selects respondents (interviewing somebody else when required to interview the male head of household), (2) asks research questions (omitting questions), and (3) records answers (recording an answer incorrectly or incompletely). The extent of the interviewer’s role determines the potential for billS.26 In-home and mall-intercept personal interviews are highly susceptible to interviewer bias given the face-to-face interaction between the interviewer and the respondent. Traditional telephone interviews and CATI are less susceptible, although the potential is still there; there is interaction but it is not face-to-face. For example, with inflection and tone of voice, interviewers can convey their own attitudes and thereby suggest answers. Computer-assisted personal interviews have a low potential for bias. Mail surveys, mail panels, e-mail, and Internet surveys are free of it.
COST For large samples, the cost of Internet surveys is the lowest. Printing, mailing, keying, and interviewer costs are eliminated, and the incremental costs of each respondent are typically low, so studies with large numbers of respondents can be done at substantial savings compared to mail, telephone, or personal surveys. However, when the sample size is small, the programming costs can be substantial and other low-cost methods such as mail and e-mail surveys should also be considered. Personal interviews tend to be the most expensive mode of data collection per completed response. In general, Internet, e-mail, mail surveys, mail panel, traditional telephone, CATI, CAPI, mall-intercept, and personal in-home interviews require progressively larger field staff and greater supervision and control. Hence, the cost increases in this order. However, relative costs depend on the subject of inquiry and the procedures adopted
Since surveys are generally targeted at specific respondent groups, the respondent characteristics should also be considered while selecting a survey method. These factors include perceived anonymity, social desirability, obtaining sensitive information,low incidence rate, and respondent control.
PERCEIVED ANONYMITY Perceived anonymity refers to the respondents’ perceptions that the interviewer or the researcher will not discern their identities. Perceived anonymity of the respondent is high in mail surveys, mail panels, and Internet surveys because there is no contact with an interviewer while responding. It is low in personal interviews (in-home, mall intercept, and computer-assisted) due to face-to-face contact with the interviewer. Traditional
telephone interviews and CAT! fall in the middle as the contact with the interviewer is not face-to-face. It is also moderate with e-mail. Although there is no contact with the interviewer, respondents know that their names can be located on the return e-mail.
LOW INCIDENCE RATE Incidence rate refers to the rate of occurrence or the percentage of persons eligible to participate in the study. As will be discussed in more detail in incidence rate determines how many contacts need to be screened for a given sample size requirement. There are times when the researcher is faced with a situation where the incidence rate of the survey respondents is low. This is generally the case when the population represents a niche or a highly targeted market. rather than the general population. Suppose a study of cosmetics for Avon calls for a sample of females age 18 to 27 who have used a cosmetic foundation at least twice in the past week. Estimates show that such people who would qualify to participate in the study represent only 5 percent of the population. Thus, only I in every 20 people in the general population would qualify, resulting in a lot of wasted effort if the general population is sampled. In such cases, a survey method should be selected that can locate the qualified respondents efficiently and minimize waste. The telephone interview can be very effective (high rating) as a method of screening potential respondents to determine eligibility. All it takes is a phone call. The three personal methods (in-home, mall intercept, and CAPI) are all inefficient because the interviewer has to make personal contact with potential respondents. Three of the self-administered methods (mail, mail panel, and e-mail) are moderate in terms of efficiency as all three are relatively low cost and can be used to contact a large number of potential respondents so the desired sample size of qualified respondents is obtained. The Internet, however, is very good in this respect (high rating). In an Internet survey, screening questions can be used to weed out ineligible respondents quickly and efficiently.
RESPONDENT CONTROL Methods that allow respondent control over the interviewing process will solicit greater cooperation and are therefore desirable. Two aspects of control are particularly important to the respondents. The first is control over when to answer the survey, and the fle ibility to even answer it in parts at different times, especially if the survey is long. The second aspect of control pertains to the ability of the respondent to regulate the rate at which she answers the survey, i.e., the flexibility to answer the survey at her own pace. Three of the self-administered methods, namely mail, mail panel, and e-mail, are the best (high rating) in terms of imparting this control to respondents. Some control is lost in Internet surveys, because in random pop-up surveys the respondents do not have the flexibility of answering at a later time. However, Internet surveys can be designed to allow respondents to come back and complete them, resulting in moderate to high rating. Telephone surveys are low to moderate as the pace is regulated by the interviewer, and although the telephone call can be rescheduled, the respondent must commit to a specific time. All the three personal methods, namely in-home, mall intercept, and CAPI, are low on this factor as the pace is regulated by the interviewer, and generally the interview cannot be rescheduled.