The Internet has grown exponentially over the last decade from its humble beginnings as a tool created by the US military in the Cold War, to its present incarnation as a seemingly omnipresent entity that, in the Western world at least, has revolutionised a lot of business and consumer behaviour (Harter, 1999). One consequence of this so-called “Internet revolution” is the impact that it may have in providing an alternative research tool for market research agencies. The Chairman of Harris Black International was quoted as saying, “all research is going to migrate to the Internet” (McDaniel and Gates, 2001) and ESOMAR also predicted that 30 per cent of research world-wide will be conducted online by the year 2010 (Savage, 2001). Although these predictions exist, there is very little research, particularly outside the USA, to determine the current usage of Internet based research and the experiences of those involved in providing such methodology.
This paper will endeavour to redress the paucity of non-US research in the area of Internet market research by examining how online market research is utilised within UK market research agencies and what opportunities or problems the market research industry is experiencing with this new research tool.
Perceived advantages and disadvantages of Internet based research
The literature suggests a number of positive and negative attributes of the new research medium. These are as follows:
1. Ease of use
If a researcher is using an Internet survey much of the administration burden of sending/receiving questionnaires and data entry is significantly reduced. Once a questionnaire is completed the data are already in electronic form and can be downloaded instantaneously into a database for complex analysis (Iyer, 1996). In effect, once the last questionnaire for a survey is complete, researchers have all the data stored in a database at their fingertips instantaneously. Reminders to complete the questionnaire can be permanently placed on Web sites or can be sent out by e-mail at no additional cost and with minimal effort.
The cost of sending e-mails or setting up a Web questionnaire is much cheaper than mail equivalents. Weible and Wallace (1998) estimated that the variable cost of sending a questionnaire (via e-mail) or survey invitations (for a Web survey) to 150 additional e-mail addresses was the equivalent of adding one extra contact to a mailing sample. Watt (1997) states that Internet interviewing becomes significantly cheaper than postal surveys particularly when there are over 500 respondents. Below this number, the set up costs will tend to make the traditional postal method more cost effective.
3. Sampling difficulties
In the UK the number of people that have Internet access has increased from approximately 960,000 in June 1997, to 19.98 million in November 2000 that represents an estimated 33.58 per cent of Britain's population (NUA Web site, 2001). Despite these impressive growth rates the penetration of the Internet has not, as yet, developed to the extent that it represents the population as a whole. Taylor (2000) describes the Internet population as being younger and possessing a higher standard of education than the public at large. This may not be as big a problem as it seems, for example in the USA, Internet research tends to focus on doing research aimed at the middle to upper income segment (Mehta and Sivadas, 1995; Oppermann, 1995).
There is, however, a major difficulty relating to the lack of a centralised database of e-mail addresses (Litvin and Kar, 2001). This is in stark contrast to mail surveys where addresses are commonly found in a wide variety of directories and databases. Even where sample frames are available, they frequently become outdated as Internet users change their e-mail providers (Dommeyer and Moriarty, 2000). In a study undertaken by Oppermann (1995) it was discovered that of the 500 e-mail addresses that were selected from the members directory of Association of American Geographers (AAG), 25 per cent were out of date. The advantage to researchers using e-mail (whether to deliver a questionnaire or as an invitation to a Web site survey) is that the error messages are sent back to the sender of the e-mail allowing the researcher (as in the case of Oppermann, 1995) to select another set of individuals from the list in order to attempt to obtain the sample size that the researcher originally envisaged.
Without a centralised e-mail database that covers the whole online population, researchers must use alternative methods to get their samples. Mehta and Sivadas (1995) selected their respondents by initiating a programme that collected the e-mail addresses from individuals that posted discussion articles on the 20 most popular discussion groups within an Internet community known as Usenet, which is basically a world-wide system of discussion groups (Kumar et al., 1999; Jackson and DeComormier, 1999). Although this method was efficient at collecting a large database of e-mail addresses it still may fail to develop a sample that is representative of the population as a whole, it may include the segment that contribute to discussion groups.
4. Response rates
Another attribute of Internet surveys surrounds the responses from the sample. Most studies agree that when comparing e-mail surveys with mail surveys, response rates can be rapid, in some extremes overnight (Mehta and Sivads, 1995; Tse et al., 1994; Bachmann et al., 2000; Taylor, 2000).
As with normal postal surveys, responses to Internet surveys tend to vary according to the study. However, Table I indicates how various studies have been constructed and how the response rates have differed. It generally appears from the summary in Table I that, with the exception of Parker's (1992) study, response rates from e-mail surveys tend to be lower than those of traditional postal surveys.
A concern brought up by Bachmann et al. (2000) is that, as the population becomes more accustomed to e-mail they may become more reluctant to respond to surveys. Essentially as the “novelty” of the Internet wears off there is the potential that online research may suffer as Internet users become more apathetic towards such studies. Burkeman (2001) has indicated that the volume of e-mails has increased substantially to the point that the phrase “e-mail burnout” has been created. He states that the volume of e-mail that an individual can receive is so great that many delete up to 60 per cent of their messages based on the subject line alone.
Due to these poor return rates (even if they tend to be much faster) there has been some investigation into using Internet survey response as an option in a mixed mode methodology. The reason for this probably came from studies such as Parker (1992) where a small proportion (28 per cent) of the e-mail respondents decided to print out hard copies of the questionnaire and send them back by more conventional mailing systems.
5. Response quality
If the consensus of the literature reviewed to date is that the responses from Internet surveys (mainly e-mail questionnaires) are lower than postal surveys then there also tends to be agreement within the literature that online surveys tend to have response quality equal if not superior to the other forms of survey. In the studies by Tse et al. (1994; Tse, 1998) the quality of answers was comparable to those from the postal survey. In fact, Bachmann et al. (2000) indicated that participants of online surveys were much more willing to respond to open-ended questions. Mehta and Sivadas (1995) also discovered that responses to open-ended questions in their Internet survey were significantly longer than those obtained through similar postal surveys.
Taking account of these issues (ease of use; cost; sampling difficulties; response rates and response quality) there is a need to understand how Internet based marketing research is currently being used by the market research industry and the extent to which the research tool is creating opportunities or problems.
The research involved sending 120 postal questionnaires to all of the member agencies of the UK's Market Research Society (the leading professional body for marketing research in the UK) that claimed to provide Internet surveys to clients. These claims were made in the agencies' entries within the members' directory.
A total of 64 usable responses (53 per cent response rate) was obtained, of which 53 (83 per cent) undertook Internet based research for clients and 11 (17 per cent) indicated that although they planned to offer such research had not as yet done so.
The majority of the respondents had only started undertaking Internet surveys during the last 24 months. However, just over half of the respondents (55 per cent) considered Internet based research to be very important to their organisation. During the previous 12 months, 57 per cent of the respondents had undertaken more than five online studies.
Reasons for offering Internet surveys
The majority of respondents that provided Internet surveys agreed that the reasons for moving into this area were principally:
- to have a balanced portfolio of research methodologies; and
- to develop new research areas .
Responding to client demand was not viewed as such a significant motivational force, neither was the desire to become an online survey specialist.
Types of market research
As shows, the majority of research being undertaken is quantitative in nature and is delivered through e-mail, pre-recruited panels or through electronic mailing lists/discussion groups. Of the respondents, 47 per cent combined Internet research with other methodologies in a mixed mode approach with techniques such as telephone or postal surveys. Only 23 per cent of the respondents had attempted qualitative research on the Web by using on-line discussion groups.
In terms of the purpose of the research, Internet based market research is frequently linked to Web-based activities such as Web site evaluation or to research areas where respondents have easy access to the Web such as is the case with employee surveys and those in business to business research.
Types of client
Internet based research tends to be conducted for organisations that have a significant Internet presence such as financial services, Internet based businesses, IT, the media and telecommunications. This may reflect the fact that Web site evaluations are one of the main types of research.
Attitudes towards Internet surveys
sets out respondents' agreement with a variety of attitude statements. As expected, the most important factor for determining whether Internet survey methodology is used is the extent to which the audience is online. The majority of respondents agreed that Internet research required less administration and saw no significant problems with security. However, there were concerns about the problems associated with sampling, drop out rates for panels and response rates. Although 42 per cent of respondents thought that Internet surveys are used for important decision making research, it is interesting to note that more than one third of respondents felt that they were not. This negative viewpoint is supported by the 25-30 per cent of respondents who stated that cost or speed rather than quality determined when online surveys are used. This suggests that there are still doubts about Internet based research and these doubts are also evident by the large proportion of respondents who were undecided and failed to agree or disagree with many of the statements.
Only 7 per cent of respondents expected the number of Internet based studies to increase significantly over the next year. The remainder expected no growth or only a moderate increase. So there does not seem to be any evidence of the dramatic growth that many industry commentators were predicting. Some commentators had suggested that potential clients would start carrying out their own online surveys, bypassing agencies due to the cost advantages and the availability of technology. However, only 10 per cent of the agencies agreed that they would be doing less Internet research in the future as a result of clients doing their own online research .
In relation to other commentators' predictions, the majority of market research firms from the study did not agree (64 per cent) that most of their quantitative research would be done through Internet surveys in the future. Almost all of the respondents agreed (91 per cent) with the statement that their firm would only use their online surveys as one part of a portfolio of research offerings and emphasised that they had no intention of becoming specialised in the use of Internet surveys as a data collection tool. However, 65 per cent of agencies that responded to this statement agreed they would be looking to use their Internet surveys for a wider range of market research study (for example - employee surveys) in the future.
The main findings from the study show that Internet surveys are generally offered by marketing research agencies in order to provide a balanced portfolio of research services to their clients. The Internet can enable researchers to access certain markets and audiences such as teenagers and business people that may prove difficult through traditional research methods. However, the dramatic growth in Internet surveys predicted by industry commentators is not evident among practising market researchers. The use of Internet research is predominately limited to specialist types of research study such as Web site evaluation, business to business and employee research, where the audience is more likely to be online, or are part of an easily accessible database list supplied by clients. As a result the sectoral concentration of activity relates to those sectors that make most use of the Internet such as financial services, Internet businesses and IT.
In terms of operational issues, there are continuing concerns about sample frames, attrition of panel members and response rates, although it is also noticeable that many attitudinal responses in the study were neutral suggesting that much of the industry has still to reach a verdict on the problems and opportunities posed by Internet research. Generally, there is not an issue with the ability of Internet surveys to collect data, the doubts relate more to the representativeness of any data collected. The nature of Internet sample frames and the characteristics of the people willing to take time to respond to Internet surveys or participate in Internet panels is likely to mean that respondents may not be representative of typical computer users, never mind the general public. This may explain why the majority of market research companies do not see Internet based research as being their main approach to quantitative research in the future, although they do agree that Internet based research will continue to make up part of their portfolio of services.
As to future academic research, there is certainly a need to track the changing attitudes towards Internet surveys over time as more and more people go online. Changes in technology, such as interactive digital broadcasting and Web-enabled mobile telephones, combined with a more widespread use of the Internet for shopping, banking and education, are likely to alter the usage and appropriateness of online research. Research is also required to determine whether attitudes towards Internet surveys are the same world-wide. Attitudes towards other methodologies vary from country to country; do similar variations occur with Internet surveys? Finally, if Internet based research is to grow, there also needs to be significantly more research undertaken to address the perceived weaknesses (i.e. sampling, response rates and panel drop out rates) of the methodology. Many practitioners are still undecided about Internet methods and need further academic research and guidance if Web-based research is to become a serious alternative to traditional research methods.
For the majority of organisations, this research has shown that Internet based research should be seen as an additional supporting methodology rather than as an alternative to traditional research approaches. In other words, the Internet is best used alongside other methodologies to pick up additional respondent groups such as business people and teenagers who may be more difficult to access through personal, telephone or postal surveys. However, for some organisations requiring specialist areas of research such as employee surveys and Web site evaluation, an Internet survey may be the most cost effective and appropriate way of carrying out the research. Wherever Internet surveys are used, a word of caution, clients and researchers should pay particular attention to the extent which the survey respondents are representative of the specific population of interest. Although, this issue may become less of a problem in the future, sample frames and response rates will continue to be critical in evaluating the worth of Internet surveys in the short to medium term.
Bachmann, D.P., Elfrink, J., Vazzana, G. (2000), "E-mail and snail mail face off in rematch", Marketing Research, Vol. 11 No.4, pp.10.
Burkeman, O. (2001), "Postmodern", Guardian Newspaper G2 Section, pp.2-3.
Dommeyer, C.J., Moriarty, E. (2000), "Comparing two forms of an e-mail survey: embedded vs attached", Journal of the Market Research Society, Vol. 42 No.1, pp.39-50.
Harter, B. (1999), "On-line data collection", Wireless Review, Vol. 16 No.18, pp.54-62.
Iyer, R. (1996), "The Internet: a new opportunity for marketing research firms", Quirk's Marketing Research Review, .
Jackson, A., DeCormier, R. (1999), "E-mail survey response rates: targeting increases response", Marketing Intelligence & Planning, Vol. 17 No.3, pp.135-9.
Kumar, V., Aaker, D.A., Day, G.S. (1999), Essentials of Marketing Research, John Wiley & Sons, New York, NY, .
Litvin, S.W., Kar, G.H. (2001), "E-surveying for tourism research: legitimate tool or a researcher's fantasy?", Journal of Travel Research, Vol. 39 No.3, pp.308.
McDaniel, C., Gates, R. (2001), Marketing Research Essentials, 3rd ed., South-Western College Publishing, Cincinatti, OH, .
Mehta, R., Sivades, E. (1995), "Comparing response rates and response content in mail versus electronic mail surveys", Journal of the Market Research Society, Vol. 37 No.4, pp.429-40.
NUA Web site, www.nua.net/surveys/how.many.online/europe/html, .
Oppermann, M. (1995), "E-mail surveys - potentials and pitfalls", Marketing Research, Vol. 7 No.3, pp.28.
Parker, L. (1992), "Collecting data the e-mail way", Training and Development, pp.135-9.
Savage, M. (2001), "Road to nowhere?", Research, No.421, pp.20-1.
Sheehan, K.B., McMillan, S.J. (1999), "Response variation in e-mail surveys: an exploration", Journal of Advertising Research, Vol. 39 No.4, pp.45.
Taylor, H. (2000), "Does Internet research work?", International Journal of Market Research, Vol. 42 No.1, pp.51-63.
Tse, A., Ching, R., Ding, Y.B., Fong, R., Yeung, E., Au, A. (1994), "A comparison of the effectiveness of mail and facsimile as survey media on response rate, speed and quality", Journal of the Market Research Society, Vol. 36 No.2, pp.349-56.
Tse, A.C.B. (1998), "Comparing the response rate, response speed and response quality of two methods of sending questionnaires: e-mail vs mail", Journal of the Market Research Society, Vol. 40 No.4, pp.353.
Watt, J. (1997), "Using the Internet for quantitative research", Quirk's Marketing Research Review, .
Weible, R., Wallace, J. (1998), "Cyber research: the impact of the Internet on data collection", Marketing Research, Vol. 10 No.3, pp.19-24.