Product Information:-

  • Journals
  • Books
  • Case Studies
  • Regional information

How to... design a survey

Options:     Print Version - How to... design a survey, part 3 Print view

Planning issues

The first issue to be concerned with when planning a survey is, what do I need to do to answer my research question? How will the instrument which I develop answer the needs of the basic question(s) I am trying to answer? And what is the population that I need to obtain the data from?

As a related issue, you need to decide if you wish to test a related theory, and hypothesis.

From this basic issue arise a whole range of other issues:

  • Who has done a similar or related survey?
  • From whom do I need to get the data – what is my population and how should I sample it?
  • What is the nature of the data that I need to obtain, and what should the mode of data collection be?
  • Will there be problems in obtaining the data from the respondents?
  • How much will it cost and how long will it take?

Who has done a similar or related survey?

As in any research exercise, you need to start off doing a literature survey. This will help uncover whether anyone has done exactly the same thing, or something similar, in which case you may be able to borrow ideas relating to their sampling technique, questionnaire design, method of analysis, etc.

From whom do I need to get the data?

The key here is to locate all members of the population you are researching and ensure that they have an equal chance of being covered.

Who is the research population? Over what geographical area is it located? You should try and be as accurate as possible about whom you target – perhaps addressing the key spender in the household, or if you are looking at a firm, finding the likely title of the person whom you will be targetting.

What sample (i.e. the actual group, the subset) of the population will you be targetting? How large should this sample be to be representative? This decision will also partly depend on how easy people are going to be to locate, and what the response rate is likely to be (the lower the response rate is likely to be, the bigger the sample you should try and obtain), but also on the types of statistical tests that you will want to perform.

Generally speaking, you will be better off getting a probability sample, that is, every member of the population has an equal chance of being chosen.

What is your sampling frame (the place where you will identify and locate your sample, e.g. a directory of members of a marketing association)?

In an article, (International Journal of Public Sector Management, Vol. 13 No. 2), Rachel Parker and Lisa Bradley surveyed the public sector in Queensland and adopted the following approach to sampling:

"A mailing list was developed from the Public Sector Directory – executive level, which was accessed over the Internet. This is a list of Queensland public sector employees with managerial responsibilities, listed by their department. The researchers chose six of the largest departments that as a group carried out a range of activities undertaken in the public sector including central co-ordination, infrastructure provision and social services. A sampling frame of 530 names was compiled from the six departments, which included all executive level names listed in this directory."

In an article, (Robert E. Morgan, Carolyn A. Strong and Tony McGuinness, European Journal of Marketing, Vol. 37 No. 10), the authors use a particular sampling frame in order to exclude smaller firms, and within the sample targetted a particular individual. For full details, look under the heading "Research method".

What is the nature of the data I need to obtain?

  • Can I identify all the variables at the outset, in such a way that I can obtain all the information by asking closed-ended questions?
  • Or, are the data less easy to capture and would it be more appropriate to ask open-ended questions?

If you are attempting to survey a large population, then a written questionnaire is probably more appropriate, particularly if the population is dispersed over a wide geographical area. On the other hand, if you are wanting to explore more in depth issues, and you have a smaller population, then an oral, interview survey would be better.

All these questions will help determine your choice of research instrument, as in the example below:

In (Journal of Small Business and Enterprise Development, Vol. 7 No. 2), Wai-sum Siu uses the interview technique despite what they perceived as its drawbacks, because this technique had been shown to be effective in Mainland China. A questionnaire instrument was devised and administered by a team of professors in Mainland China.

Will there be problems in obtaining the data from the respondents?

Will there be a problem of bias amongst the respondents – for example, will they tell you what they think you want to hear, will they be afraid to answer questions on certain issues, will they just not tell the truth?

Nearly every opinion poll in the run up to the 1992 UK general election failed to predict the eventual victory by the Conservative Party. A subsequent investigation found some technical reasons for this, but considered that just as important was the reluctance of many respondents to admit to voting Conservative on the grounds of fear of tax increases under Labour, not wishing to appear motivated by self-interest.

Will respondents just simply fail to respond, thus creating a small sample with an inbuilt bias?

A survey was conducted of non-UK students studying on a UK-based distance learning degree. Two-thousand students studying in various centres were sent a questionnaire. The response rate was in the region of 10 per cent, and while the responses were positive, ability to deduce anything from this is affected by the low response – were people only motivated to respond if they had something positive to say?

Will the respondent actually be who it says?

This is particularly important in surveys in companies, where you may have deliberately targetted e.g. the marketing director, who then passes it off to a more junior colleague.

How much will it cost and how long will it take?

Budgeting and scheduling your survey is of fundamental importance. Your departure point as always should be your research question, but you should also consider the actual time you have available, which may be framed by academic deadlines, or by your need to find something out in time to realize a business project. Likewise, you may be working to a limited budget. There will always be a trade-off between the research question and the available resources, and the former needs to be revised if the latter are not sufficient to produce the necessary data.

Schedule

To determine how long the survey will take, you need to break it down into its stages, as in the following example of a questionnaire schedule (note, some stages, notably 4 and 5, can be done simultaneously):

  1. Decide the research questions that need to be answered, and the major objectives.
  2. Read around to see what other surveys have been done.
  3. Decide the most appropriate method, and the what instrument.
  4. Determine the research population and the sample and locate the sampling frame.
  5. Prepare the questions.
  6. Design the questionnaire.
  7. Trial with a small sample.
  8. Finalize the questions.
  9. Prepare and duplicate the instrument.
  10. Send out the questionnaire.
  11. Response time – allow sufficient time, but not too long.
  12. Send out reminders.
  13. Collect in responses.
  14. Transfer responses to a computer file.
  15. Analyse responses.
  16. Produce report.

Budget

The following need to be taken into consideration:

  1. Staff time for planning.
  2. Cost of directories etc. for sampling frame.
  3. Pre-testing the questionnaire.
  4. Hiring interviewers.
  5. Designing and printing the questionnaire.
  6. Postage.
  7. Cleaning the data so that they can be put into an Excel file.
  8. Computer analysis.