Section Links

Survey Tips

Sample size

  • Keep sample small
  • Can tell respondents that they are part of a select group (appeals to scarcity)
  • If using a sample of the population, you can administer multiple surveys, using different people, so that folks don’t get tired of being surveyed.

How many times do you contact folks?

  • Send at least 3 reminders
  • Change the day of the week and time of day that you send reminders, because students and professors tend to have times that they are more likely to respond.
  • Be sure to remove those who have already responded from your reminder list.

Use a mixed-method of contact

  • Use of both paper and email contacts has been found to be effective. For example, send a letter from the provost before the survey.
  • Follow-up with a postcard
  • A study has shown that sending a post-card before the survey has increased response rate by 9% (Public Opinion Quarterly)

Structure of email

  • Subject line
    • Don’t use the word “survey”
    • Mention EMU to make your email stand out from SPAM
    • On the final reminder, one study found that having the subject line blank got the most responses.
  • Body of email
    • Keep it short and use text only (not html because some SPAM filters look for html)
    • Put the hyperlink near the top of the email.
    • Include contact information out of courtesy.
  • Techniques
    • Tell respondents that they are part of a select group.
    • Give a deadline; limited amount of time to complete.
    • Ask for their help (i.e., appeal to their helping tendencies).
    • Tell them to let you know if they don’t want to be contacted again.
    • Give a rough estimate of completion time.
    • Promise confidentiality, if you can. But, don’t overemphasize it. Overstating confidentiality can decrease response rates and leads to suspiciousness.
    • Make sure that you can defend the content of your email to the Provost and President.
  • Things that typically cause emails to go into ‘Junk Mail’ as SPAM
    • Capital letters
    • “from” name contains underlines or numbers (e.g., sally_52)
    • A large number of blank lines or spaces in the body of the text
    • “Click here” or “Click below”; it is better to use the word “submit”

Response Rate

  • Rule of thumb – at least 50%
  • If your response rate is low, check to see how the demographics of your sample compare to the population.
  • Send follow-up emails not more than 1 week apart.
  • After the first 2-3 days, the response rate typically tends to drop off.
  • Once the response rate drops off, send another follow-up email.

Web Design

  • Appearance
    • Have a short introduction at the top
    • Avoid drop-down boxes (they are easy to miss); use radio buttons or check boxes instead.
    • Make sure that the layout is vertical and that respondents don’t have to scroll horizontally.
    • Use plenty of white space and headings.
    • Number your questions so that people don’t get lost.
    • Use yes/no instead of having them check all that apply (will be easier to use the data and forces respondents to read all of the options).
      Example:
        Yes   No  
    Library?   Library   Library  
    Athletic Facilities?   Athletic Facilities   Athletic Facilities  
    Dining Hall?   Dining Hall   Dining Hall  
  • Length
    • Don’t ask demographic information if you already have it.
    • If it is long use multiple pages (downloads quicker).
    • Don’t force responses.
    • Keep the survey under 20-30 minutes to complete.
  • Salience of the survey topic
    • This is one of the best predictors of survey response.
    • Put interesting questions near the top and boring questions near the end.
  • Open-ended questions
    • Use of “other” often indicates that you have not formulated your responses well.
    • Keep short, for your own sake.
  • End of survey
    • Make sure the “submit” button is visible.
    • Have a “thank you” page after the submit button.
      • Lets them know the survey is over.
      • Can also use it to direct them to another website, or collect another survey.
  • Timeline
    • Start early, at least a month before you want to the survey to go out.
    • Developing a survey is a long process and there are often many refinements along the way.
    • Allow time for pilot testing.
    • Take feedback seriously.

Resources

  • Don Dillman is the guru in this area and has published studies on web surveys.
  • (Hartzler Library has 1983 – present.)
  • (Hartzler Library does not subscribe.)
  • Groves et al. (1992) “Understanding the decision to participate in a survey.” Public Opinion Quarterly, 56:475-495.

Modified from Steven Porter’s presentation for an Academic Impressions webinar on February 15, 2005.