Short and shorter—make the most of your survey length

Kelcey Curtis | Mar 24, 2017 Kelcey Curtis 03/24/17

With the growing popularity of mobile devices, short surveys have become a hot topic for market researchers. The problem tends to be finding a balance in length—surveys need to be short enough to keep a respondent’s attention, but long enough to provide a comprehensive view of the customer experience. A big question we often get from clients is—how short is too short?

How short is too short?

We’ve had brands test survey lengths that range from a handful to over fifty questions. Testing a shortened survey allows us to determine if the cuts we’re making are improving the survey experience for respondents while still giving us the information we need to produce great insights. Most survey tests tend to fall into two groups: short and shorter.

We broke down the two most common short survey test scenarios to find out which one had the biggest pay-off for clients.

Brief is best

We always aim to keep surveys short. Our research has found that respondents tend to abandon the survey when it takes longer than 7 minutes. However, it’s common for survey length to creep out of control over time.

Survey Abondonment Rates image

To keep your survey length reigned-in, it’s critical to conduct frequent audits and ensure you’re only collecting the information you need. Taking inventory of what’s in your survey keeps survey length down and the respondent experience positive. Get more about details on keeping your survey length in control here.

Must be this tall to ride

Clients have put our recommended template to the test by creating an even shorter version that only includes the core satisfaction questions. With this shorter survey test, questions that determine the right path for the respondent—like visit type—are cut out. Here’s what we learned from testing the shorter surveys:

  • Even though these clients saw decreases in two key indicators of good survey health—duration and abandonment—they also lost some key survey content
  • Scores dropped on questions that were previously using survey branching logic, since we weren’t asking the right people relevant questions about their visit
  • Shorter surveys tend to remove more questions but fewer pages
  • Most of the clients who tested the shorter survey returned to the recommended survey because they were losing vital information
Survey health 1

In their own words

A common misconception with creating a shorter survey is that respondents will be more likely to leave a lengthy comment if they're putting less effort into answering the survey content. When we put this notion to the test, it turned out there was no difference between the percentage of comments captured in a shorter survey than were captured in an average length survey. There also are no notable differences in comment length—comments in shorter surveys tend to contain a similar number of characters, words, and sentences as average length surveys. In more extreme "short survey" scenarios—where only OSAT and an open-end are included—we actually see comment quality decrease. Comments in these super short surveys are less rich, contain fewer words or sentences, and also include fewer topics.

Why does that happen? When SMG designs surveys, the questions are designed to help the customer reflect on all aspects of their experience. When the survey content gets too narrow, open-end comments end up being less comprehensive.

Key takeaways

Shortening surveys has its pros and cons. When surveys are too long, using SMG best practices can greatly improve the respondent’s survey experience—which means less abandonment and more actionable insights for your brand. But the benefits of shortening a survey (decreased duration, abandonment) still have to be weighed against the cons (loss of data, loss of survey branching). SMG’s survey experience and research on research can ensure that your surveys are set up to give you all the data you need to make the right decisions for your brand, while keeping your survey efficient for your customers.

Want more best practices for surveys that get the best results? Check out our white papers on survey design, parts one and two here.

Kelcey Curtis, M.A.
Research Manager

Kalli Hannam, M.A.
Research Analyst

Customer Experience Update