Fuel Cycle delivers access to millions of respondents directly to the Fuel Cycle platform and provides a method for our customers to conduct market research studies efficiently and reliably.

Fuel Cycle takes data quality very seriously. Survey data quality is influenced by two primary variables – the first are security practices that prevent fraud, while the second are survey design best practices that affect the respondent experience. Both are discussed below.

Panel Security Practices

Fuel Cycle provides robust security measures on every study. These security measures reduce and prevent fraud in online research while providing a positive user experience to researchers and respondents. Fuel Cycle constantly monitors data integrity across a broad range of surveys and participates in industry discussions on respondent quality. Survey security is an evolving practice and our commitment to our customers is to be at the forefront of panel quality.

Best-in-Class Encryption

  • All Fuel Cycle surveys employ SHA-1 Encryption automatically. SHA-1 is a cryptographic hash function that authenticates a respondents’ source when entering and exiting surveys. This ensures survey respondents cannot be rewarded for taking the same survey multiple times.

Survey Takers are Unique

  • RelevantID provides device fingerprinting on every survey conducted through Fuel Cycle Panels, unless turned off by client request (not suggested).
    • RelevantID is a proprietary technology that gathers many data points from a respondent’s computer, such as operating system version, browser version, plug-in, etc., and assigns a relative weight to each data point. The data gathered is put through deterministic algorithms to create a unique digital fingerprint of each computer.
    • The digital fingerprint identifies duplicate respondents who take the same survey more than once from the same machine. RelevantID flags a computer each time a user tries to take a survey, so it is able to detect if multiple email accounts are being used to take surveys from a single computer. In addition, RelevantID has the unique ability to identify multiple panel accounts from different research firms on the same computer. Suspect respondents are flagged in the system and, based on business rules, are either allowed, redirected or completely filtered out of surveys in which they attempt to participate. In Fuel Cycle’s case, suspect respondents are filtered away from surveys.
    • The process is invisible to the user and does not interfere with the user experience. RelevantID is also consistent with privacy and data protection laws.
  • Unique respondents are tracked with unique panelist IDs and unique IP addresses.
    • In addition to RelevantID, every respondent has a unique panelist ID captured by the survey and must have a unique IP address to gain access to the survey

Speeding Respondents are Tracked

  • Respondents are tracked for negative survey behavior, like speeding. While researchers are encouraged to use red herring questions to identify respondents not paying attention to the survey, Fuel Cycle automatically tracks respondents who complete surveys in less than 20% of the expected interview length

Survey Design to Maximize Data Quality

In addition, customers and partners utilizing Fuel Cycle Panels can ensure their studies are designed in a way that maximizes data quality.

Researchers are often concerned with negative survey respondent behavior, frequently described as “satisficing.” Satisficing is an amalgamation of the words “satisfy” and “suffice” and is used in market research to describe behavior in which respondents give answers that are good-enough but not entirely accurate. Satisficing occurs when respondents seek to minimize cognitive load relative to the expected value they expect to gain from completing a survey. Common satisficing practices include straightlining in grid questions and speeding. Respondents sometimes engage in this behavior when surveys are long or include large numbers of grid question types.

With thoughtful survey design, researchers can help reduce cognitive load for respondents and ensure they capture the highest quality data possible.

  • Keep surveys concise
    • Fuel Cycle’s research in recent years suggests that optimal survey length for data quality is under 11 minutes, or about 30-40 questions
    • Completion rates, satisficing rates, and quality of open-ended responses are all positively affected when surveys are under this range
  • Employ mobile-friendly survey design
    • Ensure your survey is mobile-friendly by using survey software that responds seamlessly to different device types and works well on smartphones and desktops
  • Use red herring questions – but sparingly
    • Red herring questions are often used in a survey battery to “trap” respondents who are not reading every question thoroughly. Red herring questions are an effective instrument, but should typically only be used once in a survey

In research on survey quality, Downes-Le Guin et al. found that “the keys to greater survey engagement lie not in graphical enhancements or greater interactivity in the presentation of survey questions, but rather in dealing more effectively with the fundamental components of respondent burden that survey methodologists have long recognized: survey length, topic salience, cognitive burden (i.e. poorly written or hard to answer questions) and frequency of survey requests.”[1]

In other words, the most effective enhancements researchers can make to enhance data quality are focusing on making surveys simple, short, and easy to complete.

Summary

Data quality is the responsibility of all market research practitioners. Fuel Cycle works diligently to provide best-in-class survey security to our clients through Fuel Cycle Panels. In conjunction, we strongly encourage researchers to create surveys that are respondent-friendly.

[1] “Myths and Realities of Respondent Engagement in Online Surveys.” The International Journal of Market Research. 2012. Volume 54, Issue 5. Page 613.