The Fraud Paradox: Most Fraud Looks Clean, Most Bad Data Isn’t Fraud

By Rep Data

• article

• Advanced Statistical Techniques
• Agile Quantitative Research
• Artificial Intelligence
• Brand Research
• Concept Testing
• Customer Experience (CX) Feedback
• Data Analytics
• Data Connectivity
• DIY Surveys
• Full Service Research
• Sample Management
• Survey Research
• Participant Recruitment
• AI

This article is based on the webinar “From Access to Trust: What 20 Years of Online Sample Teaches Us About Surviving the AI Era”. Watch the replay below:

Our research-on-research at Rep Data has produced a finding that still catches people off guard. Fraud and inattention are correlated, but the relationship is relatively weak. We have examined this across multiple studies and recently presented our findings at both IIEX and ESOMAR. This matters because researchers often approach “bad data” as a single problem requiring a single solution, when in reality, fraud and inattention are different issues that require different methods of detection and evaluation.

Most Fraud Looks Good

Start with the uncomfortable half of our findings: an overwhelming majority of fraud would sail straight through traditional data cleaning. This is not because cleaning is ineffective, but because fraudsters, increasingly tech-enabled humans rather than just autonomous bots, are providing responses that appear perfectly legitimate. They do not straight-line (select the same answer repeatedly), rush through surveys or trigger the traditional pattern checks designed to identify suspicious responses.

That is precisely what makes this type of fraud so concerning. Someone accessing surveys through a subnet (a segment of a larger network), web proxy (which masks their IP address) or emulator (which simulates a device), who may have already attempted 300 surveys that day, can provide responses that appear just as credible as those from legitimate participants. Do we want that person in our data? We would argue no, but identifying them requires looking at the technology and behaviour behind the response, rather than relying solely on the quality of the answers.

Are Inattentive Respondents the Same as Fraudsters?

Now consider the other half of our findings, which is equally important. Most inattentive respondents are not fraudsters at all; they are qualified, real people with clean digital fingerprints who may be straight-lining, speeding, or simply going through the motions because they are distracted. Picture someone half-watching their child’s football game while completing a survey. Their technical signals appear legitimate, but their responses may indicate inattention rather than any deliberate attempt to deceive.

Put the two findings together, and the distinction becomes clear. Most fraud appears attentive, while most inattention has nothing to do with fraud. A single filter designed to catch “bad data” will miss much of the fraud because the responses appear legitimate, while potentially misclassifying ordinary distraction as intentional deception. Researchers need to evaluate fraud and respondent attentiveness separately, using different signals and methods rather than relying on a single blended score.

Why Data Quality Requires All Three Pre-, In-, and Post-Survey Checks

That is why we make the case for layering multiple signals rather than relying on any single measure. Some indicators are only available before a respondent answers a question, including device and digital fingerprinting, IP addresses and browser information. Others emerge during the survey itself, such as mouse movements, typing patterns and copy-paste behaviour in open-ended responses. These behavioural signals can help distinguish genuine human engagement from automated or AI-assisted activity. Still other patterns only become apparent after data collection, when researchers can examine responses across the full dataset rather than in isolation.

That last category is where we have started identifying new fraud patterns that traditional checks would likely miss. One example is what we call batch fraud, where clusters of nearly identical responses arrive within minutes or even seconds of one another, often from the same geography. This type of coordinated activity is difficult to detect by examining individual responses, however carefully. It becomes apparent when we evaluate uniqueness and patterns across respondents.

The same principle applies to AI-generated open-ended responses. There are measurable characteristics that can help identify AI-authored text, but much of that content may be copied and pasted from a chatbot rather than entered directly by an automated bot. Copy-paste behaviour can be detected during the survey, while the content itself may require further analysis afterwards. One signal on its own leaves you guessing. A pattern across several signals lets you be genuinely confident that removing a given respondent will improve, not damage, your data.

The False Positive Problem Is Real, and the Solution Is Receipts

There is also a risk of going too far in the other direction. Push too hard on flagging, and you start excluding legitimate respondents, which brings its own cost – and different stakeholders in the research ecosystem have different perspectives on that risk. A supplier trying to fill a quota may be more inclined to accept a borderline respondent, increasing the risk of false negatives. A brand commissioning the research may be more concerned about allowing fraudulent respondents into the dataset, particularly when the findings will inform important business decisions. Both perspectives are understandable, but neither should determine data quality decisions on its own.

The way to manage that tension is to make the reasoning behind every exclusion visible, or what we think of as “giving receipts.” A recommendation to exclude someone should come with the actual reasons attached, including which signals were triggered and how strongly. In practice, this typically involves a weighted combination of several component scores rather than a single indicator. Having a VPN active, on its own, should never be enough to get someone excluded – plenty of ordinary, legitimate respondents use one. But a VPN combined with a public proxy and several other unusual technical markers warrants closer scrutiny. Evaluating multiple signals across pre-survey, in-survey and post-survey checks provides a more reliable basis for making exclusion decisions and reduces the risk of both false positives and false negatives.

It’s a Moving Target, and That’s Fine

We recognise that fraud detection will never reach a finished state. It is an ongoing cat-and-mouse game. As soon as one fraudulent behaviour is identified, new methods emerge to circumvent detection, from more natural mouse movements and varied typing patterns to increasingly sophisticated attempts to evade existing checks. This is precisely why detection methods must continue to evolve rather than relying on what worked against last year’s fraud. Every organisation faces this moving target, but those that struggle most are often the ones still treating fraud and inattention as the same problem, checking for it once and assuming that is enough.

Frequently Asked Questions

What’s the difference between survey fraud and inattention?

Survey fraud involves deliberate deception, while inattention comes from legitimate respondents rushing, straight-lining, or simply not paying attention. Rep Data’s research shows that most fraud looks perfectly legitimate, and most inattentive respondents aren’t fraudsters. Each requires different data quality checks.

Can data cleaning catch survey fraud?

Traditional survey data cleaning catches speeding, straight-lining, and inconsistent answers, but sophisticated fraud often passes those checks. Effective survey fraud detection also requires device intelligence, behavioural analytics, and checks before, during, and after survey completion.

What is batch fraud?

Batch fraud occurs when clusters of nearly identical survey responses arrive within minutes or seconds of one another. Individual answers may look legitimate, but comparing responses across the dataset can reveal suspicious patterns in timing, geography, and content.

Is a VPN enough to exclude a respondent?

No. Plenty of legitimate respondents use VPNs for privacy or security. A VPN combined with other suspicious technical or behavioural signals may warrant further investigation, but respondent validation should never rely on a single indicator.

How do you detect AI-generated open-ended responses?

AI-generated survey responses can look remarkably human. Researchers can analyse the text itself, along with behavioural signals such as typing patterns, mouse movements, and copy-paste activity. Combining these methods provides a more reliable picture than AI text detection alone.

This article is based on the webinar “From Access to Trust: What 20 Years of Online Sample Teaches Us About Surviving the AI Era”. Watch the replay below:

Authors

Learn more about