BLOG
30. März 2026
Herbert's World #21
We present issue no. 21 of the Mafo.de column ‘Herbert’s World’!
Our CEO Herbert Höckel discusses a growing data quality problem caused by AI bots, synthetic panels and AI-assisted fraud. His solutions: a better respondent experience, holistic quality management and more “care work” throughout the entire survey lifecycle. Respect, care and genuine participants, rather than mere simulation.
Enjoy the read!
Herbert's World #21
Real care work rather than synthetic convenience: the quality of data from online surveys caught between AI, bots and respondents.
It feels like it wasn’t that long ago that we had a clear set of villains in online surveys: speeders, straightliners and deliberate fraudsters. But today, it’s all packaged in a much sleeker way. Now we’re talking about AI bots, AI-assisted fraud and – as a supposed solution – synthetic panels. In other words, artificial respondents as a countermeasure to fraudulent (or fraudulent-acting) participants. To be honest: you’d have to think of something like that first.
The real problem, however, is very real and quite significant. In 2025, a team of market researchers from India published the results of an international survey of 117 market researchers from 18 countries. The sobering findings: 60% reported data quality issues, 83% worked without a data quality score model, and only around half used sample management tools for fraud detection at all. Furthermore, a significant proportion neglected basics such as trap questions, logic checks or the verification of speeding and outliers.
The authors summed it up as follows: “This highlights a critical gap in the industry and underscores the urgent need for improved data quality standards and methodologies.” Or, to put it another way: we have a quality problem rather than a culture of quality.
Old bias – but now freshly polished.
This is where synthetic panels come into play. The temptation is understandable. Fieldwork time is tight, budgets are under pressure and quotas are hard to meet. So, missing respondents are modelled. It sounds modern and, in a way, scientific, but it is extremely dangerous. For it is well known that artificial respondents do not create a new reality. Instead, they reproduce the same distributions and correlations that were previously recorded. So anyone starting with a small or skewed sample is unlikely to get any closer to the truth by adding synthetic data; instead, they may well cement results that were already distorted to begin with.
That’s not progress. It’s the same old bias – just with a freshly polished surface. Steven Millman wrote in Quirk’s in 2025 that AI-generated imputations do not yet provide reliably accurate margins of error. Statistical tests therefore quickly become a false sense of accuracy.
The succeet26 – which has only just finished – largely reflected the same insight: virtually all market research processes can be automated to a greater or lesser extent. Only the human element at the heart of it cannot. That will have to remain, whether we like it or not.
We want research involving real people, not simulated plausibility.
Synthetised data is particularly problematic where research is at its most valuable: in generating new insights! Whether regarding new products, new market dynamics, new cultural codes or new contradictions in consumer behaviour. This is precisely where synthetic participants reach their limits. For they can only draw conclusions from the past. They recognise patterns, but not surprises or detours. They spot correlations, but not genuine experiences or causal relationships. They simulate plausibility, but not the present, let alone the future. And let’s be honest: anyone in market research who just wants confirmation of what’s already in old data might as well give it a miss. Or simply rely blindly on ChatGPT & Co. Have fun with that!
Question: Despite all the temptations of AI, will we choose to continue providing the economy and society with genuine insights? I certainly hope so! My own conviction on this matter is that market research without real people is not market research. It is modelling. It may be useful, it may be quick, and it may even look nice, but it is not research about people; rather, it is ‘assumptions’ about people.
Black-boxing instead of transparent research.
My esteemed colleague Dirk Engel put it very aptly just a few days ago in his own column on Marktforschung.de: AI systems “recognise patterns and provide quick, plausible answers. But the process behind this remains hidden, which AI critics refer to as ‘black-boxing’.”
That is precisely the point. The human subject therefore remains at the heart of our discipline – not the bot, not the dashboard, and certainly not the statistically tweaked artificial construct. Anyone who seeks to solve the problem of collecting complex data using synthetic respondents – however precisely those personas may be defined – is not solving a quality issue, but merely shifting it to another level. So what should we do?
Firstly: Finally recognising the respondent experience and care work as a quality strategy.
Care work – according to Wikipedia, a term that lies somewhere between ‘caring about’ (i.e. emotional concern) and ‘taking care of’ (i.e. actively looking after and taking responsibility for people). Sounds like THE recipe for us market researchers!
However, in our industry we tend to talk about data quality as if it were purely a technical matter. It isn’t. It’s also about courtesy, respect, accessibility, fairness, a reasonable length and clear communication. It’s also about transparent incentives and the option to exit a survey at any time without having to navigate a maze of digital obstacles.
The Insights Association sums it up in its “Participant Bill of Rights”: poor participant experiences harm data quality just as much as deliberate fraud. However, if we treat participants with respect, they will place their trust in us, resulting in better data, higher completion rates and, overall, a more engaged community. The rule is: if you treat people like disposable items, you shouldn’t be surprised when your data suffers.
Secondly: Quality is essential before, during and after the interview.
The days of isolated, stand-alone measures are over. A Captcha here, an attention check there, and a quick prayer in the direction of the spreadsheet are no longer enough. The market has long since shown the way forward: providers today combine pre-survey fraud prevention with in-survey fraud detection, response scoring, behavioural analysis, open-ended checks, timing patterns and duplicate detection. This is precisely why Rep Data’s acquisition of ReDem in February 2026 is so interesting. Not because of yet another merger, but because of the signal it sends: thinking about quality holistically throughout the ENTIRE survey lifecycle, rather than merely as a clean-up operation at the end of the fieldwork.
Thirdly: People deserve better recruitment rather than being (artificially) replaced.
The future does not lie in replacing the research subject. It lies in making them more accessible. Anyone who needs to reach target groups that are difficult for their clients to access must invest more time, staff and money in panel maintenance, re-contact strategies, profiling quality, community building and well-designed surveys. Yes, AI can help with the latter. With questionnaire validation and logic checks, or with the analysis of open-ended responses. It is also increasingly becoming a useful tool for simulations and early hypothesis testing.
But there is a huge difference between a useful tool and synthetic panels used as a substitute for real respondents. We must not, and should not, blur that distinction simply for the sake of convenience.
A call for real people rather than artificial 'certainty'!
The truth, however hard it may be to accept, is this: the solution does not lie in synthetic panels, but in paying greater attention to – and thus showing respect for – the participants through careful, methodologically sound discipline. So the focus remains on the people, however old-fashioned that may sound.
Ultimately, we surely don’t want to know what an algorithm considers likely. Rather, we want to know what people think, feel, hope for and reject, and why they often do something completely different from what they said they would. The key word here is tolerance of ambiguity. It is not only this that makes market research exciting and fascinating, but also challenging at the same time. And that is precisely why people remain our most important source of data. Thankfully, that's the case.!
Sources:
https://www.marktforschung.de/marktforschung/a/mein-bauch-gehoert-nicht-mehr-mir/ (German only)
https://www.insightsassociation.org/Resources/Data-Quality-Standards/Participant-Bill-of-Rights
https://repdata.com/blog/rep-data-acquires-redem-to-deliver-end-to-end-survey-data-quality/
Herbert Höckel ist geschäftsführender Gesellschafter hier bei bei der moweb research GmbH. Seit mehr als 25 Jahren ist er Marktforscher. 2004 gründete er die moweb GmbH, welche er bis heute als Inhaber führt. Die moweb aus Düsseldorf ist international tätig und eines der ersten deutschen, auf digitale Verfahren spezialisierte Marktforschungsinstitute.