ESOMAR 28 - Questions & Answers
1. What experience does your company have in providing online samples for market research?
Talk Online Panels (hereafter “Talk”) started offering online samples in 2001, at that time under the “MindTake” brand “ name, and has now grown to conduct around 2 million online interviews per year. Talk currently operates in 21 markets through its own and affiliated panels, notably in the DACH, CEE/SEE region as well as Russia, Ukraine, Greece and Turkey. The head office is in Vienna, with additional offices in Leipzig (Germany), Athens (Greece), Sofia (Bulgaria), and Prague (Czech Republic) . Talk employs people locally in most of its countries of operation.
2. Please describe and explain the type(s) of online sample sources from which you get respondents. Are these databases? Actively managed research panels? Direct marketing lists? Social networks? Web intercept (also known as river) samples?
Talk is an online access panel for the sole purpose of conducting research. It is not used for any kind of direct marketing activities. Sources of recruitment differ from country to country but the vast majority of panel members is recruited online through a mix of sources: recruitment through media partners and in the course of conducting other surveys through affiliated research agencies or companies, co-registrations, online advertising, etc. In some countries we have partnerships for offline recruitment through face-to-face interviews and telephone interviews – however, panel members through these sources only account for less than 5% of the overall community members. In order to ensure the quality of the panel we always employ native staff for community management. Intercept and on-site surveys can be provided upon request on a project basis but Talk does not offer them as a standard.
3. If you provide samples from more than one source: How are the different sample sources blended together to ensure validity? How can this be replicated over time to provide reliability? How do you deal with the possibility of duplication of respondents across sources?
Talk Online Panels (in the following “Talk”) works primarily with its own, proprietary panels. In case an additional sample is needed, Talk has a network of trusted partners that can provide it – always transparently and with the explicit permission of the client. In the case of external samples, we always our own duplication detection / fingerprinting system to avoid duplicates as well as technically possible.
4. Are your sample source(s) used solely for market research? If not, what other purposes are they used for?
To insure the quality of the sample delivered, Talk Online Panels uses its sample exclusively for market research.
5. How do you source groups that may be hard to reach on the internet?
In order to find and recruit these hard to reach groups, Talk tries to actively recruit on special-interest portals as well as with targeted online and offline advertising campaigns. Our aim is to expand our respondent base to encompass groups that might be under- represented in mainstream online research efforts. The profile of these groups differs depending on the country.
6. If, on a particular project, you need to supplement your sample(s) with sample(s) from other providers, how do you select those partners? Is it your policy to notify a client in advance when using a third party provider?
In case an additional sample is needed, Talk has the option to work with a small, carefully- selected number of long-term partner providers. Prior to establishing a long-term working relationship with a partner, we conduct a number of test projects, and decide on a partnership depending on the results of those projects. We also constantly monitor the quality of the sample bought, so we can raise any issues with the partner in real-time, and effect a resolution as soon as possible. Talk has the policy of notifying the client in case a partner is needed, and partner samples are only commissioned with the explicit permission of the client.
7. What steps do you take to achieve a representative sample of the target population?
The first step is to establish certain parameters pertaining to the desired target group of a particular survey (for example from the national statistical offices, etc.). An automated tool then picks a random sample from the Talk panel that fulfill these criteria, with the sample size determined both by the desired final target number as well as other conditions such as average response rates. If a client for example wishes for a sample representative of the country, the ratios are determined according to the data of local statistical offices. When inviting potential respondents we also try to space out the surveys sufficiently. Further, along the invitation process, cross-lacing can be specified and within cells weighted random sampling is applied.
8. Do you employ a survey router?
Talk does not use a survey router. Respondents are invited separately to each survey.
9. Please describe the allocation process within your router. How do you decide which surveys might be considered for a respondent? On what priority basis are respondents allocated to surveys?
10.What measures do you take to guard against, or mitigate, bias arising from employing a router?
11.Who in your company sets the parameters of the router? Is it a dedicated team or individual project managers?
12.What profiling data is held on respondents? How is it done? How does this differ across sample sources? How is it kept up-to-date? If no relevant profiling data is held, how are low incidence projects dealt with?
In order to be able to offer its clients targeted surveys, Talk collects detailed information for several key aspects of a person’s sociodemographic profile (such as their household situation, education, work and finances) as well as data on a wide range of consumer and lifestyle habits, from smoking to product usage. This data forms the basis of a filtering system which allows us to conduct surveys using tailor-made samples: we can select respondents according to a single one of the above fields, or a combination of several depending on the client‘s particular needs. The basic data set of gender, age and geographical location is mandatory for becoming a Talk member, while all other profile data is optional -- as completion is incentivized however, Talk has an average profiling degree of around 70%. All profile questionnaires are open for an update once a year.
13. Please describe your survey invitation process. What is the proposition that people are offered to take part in individual surveys? What information about the project itself is given in the process? Apart from direct invitations to specific surveys (or to a router), what other means of invitation to surveys are respondents exposed to?
Panel members who are randomly picked as part of the sample receive a short email with a standardized invitation text stating key information such as survey topic, duration, validity period and points awarded for completion, along with a direct link to the survey homepage. It is crucial to us that the information provided in the invitation email is unbiased and does not include specifications of the actual target group needed in order to prevent biased behavior. Respondents are rewarded through a points-based incentives scheme, depending mainly on the duration, and occasionally also on the complexity of the questionnaire. (NB. We also give consolation incentives to screenouts, as a gesture of goodwill). Each invitation email also includes a link that allows users to manage their account, for example to update their personal data or delete their profile.
14.Please describe the incentives that respondents are offered for taking part in your surveys. How does this differ by sample source, by interview length, by respondent characteristics?
Panel members collect a certain number of points for each survey completed, with the points awarded depending mainly on the duration, and occasionally also on the complexity of the questionnaire. Once they have reached certain milestones (e.g. 2,000 points), panel members can redeem their points within the Talk incentive shop, or they can wait until they reach the next milestone (e.g. 3,000 points) so as to have access to a more ‘expensive’ incentive. Depending on the panel country, the Talk shop will offer a variety of incentive options tailored to specific local interests and conforming to local legislation. Options currently include cash, shopping vouchers for physical or online stores, magazine and newspaper subscriptions and charity donations.
15. What information about a project do you need in order to give an accurate estimate of feasibility using your own resources?
In general, basic parameters needed for feasibility calculations are the specific target group, the screening criteria and incidence rate, the length of interview, and the required quotas. If required by the client, the Talk team can provide the exact national representative distribution for each country, and can also assist with estimating incidence rates. Based on these estimates Talk calculates the project feasibility and sets the price.
16. Do you measure respondent satisfaction? Is this information made available to clients?
We place great value on the user experience of our panel members and actively seek their feedback on the content and presentation of each survey, sometimes with standalone questionnaires but most often with a feedback question at the end of each survey. This feedback question normally includes a request to rate the survey quality numerically as well as an open answer field to justify the rating. At any time and irrespective of surveys, panel members can give their feedback and improvement suggestions on all aspects of the platform and surveys through a feedback form on talkonlinepanel.com.
17. What information do you provide to debrief your client after the project has finished?
We usually agree on the debrief information with each client as part of the project planning. Depending on their specific wishes, the client may get an extensive report with information on all aspects of the survey, from the starting sample, participation rate, drop-out rate, invitation message to a description of the methodology, as well as a more detailed overview of the sample, such as a breakdown by available parameters (e.g. age, location, etc).
18. Who is responsible for data quality checks? If it is you, do you have in place procedures to reduce or eliminate undesired within survey behaviors, such as (a) random responding, (b) Illogical or inconsistent responding, (c) overuse of item non-response (e.g. “Don’t Know”) or (d) speeding (too rapid survey completion)? Please describe these procedures.
There is no single type of survey, whether direct, telephone or computer-assisted that can guarantee results that are 100 percent above suspicion. We recognize the challenges of dealing with something as fickle and complex as human opinion, and we make every possible effort to filter out unsuitable influences. This effort starts during the recruiting process, where we bounce off candidates that seem motivated exclusively by monetary gain, going instead for those genuinely interested in giving honest opinions and helping shape the market. We build on this desire by offering our panelists a broad variety of surveys and a transparent and fair points incentive system. We apply automatic IP and Browser checks and also make regular checks of data quality by looking e.g. at the length of survey times or the quality of open answers, and are always soliciting feedback from clients for surveys where our own checks are not possible. In consultation with our clients we may also integrate control questions in our surveys. Respondents that provide bad data quality are flagged and excluded from future surveys. T
19. How often can the same individual be contacted to take part in a survey within a specified period whether they respond to the contact or not? How does this vary across your sample sources?
Each panel member is invited to take part in surveys maximum six times each month and accounting for screen-outs, ignored invitations or late replies, a respondent completes 1.5 surveys per month on average. This limitation aims to minimize commercialization of the surveys by ‘professional’ survey respondents aiming to make a profit from the bonus points system.
20. How often can the same individual take part in a survey within a specified period? How does this vary across your sample sources? How do you manage this within categories and/or time periods?
In order to avoid category effects our random selection tool invites first those respondents who have not been invited to a survey of the same category the longest. The same holds for the time period. Furthermore, the client can specify a specific time frame for which he needs an absolute fresh sample (or a category fresh sample) and Talk will apply this time / category restrictions when inviting for surveys.
21. Do you maintain individual level data such as recent participation history, date of entry, source, etc., on your survey respondents? Are you able to supply your client with a project analysis of such individual level data?
We have information on entry time, entry source, promoted friends, survey invitations, survey participations, as well as profile updates. We can provide this information to the client if required.
22. Do you have a confirmation of respondent identity procedure? Do you have procedures to detect fraudulent respondents? Please describe these procedures as they are implemented at sample source registration and/or at the point of entry to a survey or router. If you offer B2B samples what are the procedures there, if any?
The reduction of fraudulent respondents starts during the recruiting process, where we bounce off candidates that seem motivated exclusively by gain, going instead for those genuinely interested in giving honest opinions and helping shape the market. If available we check data against publically available address databases and in the lifetime of a panel member we will also try to validate member data by matching it with the information they provide in the course of cashing in incentives (e.g. bank account details contain official names, as do postal addresses). Technically we apply IP and Browser checks to identify people who try to register several times. We do random identity checks by email as well as through telephone.
23. Please describe the ‘opt-in for market research’ processes for all your online sample sources.
Talk only works with double-opt-ins. Each person who registers needs to confirm his or her registration by clicking a link in a follow-up email. If available, Talk also checks address details given by new members at registration against public databases, in order to confirm the authenticity of a member’s identity.
25. Please describe the measures you take to ensure data protection and data security.
Talk Online Panel is committed to responsible and respectful handling of personal information according to the guidelines of the European Union General Data Protection Regulation (GDPR), the Austrian data protection law (Datenschutzgesetz 2000), as well as the strict guidelines laid down by the European Society for Opinion and Marketing Research (ESOMAR) and the association of Austrian market researchers (VMÖ). Talk does not sell or market personalized data to 3rd parties. Due to its multi-layered IT structure, it is ensured that people working with the panel only have access to the data that is absolutely necessary for their work. All staff members have also signed privacy agreements and are updated on privacy regulations at least once per year as part of their ongoing training.
26. What practices do you follow to decide whether online research should be used to present commercially sensitive client data or materials to survey respondents?
When registering with the Talk Online Panel all participants commit to the confidentiality of survey information. In case it is needed, we stress out privacy terms again at the beginning of a survey, and if a respondent does not actively accept, they cannot take the survey. However, if it is crucial for the client to have 100% certainty that no information regarding the content of a study leaks, we advise against conducting online research: despite all security measures there is no 100% certainty. In such cases we would offer face-to-face or telephone interviews or focus group discussions, depending on the project.
27. Are you certified to any specific quality system? If so, which one(s)?
We comply to all ESOMAR principles and are EuroLabel (Austrian E-Commerce label) certified. All relevant tasks within a sampling project are documented in our internal project management tools and checked regularly against benchmarks as part of our internal quality controls.
28. Do you conduct online surveys with children and young people? If so, do you adhere to the standards that ESOMAR provides? What other rules or standards, for example COPPA in the United States, do you comply with?
Talk Online Panels registration is allowed as of 15 years of age, therefore no online surveys are conducted directly with children and younger people. In case younger people are needed for a survey, then it would be designed to be done together with their parents: the invitation would be sent to the parents and they would decide whether or not to ask their children to participate in the specific survey.