ESOMAR 28 - Questions & Answers
1. What experience does your company have in providing online samples for market research?
Talk Online Panel (hereafter “Talk”) is a full-service research panel provider, currently operating online communities with a total of 1.3 million members across 25 countries in Europe. Talk focuses on the German-speaking area as well as Central, Eastern and Southeastern Europe. Talk stands for high data quality, panel management expertise and complete data protection compliance, ensured by the highest IT and project management standards, confirmed by an ISO 20252 certification.
Talk collects digital, anonymized data from computer-assisted surveys conducted on our network of panels across Europe. Drawing on a strong IT heritage and in-house technical team, Talk has been among the first to implement responsive and mobile-first interfaces for on-the-go completion of surveys, cross-device and on our native apps. Talk also has cookie tracking technologies measuring user traffic off a network of reputable websites as well as from the tracking of panelists. Through its native app Talk also does geo-tracking of its panelists across all countries.
With its headquarters in Vienna, and regional offices in Leipzig, Prague, Sofia, Bucharest, Sarajevo, Riga, Zurich and Athens, Talk additionally boasts local sales representatives and community assistants in place in each country where we run an online access panel - these native speakers know the culture and current market conditions, so can advise on cultural sensitivities and tailor the incentive portfolio to local tastes. The combination of international expertise and local know-how means Talk can deliver consistent high quality and top-level client service across all of the company’s European panels.
2. Please describe and explain the type(s) of online sample sources from which you get respondents. Are these databases? Actively managed research panels? Direct marketing lists? Social networks? Web intercept (also known as river) samples?
Talk’s proprietary panels are actively maintained online access panels for the sole purpose of conducting research. They are not used for any kind of direct marketing activities. Sources of recruitment differ from country to country but the vast majority of panel members is recruited online through a mix of sources: recruitment through media partners, marketing partnerships such as social media, paid advertising, newsletter, search marketing, and in the course of conducting other surveys through affiliated research agencies or companies. Furthermore, registrations in the online panels are generated through referrals and active registration. In a couple of our countries we have partnerships for offline recruitment through face-to-face interviews and telephone interviews – however, panel members through these sources only account for less than 5% of the overall community members. In order to ensure the quality of the panel we always employ native staff for community management. Intercept and on-site surveys can be provided upon request on a project basis, but Talk does not offer them as a standard.
3. If you provide samples from more than one source: How are the different sample sources blended together to ensure validity? How can this be replicated over time to provide reliability? How do you deal with the possibility of duplication of respondents across sources?
Talk works primarily with its own proprietary panels. In case an additional sample is needed, Talk has a network of trusted partners that can provide it – always transparently and with the explicit permission of the client.
As part of panel management protocol, entries made by panelists when registering are checked for duplications using our own identification algorithms that flag suspicious profiles for manual review by a support agent. When using external samples, we have our own fingerprinting system to detect and avoid duplicate people as far as technically possible.
4. Are your sample source(s) used solely for market research? If not, what other purposes are they used for?
To insure the quality of the sample delivered, Talk Online Panels uses its sample exclusively for market research.
5. How do you source groups that may be hard to reach on the internet?
As a result of Talk’s diversified recruitment processes, the Internet population can be portrayed in a representative way. Through our deep profiling special-interest target groups can be addressed directly in most cases.
We always aim to expand our respondent base to encompass groups that might be underrepresented in mainstream online research efforts. In order to find and recruit such hard to reach groups Talk tries to actively recruit on special-interest portals as well as with targeted online and offline advertising campaigns.
6. If, on a particular project, you need to supplement your sample(s) with sample(s) from other providers, how do you select those partners? Is it your policy to notify a client in advance when using a third party provider?
In case an additional sample is needed, Talk has the option to work with a carefully selected number of long-term partner providers. Prior to establishing a long-term working relationship with a partner, we conduct a number of test projects, and decide on a partnership model (or not) depending on the results of those projects. Further, we constantly monitor the quality of the sample bought, so we can raise any issues with the partner in real-time and effect a resolution as soon as possible. As per our general philosophy of transparency, Talk follows the policy of notifying the client in case a partner is needed. Partner samples are only commissioned with the explicit permission of the client.
7. What steps do you take to achieve a representative sample of the target population?
The sampling is generally based on parameters and quotas used to define the representativeness of the sample for the specific project - usually specified by the client or researched at statistical offices. Our panel management system then selects randomly the sample from the Talk panel who meet these criteria. The sample size is determined both by the desired final target number as well as other conditions such as average response rates. If a client for example wishes a sample which is representative of the country, the ratios are determined according to the data of local statistical offices. When inviting potential respondents, the system also assures technically that one respondent is not invited too often. Further, along the invitation process, cross-lacing can be specified and within cells weighted random sampling is applied.
8. Do you employ a survey router?
Talk does not use a survey router. Respondents are invited separately to each survey.
9. Please describe the allocation process within your router. How do you decide which surveys might be considered for a respondent? On what priority basis are respondents allocated to surveys?
10.What measures do you take to guard against, or mitigate, bias arising from employing a router?
11.Who in your company sets the parameters of the router? Is it a dedicated team or individual project managers?
12.What profiling data is held on respondents? How is it done? How does this differ across sample sources? How is it kept up-to-date? If no relevant profiling data is held, how are low incidence projects dealt with?
In order to be able to offer its clients targeted surveys, Talk collects detailed information for several key aspects of a person’s sociodemographic profile (such as their household situation, education, work and finances) as well as data on a wide range of consumer and lifestyle habits, from smoking to product usage. This data forms the basis of a filtering system which allows us to conduct surveys using tailor-made samples: we can select respondents according to a single one of the above fields, or a combination of several depending on a client‘s particular needs. The basic data set of gender, age and geographical location is mandatory for becoming a Talk panel member, while all other profile data is optional. As completion is incentivized however, Talk has an average profiling degree of around 70%. All profile questionnaires are available for members to update in their profile at any time. According to the ISO20252 specifications, all profile information is due to be updated at least once a year. For this purpose, we actively invite all panel members, depending on the half-life period of certain profile categories, up to four times per year. Both the polling of the profile data and its storage comply with data protection regulations.
13. Please describe your survey invitation process. What is the proposition that people are offered to take part in individual surveys? What information about the project itself is given in the process? Apart from direct invitations to specific surveys (or to a router), what other means of invitation to surveys are respondents exposed to?
Panel members who are randomly picked as part of the sample receive a short e-mail as the exclusive mean of invitation, with a standardized invitation text stating key information such as a general survey topic, duration, validity period, device compatibility, and points awarded for completion, along with a direct link to the survey homepage. It is crucial to us that the information provided in the invitation email is unbiased and does not include specifications of the actual target group needed in order to prevent biased behavior.
Generally, our panel members are invited to take part in the survey to express their opinion so that companies can improve their products. Nevertheless, we reward respondents through a points-based incentive system to compensate their efforts.
Each invitation email also includes a link that allows users to manage their account, for example to update their personal data or, according to data protection regulations, to delete their profile.
14.Please describe the incentives that respondents are offered for taking part in your surveys. How does this differ by sample source, by interview length, by respondent characteristics?
Panel members collect a certain number of points for each survey completed, with the points awarded depending mainly on the survey duration, and occasionally also on the complexity of the questionnaire. As a gesture of goodwill, we give consolation incentives also to screened-out respondents to also reward their efforts.
Once they have reached certain milestones (e.g. 2,000 points), panel members can redeem their points within the Talk incentive shop, or they can wait until they reach the next milestone. Depending on the panel country, the Talk shop will offer a variety of incentive options tailored to specific local interests and conforming to local legislation. Options currently include cash, shopping vouchers for physical or online stores, magazine and newspaper subscriptions and charity donations.
In general, for the points structure Talk follows ESOMAR recommendations. Further, the points value is adapted according to the standard of living and purchasing value in each country. Talk remains committed to both quality and variety in its loyalty program towards panel members, which is also explained in its Terms and Conditions of Use.
15. What information about a project do you need in order to give an accurate estimate of feasibility using your own resources?
In general, basic parameters needed for feasibility calculations are the specific target group, at best defined by the screening criteria and/or an incidence rate. Required quotas as well as the planned length of interview are further necessary criteria. If required by the client, the Talk team can provide the exact national representative distribution for each country and can also assist with estimating incidence rates. Based on these estimates Talk calculates the project feasibility and sets the price.
16. Do you measure respondent satisfaction? Is this information made available to clients?
We place great value on the user experience of our panelists and actively seek their feedback on the content and presentation of each survey, sometimes with standalone questionnaires but most often with a feedback question at the end of each survey. This feedback question normally includes a request to rate the survey quality numerically as well as an open answer field to justify the rating. On request we make the results available to our clients.
At any time and irrespective of surveys, panel members can give their feedback and improvement suggestions on all aspects of the platform and surveys through a feedback form on each local-language page of talkonlinepanel.com.
17. What information do you provide to debrief your client after the project has finished?
As part of standard project planning, we discuss and agree with each client the specifics of the debrief and intermediate reports, should they be requested. Depending on their wishes, and in accordance with our obligations under our ISO certification, the client may get an extensive report with information on all aspects of the survey: this debrief information may contain the starting sample, participation rate, drop-out rate, invitation message as well as a description of the sampling methodology, or a more detailed overview of the sample, such as a breakdown by available parameters (e.g. age, location). As a minimum, however, customers will receive debrief information on the core specifications such as incidence rate, interview length and number of interviews as well as quotas if applicable.
18. Who is responsible for data quality checks? If it is you, do you have in place procedures to reduce or eliminate undesired within survey behaviors, such as (a) random responding, (b) Illogical or inconsistent responding, (c) overuse of item non-response (e.g. “Don’t Know”) or (d) speeding (too rapid survey completion)? Please describe these procedures.
There is no single type of survey, whether direct, telephone or computer-assisted that can guarantee results that are 100 percent above suspicion. We recognize the challenges of dealing with something as fickle and complex as human opinion, and we make every possible effort to filter out unsuitable influences. This effort starts during the recruiting process. We specifically select the recruitment sources and bounce off candidates that seem motivated exclusively by monetary gain. We target those genuinely interested in giving honest opinions and helping shape the market. Further, continuous automated quality checks are carried out for all participant input processes to maintain panel quality and reduce the proportion of fraudsters and speeders to a minimum.
Quality checks associated with surveys are carried out as part of data cleansing whenever we carry out the programming and hosting of the data for the client. The data analysis and cleansing follow specifically combined criteria that allow the identification of random or inconsistent responses as well as overuse of non-response items and speeding by checking the response behavior, quality of open answers, and the response time.
Further, Talk has an automated escalation process in place, in which suspicious respondents are marked and warned in stages by e-mail - the final step is getting blacklisted. Feedback from sample-only customers on data quality, which we very much welcome, is processed in the same way.
19. How often can the same individual be contacted to take part in a survey within a specified period whether they respond to the contact or not? How does this vary across your sample sources?
Each panel member is invited to take part in surveys a maximum of six times a month. Accounting for unaccepted invitations or screen-outs, a respondent completes 1.5 surveys per month on average. In individual cases, one reminder is sent out for a survey, at maximum two.
This limitation aims to minimize commercialization of the panel by ‘professional’ survey respondents aiming to make a profit from the bonus points system.
20. How often can the same individual take part in a survey within a specified period? How does this vary across your sample sources? How do you manage this within categories and/or time periods?
By default, a panel member can only participate once in a survey, provided he or she has completed it, as complete, screen-out or quotafull. A renewed access to the questionnaire is then no longer possible.
The invitation and participation in surveys is automatically recorded in the system for each panel member.
Furthermore, the client can specify a specific time frame for which he needs an absolute fresh sample (or a category fresh sample) and Talk will apply this time / category restrictions when inviting for surveys. Especially for tracking studies, the exclusion of participants of the preliminary waves can be ensured according to agreement.
21. Do you maintain individual level data such as recent participation history, date of entry, source, etc., on your survey respondents? Are you able to supply your client with a project analysis of such individual level data?
Information relevant to project work, such as entry time, entry source, promoted friends, survey invitations, survey participations (also associated quotas) as well as profile data and updates, will be provided in anonymized form upon request and in accordance with data protection regulations. Further information about the panel members is reserved for the purpose of panelist support by the respective teams, according to data protection regulations.
22. Do you have a confirmation of respondent identity procedure? Do you have procedures to detect fraudulent respondents? Please describe these procedures as they are implemented at sample source registration and/or at the point of entry to a survey or router. If you offer B2B samples what are the procedures there, if any?
Quality checks to verify panel members and identify fraudulent participants take place at almost all touchpoints, starting with the very selective use of recruitment sources. Talk uses quality assessment procedures for all instances of participant data input such as registration or profile information (plausibility), and incentive booking (e.g. address details, name). Finally, an escalation process up to exclusion from the panel also exists for identified bad quality-respondents in surveys.
The quality assessment procedures include a continuous automated process based on plausibility checks of the data entries as well as IP and browser ID checks. In a further step, a manual evaluation is carried out by a team of community assistants that are familiar with local specificities and are trained to validate the data via secondary research or even a personal contact.
23. Please describe the ‘opt-in for market research’ processes for all your online sample sources.
Talk works exclusively with a Triple-opt-in registration. Each person who registers needs to confirm his or her registration by clicking a link in a follow-up email (double opt-in). In order to complete the registration, the respondent needs to fill in a basic set of demographic profile data (such as gender, age, education) and further has to participate in at least 1 survey (triple opt-in).
25. Please describe the measures you take to ensure data protection and data security.
Talk Online Panel is committed to responsible and respectful handling of personal information according to the European General Data Protection Regulations as well as the strict guidelines laid down by the European Society for Opinion and Marketing Research (ESOMAR) and the national market research associations of all the countries we are running online-panels. Not least, our ISO certification ISO 20252 confirms our compliance with all applicable legal regulations.
Talk does not sell or market personalized data to 3rd parties. Due to its multi-layered IT structure, it is ensured that people working with the panel only have access to the data that is absolutely necessary for their work. All staff members have signed privacy agreements, are familiar with Talk’s Technical Organizational Measures and are updated on privacy regulations at least once a year as part of ongoing training. Compliance with data protection regulations in daily work is continuously monitored.
26. What practices do you follow to decide whether online research should be used to present commercially sensitive client data or materials to survey respondents?
When registering with the Talk Online Panel, all participants commit to the confidentiality of survey information. In case it is needed, we reiterate the privacy terms at the beginning of a survey, and if a respondent does not actively accept, they cannot take the survey.
However, if it is crucial for the client to have absolute certainty that no information regarding the content of a study leaks, we advise against conducting online research: despite all security measures, it is not practically possible to guard 100% against leaks, benevolent or malevolent. In such cases we would offer face-to-face or telephone interviews or focus group discussions, depending on the project.
27. Are you certified to any specific quality system? If so, which one(s)?
We have adhered to all ESOMAR principles since the launch of the panels, and we are additionally certified according to ISO 20252. This confirms compliance with all legal business requirements, a responsible approach towards panel members and the highest possible transparency towards customers.
28. Do you conduct online surveys with children and young people? If so, do you adhere to the standards that ESOMAR provides? What other rules or standards, for example COPPA in the United States, do you comply with?
The minimum age for registration in our Online Panels is determined by the applicable laws in the respective countries - Talk’s legal team checks for updates on the legislation on a regular basis and changes are implemented whenever needed. The registration process always follows local legislation, so persons under the legal age in their country jurisdiction are not technically allowed to join the panel – if they register fraudulently, they are discovered during later quality control identity checks and excluded.
As a general rule, no online surveys are conducted directly with children. In case the input of children is needed for a survey, then it would be designed to be done together with their parents: the invitation would be sent to the parents, who would need to consent to their child participating.