While often portrayed ominously by Hollywood as the technology that will one day take over the world from humans, Artificial Intelligence (AI) in real life has proven to be capable of replacing humans, but in a manner that’s productive to us. This is due to AI’s ability to process massive amounts of unstructured datasets in a short time and extract useful information, discern patterns and carry out actions based on the analysis.
It is this capability of AI that makes it perfectly suitable to take over roles currently held by humans in the area of market research. While AI is already being employed on a limited or trial basis by market researchers, continued improvements in the areas of machine learning (ML) will drastically improve AI’s capability to analyse qualitative and quantitative data, turning it into the perfect MR tool.
The ultimate goal is to offload manual tasks to AI and derive costs savings through improved efficiency and reduced human intervention. In this context, here are 4 areas of MR where AI can play a crucial role.
1. Community managementManaging the sheer number of respondents makes managing panel communities traditionally a tedious manual task, and at the same time, the ideal area for AI-techniques to show shine. The idea is simple enough. An ML system is trained to recognize behavioural patterns of individual panelists by feeding it the all available data. Within a short period of time, the system becomes capable of predicting panelist behaviour. This capability can be applied in 3 key areas of community management, an exercise that Borderless Access has already commenced last year.
a) Predictive sampling: Manually sending samples is a low-efficiency exercise that generally results in poor response rates. In contrast, automation of sampling through ML can significantly improve the outcome. AI, trained with panelists’ historical data about survey response rates and completion rates and send out samples based on this knowledge to achieve the best possible results.Such a system can also be trained on panelist profiling parameters to predict future panelist behaviour and further improve the sampling efficiency.
The best part about predictive sampling is that it can be practically applied to the current state of AI. At Borderless Access, we implement supervised and unsupervised machine learning techniques in sampling using SmartSightTM – Our Intelligent Community Management Platform!
Besides significantly improving better survey response and completion outcomes, our predictive models are also trained to generate panelist ranking to support various business objectives.
b) Dynamic incentive management: Once again, predictive models can be employed to automatically set, as well as distribute incentives given out to panelists. This is one of the areas where our aforementioned panelist ranking system comes into the picture. Borderless Access has taken this step further where the AI can automatically modify the incentives in the middle of a project based on situational requirements.This enables us to bring a weak survey back on track through better incentives or reduce expenditure on a project that’s sure to achieve the desired results.
ML-based incentive management could also be utilised to reduce panelist churn rate, where the system learns from a panelists’ behaviour and rewards them accordingly to retain them as well as improve their productivity and quality. We also use the same raking system to root out non-responsive and non-productive panelists to streamline a panel community.
c) Demand-based recruitment: Finding the right set of respondents, specifically at the time of requirement is a constant challenge. By designing an ML model to analyse and learn from past project requirements, Borderless Access’ ML model can be employed to predict future project needs in terms of respondents as well as the best ways to fulfill these requirements. Such a system can determine in advance, the right target audience for recruitment as well as recommend the most suitable recruitment channels in terms of cost, quality and performance.
The functionality of such a demand-to-supply matching ML model can be further expanded to include the ability to provide recommendations in terms of which project is likely to convert (and at what CPI) and why a particular RFP was lost.
2. Making sense of open-ended responsesWe now move from AI in community management to its applications in the research part of MR. Natural Language Processing (NLP) and Natural Language Generation (NLG) are areas of AI that are advancing steadily.
What this means is that AI is getting more and more reliable at understanding human language (written as well as spoken) and its ability to make sense of word associations and even sentiments is good enough for practical applications such as processing open-ended data – a section of data that’s tedious to process manually and where humans are prone to error. What’s more, on a large scale, AI can accomplish this task significantly faster than humans. This is yet another area of AI implementation that we as a new age digital market research company have already started exploring in our operations.
3. Virtual panels
While human respondents cannot be done without in most cases and have their advantages, there are instances where a virtual respondent makes sense and not to mention, are literally free of cost. The idea is to create virtual respondent clusters based on various human behavioural attributes based on the characteristics of the existing panel of respondents. These virtual panels could then be engaged whenever required (including simultaneously) and derive results that could potentially be more accurate than the results obtained from actual panelists.
While the data gathered from such a panel could be considered representative, the quality of results can be continuously improved with time by feeding the ML system with more real panelist data, removing biases and cleansing the dataset off errors. Theoretically, overtime, a virtual panel could provide higher-quality results than actual respondents. On the flip side, errors and biases in the datasets used to teach the ML system would result in skewed and false results, which is why we haven’t treaded this path, as quality & right data and insights are our top priority.
4. Virtual moderators
Virtual moderators would perform the same tasks as human moderators, anything from conducting surveys to generating reports from survey results. But in a much shorter time, with fewer errors and for free of cost. The limitations of virtual moderators are similar to the panel counterparts, in that they can only be as good as the dataset they are trained on.
An AI-powered moderator can also automatically send out questionnaires to the right set of respondents, at the right time and location to achieve the best survey outcome. We have spoken in detail about the implementation of AI-based survey moderation in our earlier blog.
Another area where virtual moderators and panels can shine is in the case of unconscious errors and bias, which are introduced due to natural human limitations and are impossible to completely get rid of. These could be as simple as missing out on a survey question, which inadvertently alters the outcome of a survey. Since an AI can perform repetitive tasks efficiently and quickly and is devoid of human shortcomings such as forgetfulness, fatigue, etc., the outcome is of a higher quality.
Conclusion
The aim of implementing AI in market research is not, as it might appear, to replace humans, but to enhance our capabilities to maximise efficiency. While AI could take over certain activities from humans, in the long run, this will only create new opportunities and avenues of growth for the market research industry.
