The credibility and reliability of test markets are hindered or negatively impacted by biased and prejudiced responses.

October 19, 2019

Every marketing executive must understand the necessity and values of test marketing. The problem is when the product is partly ready it is introduced to the market and the marketers does not know when the test market is in order. Compounding this difficulty is that the goals of test marketing are sometimes unclear and that the information, once gathered, is often improperly used. The biases involved can encompass both favorable and unfavorable assessments—manifest involuntarily without an individual’s awareness or intentional control.

Unconscious biases in the workplace can hinder diversity, recruiting, and retention efforts and can shape an organization’s culture in detrimental ways. It can also hinder talent and performance reviews. As researcher Hayley Roberts wrote, More than 85 percent of all Americans consider themselves to be unprejudiced. This means that if a research is conducted on these people majority of them will hold some degree of implicit racial bias.

An effective marketing planning process is central to conventional prescriptions for the implementation of marketing.

A thesis done by (Flintermann, 2014) on the quality of market research was selected to discuss further about this topic. This thesis focuses on the case of Marketline Advantage and the quality of its market research reports published about the automotive industry.

Importance of identifying bias in test market
According to EMC (‘Digital Universe’ report from 2014), the quantity of data is going to see a tenfold increase between 2013 and 2020. Therefore most of the decisions will be made considering the insights supported by adequate data, analytics, research methods and techniques.
However it is important to remember that decisions are mostly not necessarily guaranteed to be successful just because they are based on data. These data can be sometimes misinterpreted or miscalculated. This leads the final outcome of the analysis to be taken as true and the decisions based on that turnout to have been incorrect.
The main reason for these biased wrong decisions are because of the interference in the outcomes of research by predetermined ideas, prejudice or influence in a certain direction. Not only data but the people who are analyzing the data can also be biased when taking the sample or analyzing the results.

Research Methodology
In the thesis in order to find an answer to the quality of the market research, a secondary research was conducted through access to the market research which was under investigation. These reports were easily accessible, time saving and free of cost. Besides Marketline Advantage data, secondary data from sources such as government agencies or associations are used. They even collected their own data by conducting interviews with managers with the research reports on focus.

Data Analysis
In order to develop a better understanding on the reliability/validity of Marketline Advantage research reports, it is compared to other companies such as KPMG, the ACEA, OICA and Statista Inc. When the companies were compared, the process of preparation, data collection and analysis are not clearly presented by the Marketline Advantage. The quality check data, data is coded or transformed how it is dealt with missing data are unclear.

Result analysis of the specification of the data collection and analysis
The reports of KPMG reports provide all the information, about the data used and in-depth interviews to whom it was conducted. Even though they KPMG published a transparency report in which it is described how the quality of the data used and reports published is controlled, information on how it is dealt with missing data is not presented by KPMG on the research reports.

On the other hand ACEA, OICA and Statista Inc. does not provide any information on the research type, how data is transformed and coded, quality is controlled or missing data is handled. Therefore with the information provided by the companies there may be an issue with the reliability as it cannot be confirmed how the findings and interpretations were derived.

Besides presenting the individual degree of reliability and validity per indicator and company, the traffic light rating system is used to effectively visualize the differences between Marketline Advantage and the companies compared to it. The colors used have the following meaning: red = potential issue with reliability/validity, yellow = could /could not be an issue with reliability/validity, green = no potential issue with reliability/validity.

Results analysis of the specification about potential changes in procedures
After analyzing the specifications about potential changes in procedures, the collection or assembly methods are not to be found in the market research report. Like Marketline Advantage, neither KPMG (2011a), the OICA (2014), the ACEA (2014) nor Statista Inc. (2014b) provide information on potential changes in methods, sources or definitions used from one report to another. These lack of miss communication of information about potential changes can be an issue with reliability.  
According to the given table below all the companies reliability and validity rate are at low.

Identifying the bias is a difficult task to do. But the research conducted shows that one loses control over how data was generated and collected, data is already modified and poor documentation exists. These leads to an issue with reliability and credibility of the analyzed market research. Therefore the outcomes of the research and analysis are influenced by bias. Data collection and analyzing is a shared responsibility of everyone directly involved (including the marketer and the analysist) to reach a verdict to the bias. The researchers should always rely on the correct facts.

Flintermann, B. (2014). The quality of market research reports (The case of Marketline Advantage and the automobile industry). The case of Marketline Advantage and the automobile industry, 3-35.

Share this article :
Blogger Tips and TricksLatest Tips And TricksBlogger Tricks

FB Page

Copyright © 166/A-1/2017/19. ELITE Institute - All Rights Reserved