How reliable are the consumers? Comparison of sensory profiles from consumers and experts

Abstract

This study compares expert and consumer sensory profiles for the same 12 perfumes in different ways: the discriminatory ability and reproducibility are analyzed through ANOVA and the panelists’ consensus through the correlation coefficients. Next, the two product spaces are first analyzed separately for each panel, and then compared through multiple factor analysis. Finally, the two panels are compared using the confidence ellipses methodology. These analyses show that the two panels give similar results with respect to the important criteria for panels (discrimination, consensus, reproducibility). The comparison of the two products spaces shows high similarity. From the confidence ellipses, it can be concluded that no significant differences exist for a given product between the two panels. Hence, in this particular case, the use of consumers appears to be a good alternative to the classical sensory profile provided by a trained panel.

Introduction

In sensory analysis, one of the most important tools is the quantitative characterization of the perceivable product attributes. In the literature, this tool is referred to as “descriptive analysis”, or “profiling” (two frequently used profiling methods are quantitative descriptive analysis (QDA^®, Stone, Sidel, Oliver, Woosley, & Singleton, 1974) and Spectrum^™ (Meilgaard, Civille, & Carr, 2006)). These methods use trained or expert panels. Because of their routinely use of the type of products in question, and because of dedicated training sessions, these panels seem to be more able to characterize products in an accurate way than naïve consumers. On the other hand, hedonic questions are also of great importance and most practitioners use consumers for hedonic tasks. So trained panels are required for sensory profiles and consumers are required for hedonic profiles. In the literature, many warnings are given concerning the use of consumers for profiling:

• “…as with any untrained panel, beyond the overall acceptance judgment there is no assurance that the responses are reliable or valid” (Stone & Sidel, 1993)

• “…consumers can only tell you what they like or dislike” (Lawless & Heymann, 1999)

According to these practitioners, profiling results from consumers lack two essential qualities: consensus between respondents and reproducibility.

Moreover, it has also been shown, that asking consumers liking and intensity questions in the same test can return an unwanted halo effects (Earthy, MacFie, & Hedderley, 1997). The potential impact of the attribute questions on the hedonic ratings is a key point in the objection of the use of getting sensory information from consumers. Since the aim of this paper is to compare experts’ and consumers’ sensory profiles, this point is not studied here.

In market research, most companies need quick answers about their products. Hence, they do not always have the possibility to train panels (which is time consuming). Profiles obtained with consumers can be a good alternative, depending on the type of tests one is interested in, especially in view of the fact that consumers’ profiles also meet the requirements discrimination, panelists’ consensus and reproducibility (Husson, Le Dien, & Pagès, 2001). Moskowitz also showed that consumers can be used to assess the sensory descriptions of sauces, and hence “refutes the notion that consumers are incapable of validly rating the sensory aspects of products” (Moskowitz, 1996).

Because of these two notions (training panels takes time, and consumers are not allowed to profile products), a number of faster methods for collecting sensory data have been developed. Among them is free-choice profiling (Williams & Langron, 1984), Flash profiling (Sieffermann, 2000, Siefferemann, 2002), and ultra flash profiling (Perrin et al., 2008). These methods have in common that they avoid training sessions beforehand, and that they use naïve consumers (Gazano et al., 2005, Nestrud and Lawless, 2008). Paradoxically, it is well accepted that consumers can be used for profiling products using these methods, but the use of consumers with standard QDA^® type profiling using a fixed, predefined vocabulary is still subject to criticism. The question remains: How reliable are consumers? To answer this question, classical sensory profiles, obtained from an expert and a consumer panel on the same products, are compared.