Metaknowledge

Why is the “wisdom of crowds” sometimes spectacularly wrong? To assess the accuracy of a crowd, you need to look at the individuals forming the group – more specifically, at the individuals’ “metaknowledge.”

Metaknowledge involves individuals’ awareness of how their knowledge about a given subject “stands in relation to other people’s” knowledge of the same topic. High levels of expertise tend to correlate with high levels of metaknowledge: The more you know about a subject, the more likely you are to predict the knowledge level of others accurately. Thus, by comparing how closely an individual’s metaknowledge – his or her prediction of how others will respond – correlates with the average response of the group, researchers can determine a particular individual’s value to the “hive mind.”

“Metaknowledge means you are aware of what you know or don’t know, and of where your level of knowledge stands in relation to other people’s.”

Measuring metaknowledge can help researchers improve the accuracy of survey data. Metaknowledge can serve as a kind of “truth serum” for survey respondents: Those with less metaknowledge on a given topic are more likely to base their predictions of the group response on their own beliefs. This phenomenon, called the “false-consensus effect,” helps reveal private beliefs that respondents might deny publicly.

“Experts are more likely to recognize that other people will disagree with them. Novices betray themselves by being unable to fathom any position other than their own.”

Metaknowledge can help prompt more truthful survey responses: When researchers presented participants in a study with an incentive for truth-telling and told them that their survey answers would be measured against metaknowledge, people answered survey questions with greater honesty.

Measuring metaknowledge can also help researchers differentiate between group members who know the answers in a test and members who are guessing. In one survey, researchers asked students to name state capitals and to predict the percentage of their peers who would know the answers. Once the individuals with low metaknowledge were eliminated from the group, the accuracy of crowd wisdom improved significantly. Notably, adjusting the collective response this way only improved the accuracy if the respondents were torn on the response: In general, a majority consensus correlates with accurate knowledge.