2/26/2012

Knowing about Knowledge: thoughts on the "performance-competence distinction"

One of the foundational concepts in developmental psychology is the "performance-competence distinction," the difference between people's (low) performance in an experiment designed to test their understanding of a concept and their (presumably higher) understanding of the concept.  The basic idea is that it is very difficult to test just the ability of interest--say, if-then reasoning, or understanding that other people can have beliefs different than one's own--without tapping other abilities that vary among individuals, such as attention, working memory, or prior knowledge.  An example: people get more if-then reasoning problems right when their content is neither obviously true nor obviously false, like (a), than when they are factually incorrect, like (b), because people's real-world knowledge interferes with their reasoning.

(a) If the car hits the wall, then I like turtles.
(b) If the car hits the wall, then it speeds up.

All this makes intuitive sense, although in practice, it is difficult to design an experiment that is not in principle subject to performance errors.  In fact, I am currently running an experiment based on the performance-competence concept, testing whether a common finding on children's understanding of number lines disappears once the task has been simplified.  So I was surprised to discover that not all psychologists think the performance-competence distinction is a valid one.

Let's put psychological theory aside for a moment and think about the possible sorts of knowledge one might have.  You've probably tried to explain something (we'll call it A) to someone and found that you knew what you wanted to say, but somehow you could not find the right words to articulate it.  You knew A, you knew that you knew A, but you could not verbalize A.  There are also situations where you know something but do not know you know it until later.  I suspect a lot of insights during therapy consist of this knowledge ("oh, so that's why I was doing that!").  Children who are just about to understand conservation of matter--the fact that stuff stays the same quantity when you pour it in a new container even if it changes its shape--will say verbally that the amount of stuff changes because it looks taller in the new container.  However, their gestures indicate that the amount of stuff stays the same.  Thus, their gestures suggest that on some level, they understand the same amount of matter is present, but they do not seem to know they know it.  Finally, you may have had to solve math problems where you knew which procedure to use, you applied it correctly, but you made a subtraction error and got the wrong answer.  Thus, one can know how to do something but still fail to do it correctly, particularly when tired, stressed, or otherwise not at one's best.

So, in theory, knowledge is more than a binary presence or absence; it is gradated, perhaps as follows:

1) You know A, you know that you know A, and you can verbalize A, and you can use the knowledge to correctly perform the task.
2) You know A, you know that you know A, and you can verbalize A, but something about the task prevents you from using your knowledge. (The math problem example).
3) You know A, you know that you know A, but you can't verbalize A. (The "I can't explain this" example).
4) You know A, you don't know that you know A, and you can't verbalize A. (The kids' gestures example).
5) You don't know A.

If you ask someone to give you a verbal answer, you can distinguish between (1) and the other options, but not between (2), (3), and (4). 

Experiments based on the performance-competence distinction tend to simplify experiments in three ways:
(a) by reducing the working memory and attention required to respond;
(b) by getting rid of the demand for a verbal answer, e.g. by telling the participant to point; or
(c) by not explicitly asking for a response at all, instead measuring patterns of neural activity or behavior like eye movements, of which the participant is not aware.

Reducing factors like working memory and attention might eliminate situations like my math problem example, and so would distinguish between (1) and (2).  However, it does not reveal whether someone unable to verbalize the answer really lacks the knowledge entirely.

Allowing the participant to point or give some other deliberate nonverbal answer eliminates this problem, allowing the experimenter to distinguish (1), (2), and (3), but does not say anything about participants who do not know whether they know.

Measuring brain activity or unconscious responses allows researchers to test whether people understand the concept at any level, however incomplete, distinguishing (4) and (5).

So, in my opinion, experiments that assume a performance-competence distinction are extremely valuable because they can sort out participants' levels of knowledge.  The problem comes when researchers attempt to interpret these studies.  There are a lot of "superbaby" studies claiming that babies have incredibly sophisticated proto-theories about how objects move, how other people's minds work, and how to count.  The result, at least in the if-then reasoning literature, is a mix of studies claiming that preschoolers are incredibly accurate while adults are incompetent! 

The problem is that these researchers interpret babies as either knowing something or not knowing it.  Can babies understand that other people have intentions and false beliefs?  Can they perform addition with the numbers 1-4?  Can elementary school children reason accurately?  They do have partial knowledge that can be elicited under very specific conditions, as in situations 2-4.  Because it seems perverse to say that children with partial knowledge have no understanding of the concept, researchers end up overstating what children can do.  Other researchers realize these studies are overstated, but respond by arguing that children's partial knowledge does not count as knowledge at all.  If you read the developmental literature, you would be forgiven for thinking either that children are superhuman or completely blank slates.


Of course, children are neither superhuman nor utterly blank slates.  We are programmed by the genetically-inherited structure of our brains and by the fetal environment to learn about a wide variety of domains--the physics of motion, other people's minds, and types of animals, to name a few.  For example, all babies are prebiased to learn a language, but it takes months for their ears to become attuned to the sounds of the specific language(s) spoken around them.  To say children are all-knowing superbabies seems unrealistic, but to deny the knowledge they do have does them a disservice.  Perhaps instead of asking whether a child exhibits performance or competence, we should ask about their degree of knowledge.