2/26/2012

Knowing about Knowledge: thoughts on the "performance-competence distinction"

One of the foundational concepts in developmental psychology is the "performance-competence distinction," the difference between people's (low) performance in an experiment designed to test their understanding of a concept and their (presumably higher) understanding of the concept.  The basic idea is that it is very difficult to test just the ability of interest--say, if-then reasoning, or understanding that other people can have beliefs different than one's own--without tapping other abilities that vary among individuals, such as attention, working memory, or prior knowledge.  An example: people get more if-then reasoning problems right when their content is neither obviously true nor obviously false, like (a), than when they are factually incorrect, like (b), because people's real-world knowledge interferes with their reasoning.

(a) If the car hits the wall, then I like turtles.
(b) If the car hits the wall, then it speeds up.

All this makes intuitive sense, although in practice, it is difficult to design an experiment that is not in principle subject to performance errors.  In fact, I am currently running an experiment based on the performance-competence concept, testing whether a common finding on children's understanding of number lines disappears once the task has been simplified.  So I was surprised to discover that not all psychologists think the performance-competence distinction is a valid one.

Let's put psychological theory aside for a moment and think about the possible sorts of knowledge one might have.  You've probably tried to explain something (we'll call it A) to someone and found that you knew what you wanted to say, but somehow you could not find the right words to articulate it.  You knew A, you knew that you knew A, but you could not verbalize A.  There are also situations where you know something but do not know you know it until later.  I suspect a lot of insights during therapy consist of this knowledge ("oh, so that's why I was doing that!").  Children who are just about to understand conservation of matter--the fact that stuff stays the same quantity when you pour it in a new container even if it changes its shape--will say verbally that the amount of stuff changes because it looks taller in the new container.  However, their gestures indicate that the amount of stuff stays the same.  Thus, their gestures suggest that on some level, they understand the same amount of matter is present, but they do not seem to know they know it.  Finally, you may have had to solve math problems where you knew which procedure to use, you applied it correctly, but you made a subtraction error and got the wrong answer.  Thus, one can know how to do something but still fail to do it correctly, particularly when tired, stressed, or otherwise not at one's best.

So, in theory, knowledge is more than a binary presence or absence; it is gradated, perhaps as follows:

1) You know A, you know that you know A, and you can verbalize A, and you can use the knowledge to correctly perform the task.
2) You know A, you know that you know A, and you can verbalize A, but something about the task prevents you from using your knowledge. (The math problem example).
3) You know A, you know that you know A, but you can't verbalize A. (The "I can't explain this" example).
4) You know A, you don't know that you know A, and you can't verbalize A. (The kids' gestures example).
5) You don't know A.

If you ask someone to give you a verbal answer, you can distinguish between (1) and the other options, but not between (2), (3), and (4). 

Experiments based on the performance-competence distinction tend to simplify experiments in three ways:
(a) by reducing the working memory and attention required to respond;
(b) by getting rid of the demand for a verbal answer, e.g. by telling the participant to point; or
(c) by not explicitly asking for a response at all, instead measuring patterns of neural activity or behavior like eye movements, of which the participant is not aware.

Reducing factors like working memory and attention might eliminate situations like my math problem example, and so would distinguish between (1) and (2).  However, it does not reveal whether someone unable to verbalize the answer really lacks the knowledge entirely.

Allowing the participant to point or give some other deliberate nonverbal answer eliminates this problem, allowing the experimenter to distinguish (1), (2), and (3), but does not say anything about participants who do not know whether they know.

Measuring brain activity or unconscious responses allows researchers to test whether people understand the concept at any level, however incomplete, distinguishing (4) and (5).

So, in my opinion, experiments that assume a performance-competence distinction are extremely valuable because they can sort out participants' levels of knowledge.  The problem comes when researchers attempt to interpret these studies.  There are a lot of "superbaby" studies claiming that babies have incredibly sophisticated proto-theories about how objects move, how other people's minds work, and how to count.  The result, at least in the if-then reasoning literature, is a mix of studies claiming that preschoolers are incredibly accurate while adults are incompetent! 

The problem is that these researchers interpret babies as either knowing something or not knowing it.  Can babies understand that other people have intentions and false beliefs?  Can they perform addition with the numbers 1-4?  Can elementary school children reason accurately?  They do have partial knowledge that can be elicited under very specific conditions, as in situations 2-4.  Because it seems perverse to say that children with partial knowledge have no understanding of the concept, researchers end up overstating what children can do.  Other researchers realize these studies are overstated, but respond by arguing that children's partial knowledge does not count as knowledge at all.  If you read the developmental literature, you would be forgiven for thinking either that children are superhuman or completely blank slates.


Of course, children are neither superhuman nor utterly blank slates.  We are programmed by the genetically-inherited structure of our brains and by the fetal environment to learn about a wide variety of domains--the physics of motion, other people's minds, and types of animals, to name a few.  For example, all babies are prebiased to learn a language, but it takes months for their ears to become attuned to the sounds of the specific language(s) spoken around them.  To say children are all-knowing superbabies seems unrealistic, but to deny the knowledge they do have does them a disservice.  Perhaps instead of asking whether a child exhibits performance or competence, we should ask about their degree of knowledge.

6 comments:

  1. Part of the problem, for me, is that this all assumes there that there is some hidden competence to be found. For me there is no distinction - competence just is performance; to think otherwise is dualism. I get this from knowing Esther Thelen at IU; she and Linda Smith have done most of the seminal work looking at how infant behaviour emerges from task dynamics rather than simply reflecting the presence or absence of some hidden cognitive competence.

    The idea is this: you have a complex system, which is composed of numerous elements placed in some relationship to one another. These operate as a system to perform some task. Break any one of these and if the system then fails to do the task, in a very real sense the system doesn't have competence in that task.

    'Degree of knowledge' is closer to what I mean; but I would phrase it in terms of stability in the face of perturbation, etc - dynamical systems language.

    ReplyDelete
    Replies
    1. True, the null hypothesis is that there is no hidden competence, and the burden of proof is on researchers who want to demonstrate it. That's one of the reasons that motion trajectory study you talked about was so good. (I have no ideological commitment to a lack of mental representations, but I do to good reasoning and study design!). :)

      Esther Thelen and Linda Smith's work sound interesting--I'll have to look them up. Thanks!

      I guess I don't understand why it necessarily implies dualism? Surely one can conceptualize increasingly complex cognitive abilities as increasingly complex and flexibly generalizable routines for behavior and/or sensory filtering? I'm drawing on the John Anderson concept that one can think of a lot of abstract thought, such as doing a math problem, as procedural routines automatized through learning. Arguably you don't fully "know" something if you have declarative knowledge about it but lack procedural knowledge. :)

      Delete
    2. It's dualism if you assume there is a mind that contains some knowledge, and a body that must express that knowledge, and that the core competence is the former. A more systems approach requires it all.

      Thelen & Smith's stuff is all excellent. You name it, they've been in it doing great science. Esther was primarily motor development, Linda is more language and cognition. Their book (http://mitpress.mit.edu/catalog/item/default.asp?ttype=2&tid=6978) is a must read on dynamical systems in developmental psychology.

      Delete
  2. Just to further complicate matters, you can think you don't know something when you actually do! See eg blindsight, implicit face recognition in some cases of prosopagnosia...

    ReplyDelete
    Replies
    1. True, those are some great examples! That would fit into #4 in my list, yes?

      Delete
    2. Oh. Yep, misread you there! Although there's possibly a slight difference between not knowing that you know something versus actively thinking that you don't know something. not sure if that's important.

      If you haven't read it already, you should take a look at Annette Karmiloff-Smith's 1992 book, Beyond Modularity.

      Delete