Non-linguistic Processes

Speech categories aren't the only kind of categories we're interesting. In order to categorize musical chords, colors, faces, and common objects the brain must solve very similar problems to speech. It must process a highly variable and context dependent input and idenitify a unique category; it must cope with noise and irregularities; and it must do it fast. Thus, in trying to understand how these general categorization processes work, we've been examining a range of questions in both infants and adults.


One important question concerns whether categories are discrete symbolic units, or are more graded and probabilistic. This has been a major topic of study in our work on speech perception, but it is also relevant to color. Color, of course, enters our eyes as a gradient, continuous measure. However, very quickly our brain is able to determine the discrete label for any given color. Stephanie Huette has been examining this using the mouse-tracking task of Spivey, Grosjean & Knoblich (2005) to determine if color categories (like speech) are continuous in nature, and to examine the way in which people make these sorts of perceptual judgements in the moment.

Speech perception also has a strong temporal component. The perceptual information comes in over time and the system must form expectations and accumulate knowledge as time goes by. Music perception poses a similar problem. One strategy that the speech perception system uses to cope with this is for the system to use contextual knowledge and music by looking at the way that a preceding musical context shapes listeners perceptions of major and minor chords. Our results are striking suggesting that very different processes may be at work in speech and music perception.

Developmentally, infants are capable of learning categories for common objects and animals begin in very early infancy--as early as 4 months! Interestingly, the kinds of categories that they form in the lab is dependent on how similar the objects are. If you shown them a bunch of similar looking dogs, infants will form a category of dogs that excludes cats. If you show them a bunch of dissimilar dogs, their category for dogs may actually include cats (Oakes, Coppage & Dingel, 1997). This mirrors work on statistical learning in speech perception where the distribution of the sounds determines how many categories are formed. One crucial question is how infants figure out what objects are similar to one another. Kristine Kovack are currently studying this, using infant eye-tracking to look specifically at the sort of online comparison process that infants engage in as they consider two animals.

Ultimately, however, our ability to understand infants categorization requires methods that allow us to determine how an infant categorizes a given stimulus. Habituation methods (and variants of it) have given us a lot of answers to this question, but in many ways, this has not been sufficient. Thus, the MACLab has been developing AntiEM, a technique based on anticipatory eye-movements to help us understand this important process.

Non-linguistic Processes

Color Gradiency

Color perception has many of the same problems as speech perception making it an interesting way to test the cross-domain applicability of our theories. However it can be difficult to study using the eye-tracker -- when you flash a color up, subject typically look at that, not the responses. Thus, we have been adapting the mouse-tracking task of Spivey, Knoblich & Grosjean (2003) to study color perception. In this task, we flash up a color for subjects to identify and they click on the corresponding category. However, rather than tracking their eyes while we do this, we track the detailed trajectory of their mouse. Spivey has shown that this can reveal competition dynamics between possible responses. For example, if the stimulus is teal subjects may initially veer towards green but later correct and click on the blue. However, when pure-blue is seen, subjects move directly to blue.


Alumni Collaborators
Stephanie Huette

Musical Expectancy

Just like in language musical passages persist over time and can create strong expectancies. In language, for example, the phrase "happy birth day to..." creates a strong expectancy for the word "you". Likewise, in music the corresponding notes also create an expectancy for what's coming next. Our work in speech on place assimilation and lexical feedback clearly shows these processes at work in language. However music gives us an opportunity to discover if such processes are specific to language or are more general perceptual mechanisms.

In this study, trained musicians here passages like the ones below and are asked to determine if the final chord is major or minor. The rules of music and first set of chords should bias them to hear one or the other. In the first phrase below, the final chord is consistent with your expectancy and sounds quite nice. In the second phrase, the final chord is inconsistent and should be a bit unexpected. The interesting case is what happens for a perfectly ambiguous chord (the third phrase). Our work in speech suggests that you should hear this chord as consistent with context (e.g. a major in this sequence). However, most of the musicians rates this chord as more minor. This suggests that context can form expectations (since the decision was affected by context), but that it might not work in the same way as speech.


Alumni Collaborators
Joel Dennhardt
Andrew Struck-Marcell

Papers
McMurray, B., Dennhardt, J., and Struck-Marcell, A. (submitted) Interactive effects of musical expectancy on chord categorization: Are speech and music different?

Comparison and Animal Categories

In this experiment, we are interested in learning about what features young infants and adults are looking at when they see pairs of animals and their pattern of comparison when they look between two stimuli. Specifically, participants are shown pairs of cats, dogs, and a cat and a dog. While they look at these animals, we are examining what they pay attention to with our eye-tracker. Do they match parts (e.g. a look to the head on the left followed by a look to the head on the right?)? Do they scan one animal for a while before switching? Or are they doing something completely random (we wouldn't put it poast them!)?

This study is examining 4-month-old infants, many of whom may have had a significant amount of experience that could help them in this task (e.g. picture books, having a pet at home, etc). Thus, we are also examining if having experience with a pet at home influences what infants look at and their pattern of comparison. Another factor that may be important is the way in which the pictures are presented--it may be easy for infants to compare animals, for example, if they are both standing and pointing in the same direction. Thus, for half of the pairs both animals in the same position (e.g., both are standing and facing the right) and for the other half, the animals are in different positions (e.g., one sitting and one standing).

By looking at how all of these factors interact with the pattern of eye-movements that the infants make, we hope to discover if infants and adults use different comparison strategies for different types of stimuli, and how their background knowledge might influence this.


Students
Kristine Kovack-Lesh 		Collaborators
Lisa Oakes (UC Davis)

Back to top

The Anticipatory Eye-Movement Paradigm (AntiEM)

The MACLab has been developing a new paradigm for teting infants in two-alternative tasks, the AntiEM paradigm. A complete description of the paradigm can be found in the references section. Current work in this paradigm is assessing a variety of categories, and looking for ways to improve the training procedure.

Post-docs
Shannon Ross-Sheehy 		Collaborators
Richard Aslin (University of Rochester)

Papers

Site maintained by Bob McMurray, Cheyenne Munson, and Joe Toscano.