The overall aim of my research is to understand how the color content of the visual scene is encoded and analyzed within the human visual system. Initially, experimental approaches to color vision were primarily concerned with the very first stages of vision, namely with identifying the visual pigments that absorb light prior to the encoding of the visual scene. Although these first receptoral stages are a prerequisite for vision, they tell us little about how color is encoded, since the sensation of color is constructed post-receptorally by the responses of the color opponent neurons in the retina, visual pathways and the visual cortex of the brain.
Several decades ago a new approach emerged in the investigation
of color vision based on the use of visual stimuli that uniquely
the responses of the color sensitvie pathways in the visual system, so
separating the brain's response to color from its response to black
white. Such stimuli (termed isoluminant) have become an important
tool for the investigation of the neural basis of color vision. I have
used these methods extensively. My general approach is to develop and
models and ideas of how the visual system encodes colour information
the behavioral testing of human vision (psychophysicics) and/or fMRI
methods. These can
also be linked with available primate anatomical and physiological data.
Below are brief descriptions of some of the key issues that I have addressed in color vision research:
1. fMRI investigations
of color processing in the human brain.
2. Color and motion processing - does the dorsal pathway contribute to color vision?
3. Color and form perception - defining the role of the ventral pathway in color vision.
4. Color vision across the visual field: is cone opponency lost in peripheral vision?
5. How do the cones combine into the opponent processes?
6. The cognitive development of color vision in young children.
Example publications for normal vision:
Mullen, K.T. Thompson, B. & Hess, R.F. Responses of the human visual cortex and LGN to achromatic and chromatic temporal modulation: an fMRI study. Journal of Vision, 10(13): 13, 1–19, 2010. http://www.journalofvision.org/content/10/13/13.full.pdf
Mullen, K.T., Dumoulin, S.O. & Hess, R.F. Color responses of the human lateral geniculate nucleus: Selective amplification of S-cone signals between the lateral geniculate nucleus and primary visual cortex measured with high-field fMRI. European Journal of Neuroscience, 28, 1911-1923, 2008. (PDF)
Mullen, K.T., Dumoulin, S.O., McMahon, K.L., de Zubicarary, G.I. & Hess, R.F. Sensitivity of human retinotopic visual cortex to red-green, blue-yellow and achromatic contrast. European Journal of Neuroscience, 25, 491-502, 2007. (PDF)
Example publications in amblyopia:
Hess, R.F., Thompson, B., Gole, G. & Mullen, K.T. The amblyopic deficit and its relationship to geniculo-cortical processing streams. Journal of Neurophysiology, 104(1), pp475-483, 2010. http://jn.physiology.org/cgi/reprint/104/1/475
Hess, R.F., Thompson, B., Gole, G. & Mullen, K.T. Deficient
responses from lateral geniculate nucleus in humans with amblyopia. European Journal of Neuroscience, 29,
1064-1070, 2009. http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2695153/
Normal human color vision is known to be
deficient in the perception of
motion. This has lead to a 'text-book' view of the human cortex in
which there are
distinct 'color' and 'motion' streams: a color-blind dorsal
stream that encodes motion information, and a motion-blind ventral
stream that conveys
color information. We are presently investigating using both
psychophysical and fMRI methods the extent to which these streams are
really separate for the processing of color and motion. This involves
making a distinction between two types of motion: first order (linear)
and higher order (nonlinear). We find that color is defective in
the processing of linear motion, but retains somecapacity to process
non linear motion. One of the outstanding quesitons is where first and
higher order motion are processed in the brain, and why only one of
these types should be blind to color contrast.
Garcia-Suarez, L. & Mullen, K.T. Global motion processing in human color vision: a deficit for second order stimuli. Journal of Vision, 10(14): 20, 1–11, 2010. Free access: http://www.journalofvision.org/content/10/14/20.full.pdf
Michna, M. L. & Mullen, K.T. The contribution of color to global motion. Journal of Vision, 8 (issue 5 article 10), 1-12, 2008. Free acess: JOV: http://journalofvision.org/8/5/10/Michna-2008-jov-8-5-10.pdf
Michna, M. L. & Mullen, K.T. Chromatic mechanisms mediating motion for S-cone isolating stimuli. Vision Research, 47, 1042-1054, 2007. (PDF).
Mullen, K.T., Yoshizawa, T., & Baker, C.L. Luminance signals mediate the motion of red-green isoluminant gratings: the role of “temporal chromatic aberration”. Vision Research, 43, 1235-1247, 2003. (PDF)
Yoshizawa, T., Mullen, K.T. & Baker, C.L. Absence of a chromatic linear motion mechanism in human vision. Vision Research, 40, 1993-2010, 2000. (PDF)
For several decades color vision was considered very poor at seeing form and shape, and even called 'form blind'. This view was supported by phenomena such as the lowpass and low acuity nature of the color contrst sensitivity function (e.g. Mullen, 1985), the blurred appearance of chromatic borders, and by the loss of 3D shape perception under some conditions. We (McIlhagga & Mullen, 1997) dubbed this view of color vision the "coloring book model" because it describes a subordinate role for color vision in the extraction of shape and form: color vision simply fills-in the contours and boundaries of objects that are primarily defined by luminance contrast (black & white). Subsequently, we have shown that this cannot be the main way the brain uses color. Instead my results suggest that both color and luminance vision contribute to the primary stages of spatial processing in a very similar manner, with relatively little deficiency found for color vision. These results indicate suggest that color and luminance contrast may feed common form processing mechanisms at some stage in the visual pathway. So far, we find little evidence for a segregated pathway for color vision with poor form processing.
i. The processsing of orientation in color vision: The encoding of orientation is a key part of spatial processing. Studies in my lab have revealed the role that color contrast plays in encoding orientation. We find color has only a mild deficiency of orientation processing. This is interesting as there is a populaton of chromatic neurons in the cortex that show very little orientation tuning. Our results therefore suggest that these are not part of a neural pathway encoding form in color vision but instead must have another role.
Beaudot, W.H.A. & Mullen, K.T. Orientation selectivity in luminance and color vision assessed using 2-d bandpass filtered spatial noise. Vision Research, 45, 687-696, 2005. (PDF)
Beaudot, W.H.A. & Mullen, K.T. Orientation discrimination in
human vision: psychophysics and modeling. Vision Research, 46, 26-46,
ii. The role of color in contour perception: The detection of contours and object borders is fundamental to the extraction of the salient features of the visual scene. We know that the perception of a border or contour depends on the integration or grouping of multiple outputs of many local neurons (detectors) in the visual system. I have used a contour integration task to compared the performance of the two color systems (red-green and blue-yellow) and the luminance (black & white) system in this role. A black & white stimulus is illustrated below.
From Mullen, Beaudot & McIlhagga, 2000 (PDF). The wiggly path shown on the right is embedded in the figure shown on the left.
The left pannel contains a winding 'contour' made up of 10 aligned elements (Gabors) embedded in a background of randomly oriented Gabors. The 'contour' alone is shown in the right pannel. Accurate detection of the contour relies on the integration of these multiple small oriented elements (Gabors) and requires the linking of the orientations of the contour elements across space. Our results reveal striking similarities in the way the color and luminance systems perform contour integration, in terms of their overall performances, efficiency, internal orientation noise, contrast dependence and dependence on curvature (McIlhagga & Mullen, 1996; Mullen, Beaudot & McIlhagga, 2000). We suggest that this indicates that color and luminance vision use a common neural process in the early cortical stages of form processing.
McIlhagga, W.H. & Mullen, K.T. Contour integration with color and luminance contrast. Vision Research, 36, 1265-1279, 1996. (PDF)
Mullen, K.T., Beaudot, W.H.A. & McIlhagga, W.H. Contour integration in color vision: a common process for the blue-yellow, red-green and luminance mechanisms? Vision Research, 40, 639-655, 2000. (PDF)
Beaudot, W.H.A. & Mullen, K.T. Processing time of contour integration: the role of color, contrast and curvature. Perception, 30, 833-853, 2001. (PDF)
Hess, R.F., Beaudot, W.H.A. & Mullen, K.T. Dynamics of contour integration. Vision Research, 41, 1023-1037, 2001. (PDF)
Beaudot, W.H.A. & Mullen, K.T. How long-range is contour
in human color vision? Visual Neuroscience, 20, 51-64,
iii. Global shape processing: We have shown that color has a deficiency in a task used to test global shape perception (Mullen & Beaudot, 2002). Stimuli are concentric ring patterns termed radial frequency patterns, as shown on the right for red-green stimuli. The task is to discriminate of the shape of the pattern from circular (in this case a diamond shape from circular). For the top pair of chromatic stimuli, discrimination of the shape from the circle is relatively easy, but is harder for the lower pair. The task is made progressively harder to determine the shape discrimination threshold. We find that red-green and particularly blue-yellow color vision shows a significant deficit on this task in comparison to achromatic (black & white) vision. This is an interesting result as it may indicate that the specilialzed shape processing areas of the brain are better at using black & white than color contrast. In further work we have addressed ideas on exactly how the shape discrimination is performed in order to understand what aspect is more poorly processes in color vision.
Mullen, K.T. & Beaudot, W.H. The contribution of color vision to a global shape discrimination task. Vision Research, 42, 565-575, 2002. (PDF)
Mullen, K.T., Beaudot, W.H.A. & Ivanov, I.V. Evidence that global processing does not limit thresholds for RF shape discrimination. Journal of Vision, 11(3): 3, pp1-21, 2011. http://www.journalofvision.org/content/11/3/6.full.pdf
Mullen, K.T. & Losada, M.A. Evidence for separate pathways for color and luminance detection mechanisms. Journal of the Optical Society of America A., 11, 3136-3151, 1994. (PDF)
Losada, M.A. & Mullen, K.T. Color and luminance spatial tuning estimated by noise masking in the absence of off-frequency looking. Journal of the Optical Society of America A., 12, 250-260, 1995. (PDF)Mullen, K.T. & Losada, M.A. The spatial tuning of color and luminance peripheral vision measured with notch filtered noise masking. Vision Research, 39, 721-731, 1999. (PDF)
There has been much debate about how red-green opponency is 'wired' into the visual system. Some groups proposed that red-green opponency arises as a by-product of the very small receptive fields found in the retinal P cells of central vision (see Mullen & Kingdom, 1996). The so-called 'hit and miss' hypothesis suggested that cone opponency can arise by chance in the small receptive fields of midget ganglion cells because, when cone numbers are low, proportions of L and M cones are likely to be distributed differentially to the centre and surround of a receptive field by chance. This implies that red-green cone opponency and fine visual acuity evolved in tandem in Old World primates, as midget retinal ganglion cells mediate both. Other groups suggested that, like blue-yellow opponency, RG opponency is selectively wired into large retinal neurons, and so is not dependent on small receptive fields and can extend into the peripheral visual field. Mullen & Kingdom (1996) developed a quantitative model to define the loss in cone opponency expected as the cone numbers feeding a receptive field centre and surround increase. We find that the rapid loss in red-green cone opponency away from the fovea is consistent with the predictions of a random selection of cones feeding centre and surrounds of the midget ganglion cell receptive fields. This model of red-green cone opponency has also recently received direct physiological support in Macaque retina.
Sakurai, M. and Mullen, K.T. Cone weights for the two cone opponent systems in peripheral vision and asymmetries of cone contrast sensitivity. Vision Research, 46, 4346-4354, 2006. (PDF)
Mullen, K.T., Sakurai, M & Chu, W. Does L/M cone opponency
disappear in human periphery? Perception, 34, 951-959, 2005. (PDF)
Mullen, K.T. & Kingdom F.A. Differential distribution of red-green and blue-yellow cone opponency across the visual field. Visual Neuroscience, 19, 1-10, 2002. (PDF)
Mullen, K.T. & Kingdom F.A. Losses in peripheral color sensitivity predicted from 'hit & miss' post-receptoral cone connections. Vision Research, 36, 1995-2000, 1996. (PDF)
Anderson S.A., Mullen, K.T. & Hess, R.F. Human peripheral spatial resolution for achromatic and chromatic stimuli: Limits imposed by optical and retinal factors. Journal of Physiology, 442, pp47-64, 1991. (http://www.ncbi.nlm.nih.gov/pubmed/1798037?dopt=Citation)
Mullen, K.T. Colour vision as a post receptoral specialization
of the central visual field. Vision Research, 31,
I have also established the independence of the two color and luminacne mechanisms at detection threshold using spatial summation (Mullen et al. 1997, Mullen & Sankeralli, 1999) and noise masking methods.
Sankeralli, M.J. & Mullen, K.T. Estimation of the L-, M- and S-cone weights of the post-receptoral detection mechanisms. Journal of the Optical Society of America A, 13, 906-915, 1996. (PDF)
Sankeralli, M.J. & Mullen, K.T. Postreceptoral chromatic detection mechanisms revealed by noise masking in three-dimensional cone contrast space. Journal of the Optical Society of America A. 14, 2633-2646, 1997. (PDF)
Mullen, K.T., Cropper, S.J. & Losada, M.A. Absence of linear subthreshold summation between red-green and luminance mechanisms over a wide range of spatiotemporal conditions. Vision Research, 37, 1167-1175, 1997. (PDF)
Mullen, K.T. & Sankeralli, M.J. Evidence for the stochastic independence of the blue-yellow, red-green and luminance detection mechanisms revealed by subthreshold summation. Vision Research, 39, 733-745, 1999. (PDF)
Sankeralli, M.J. & Mullen, K.T. A ratio model for suprathreshold hue increment detection. Journal of the Optical Society of America A., 16, 2625-2637, 1999. (PDF)
Sankeralli, M.J. & Mullen, K.T. Assumptions concerning orthogonality in threshold-scaled versus cone-contrast colour spaces. Vision Research, 41, 53-55, 2001. (PDF)
Sankeralli, M.J. & Mullen, K.T. Bipolar or rectified chromatic detection mechanisms? Visual Neuroscience, 18, 127-135, 2001. (PDF)
Sankeralli, M.J., Mullen, K.T. & Hine, T.J. Ratio model serves suprathreshold chromatic-luminance discrimination. Journal of the Optical Society of America A 18, 425-435, 2002. (PDF)
I use color vision as a model for investigating how we establish internal or conceptual representations of perceptual properties. The sense of color involves not only the ability to perceive different colors but also the ability to recognize and identify colors. For example, consider being asked to find a red object in a room. This requires you to be able to see red but also to possess an internal concept of what 'red' is. If either one of these is lacking (percept or concept) the task is impossible. My interest in this area arose after I investigated a color agnosic, who could see colors perfectly but lacked the ability to identify them (Woodward et al. 1999). His color naming and color comprehension (pointing to a named color) were destroyed yet he could perceive color differences normally, had normal color contrast sensitivity, and could generate color words. This demonstrates a fundamental distinction between perceptual versus conceptual processing in the brain.
Young children pass through a stage in which they have difficulty establishing color concepts, as the following quote from Charles Darwin in 1877 about his own 4 children shows: "soon after they had come to the age when they knew the names of all common objects, I was startled by observing that they seemed quite incapable of affixing the right names to the colors in colored engravings, although I tried repeatedly to teach them." (from Bornstein, 1985).
The older literature (up to the 1980s) claims that reliable color naming develops surprisingly late at 4-7 years. Yet it is explicitly clear that this does not reflect a deficiency in color perception since infants can distinguish colors in the first few months of life, and color and luminance contrast sensitivity develop in parallel. Thus the delay lies in the development of color concepts, not perception, that underpin our ability to categorize and identify colors. In this sense, young children bear a striking resemblance to the adult color agnosic described above. My research supports this view by showing that children have accurate color discrimination before they can reliably comprehend color terms. My research has explored two key questions:
1) Is color cognition selectively delayed in young children, or there in fact a more general delay in the conceptual development of all comparable visual attributes? Note that Darwin was comparing the learning of color words, which describe objects (ie. perceptual adjectives), to words for 'common objects' (ie. nouns). In a series of controlled psychophysical experiments on 47 children (2-6 years) we compared the development (comprehension and naming) of 4 visual attributes: form, size, speed, and color, all equated for visual saliency (Pitchford & Mullen, 2001). Results indicate no differential delays in the development of the comprehension and naming of these four visual attributes, suggesting that the delay for color so common reported in the literature may in fact may reflect a general delay in the visual cognition of object attributes. This new finding contradicts the historical contention that color is a special case in visual cognitive development and raises new and important questions for investigation.
2) What limits the development of color concepts? In a classic study Berlin & Kay (1969) made three predictions that radically altered the way we think about color processing. They postulated that there are 11 physiologically based basic perceptual color categories (red, green, blue, yellow, black, white, pink, purple, orange, brown and grey). They also claimed the evolutionary order by which color terms appear in world languages mirrors the salience of the conceptual representation of each color, and they predicted that children acquire color terms in the same order. While there is good support for the existence of 11 basic color categories, their other two predictions remain controversial. In an extensive study we addressed these important and unresolved issues in the development of color cognition (Pitchford & Mullen, 2002, Pitchford & Mullen, 2003). We find that color names develop earlier than previously reported. Furthermore, we show that for most of the 11 colors there is no specific order for acquiring color names; nine of the 11 color names develop within a 3-month window with no predictable order. Hence we find no evidence that the 'primary' colors develop first, or in the order predicted by Berlin & Kay (1969). Curiously, we find a clear second stage of development with reliable naming and comprehension of brown and grey developing at least 6 months later than the other. We do not believe this developmental order is driven by visual saliency as highly salient colors like red appear no earlier than less salient ones like blue or green.
Pitchford, N.J. & Mullen, K.T. The role of perception,
language and preference in the developmental acquisition of basic
colour terms. Journal of Experimental Child Psychology, 90,
275-302, 2005. (PDF)
Pitchford, N.J. & Mullen, K.T. The development of conceptual colour categories in pre-school children: influence of perceptual categorization. Visual Cognition, 10, 51-77, 2003. (PDF)
Pitchford, N.J. & Mullen, K.T. The developmental order of color term acquisition in young children. Perception, 31, 1349-1370, 2002. (PDF)
Pitchford, N.J. & Mullen, K.T. Conceptualization of perceptual attributes: A special case for color? Journal of Experimental Child Psychology, 80, 289-314, 2001. (PDF)