We have just published a pre-print in bioRxiv on the work from my graduate student Raffles Zhu.

In human vision, the retinal input is transformed into internal representations through a series of stages. In earlier stages, the signals from a particular visual field locus are passed in parallel from one visual processing area to the next. The connections at each stage may therefore introduce “error”, where incorrect or convergent projections result in a loss of spatial precision. Psychophysical and physiological studies have implicated spatial scrambling of this sort as a cause of the visual deficits in amblyopia. Several methods to measure scrambling (both in amblyopia and in healthy vision) have been developed in recent decades. In this work, we introduce a new approach. We consider two stages of visual processing where scrambling may occur: either at the input to or the output from the simple cell stage in V1. We refer to these as “subcortical” and “cortical” scrambling respectively. We investigated the impact of these two types of scrambling on a letter identification task. A physiologically-inspired decomposition and resynthesis algorithm was used to generate letter stimuli that simulate scrambling at each of these two stages. To establish a performance benchmark, we trained separate Convolutional Neural Networks (CNNs) to perform the task with each scrambling type. Comparing CNN performance against that of eight humans with normal healthy vision, we found humans exhibited greater resilience to subcortical scrambling compared to cortical scrambling. We further investigated performance by comparing confusion matrices. Compared to a simple template matching model, we found the human strategy to be more consistent with our CNNs. We conclude: i) the human resilience for subcortical scrambling suggests this may be the stage at which a greater degree of scrambling is introduced in the visual hierarchy, and ii) humans employ flexible strategies for identifying scrambled stimuli, more sophisticated than a simple template match to the expected target.