All higher organisms are able to integrate information from multiple sensory modalities and use this information to select and guide movements. In order to do this, the central nervous system (CNS) must solve two problems: (1) Converting information from distinct sensory representations into a common coordinate system, and (2) integrating this information in a sensible way. This dissertation proposes a computational framework, based on statistics and information theory, to study these two problems. The framework suggests explicit models for both the coordinate transformation and integration problems, which are tested through human psychophysics.
The experiments in Chapter 2 suggest that: (1) Spatial information from the visual and auditory systems is integrated so as to minimize the variance in localization. (2) When the relation between visual and auditory space is artificially remapped, the spatial pattern of auditory adaptation can be predicted from its localization variance. These studies suggest that multisensory integration and intersensory adaptation are closely related through the principle of minimizing localization variance. This principle is used to model sensorimotor integration of proprioceptive and motor signals during arm movements (Chapter 3). The temporal propagation of errors in estimating the hand's state is captured by the model, providing support for the existence of an internal model in the CNS that simulates the dynamic behavior of the arm.
The coordinate transformation problem is examined in the visuomotor system, which mediates reaching to visually-perceived objects (Chapter 4). The pattern of changes induced by a local remapping of this transformation suggests a representation based on units with large functional receptive fields. Finally, the problem of converting information from disparate sensory representations into a common coordinate system is addressed computationally (Chapter 5). An unsupervised learning algorithm is proposed based on the principle of maximizing mutual information between two topographic maps. What results is an algorithm which develops multiple, mutually-aligned topographic maps based purely on correlations between the inputs to the different sensory modalities.
171 pages, 1.0 Mb: postscript.