Using All Our Senses: the need for a Unified Theoretical Approach to Multi-sensory Information Visualization

There is an opportunity to consider the theory of visualization for each of our senses together. We should have a united theory of visualization that covers all senses and allows users to integrate modalities. We believe there are several important facets to the theory that need to be addressed at all stages of the multi-sensory visualization process. (1) Data enhancement for multi-sensory visualization; (2) perceptual variables for each sense; (3) Transference of the visualization methods and designs from one domain to another. (4) Cross-modal device integration. How multi-sensory devices are built and are used together; (5) Sensory integration and cross-modal interference: how different sensations interfere or reinforce each other. We need a body of research that will help researchers tackle questions such as ‘what are the perceptual variables that are available?’, ‘what are their limitations?’, ‘Do ideas and concepts that work in one modality transfer to another?’ and ‘what are the best design strategies for depicting information in another modality?’.