Close collaboration with other scientific fields is an important goal for the visualization community. Yet engaging in a scientific collaboration can be challenging. The physical sciences, namely astronomy, chemistry, earth sciences and physics, exhibit an extensive range of research directions, providing exciting challenges for visualization scientists and creating ample possibilities for collaboration. We present the first survey of its kind that provides a comprehensive view of existing work on visualization for the physical sciences. We introduce novel classification schemes based on application area, data dimensionality and main challenge addressed, and apply these classifications to each contribution from the literature. Our survey helps in understanding the status of current research and serves as a useful starting point for those interested in visualization for the physical sciences.