This paper explores the extent to which ideas developed in The Senses Considered as Perceptual Systems and further refined in The Ecological Approach to Visual Perception (Gibson, 1966; 1979) can be applied to the analysis of perception and action in musical settings. The ecological approach to perception has rarely been applied to music, although some recent work in ecological acoustics, music theory and music psychology has begun to show an interest in direct perception of events and objects. We would argue that despite this pioneering work, Gibson’s most radical and controversial idea, that of the direct perception of affordances (Gibson, 1979), has not been adequately addressed in a musical context. Following an introduction to the theoretical background to affordances and a review of the ways in which previous authors have investigated ecological approaches to auditory perception, we show how both the production and perception of music can fruitfully be analysed using the concept of affordances, and how such an approach neatly integrates seemingly active and passive engagement with music. In addition, we place this ecological approach to music within a broader empirical context, giving examples of music-psychological, ethnomusicological and neuroscientific evidence which complement our more theoretical approach. In conclusion, we argue that the links between the performance, composition and reception are underpinned by the mutuality of perception and action.