Commentary: Predictions and the brain: how musical sounds become rewarding
Niels Chr. Hansen
Martin J. Dietz
Peter Vuust- 1Cognitive and Systematic Musicology Laboratory, School of Music, Ohio State University, Columbus, OH, USA
- 2Center for Music in the Brain, Department of Clinical Medicine, Aarhus University and The Royal Academy of Music Aarhus/Aalborg, Aarhus, Denmark
- 3Center for Functionally Integrative Neuroscience, Department of Clinical Medicine, Aarhus University, Aarhus, Denmark
A commentary on
Predictions and the brain: how musical sounds become rewarding
by Salimpoor, V. N., Zald, D. H., Zatorre, R. J., Dagher, A., and McIntosh, A. R. (2015). Trends Cogn. Sci. 19, 86-91. doi: 10.1016/j.tics.2014.12.001
Converging research efforts have proposed that musical sounds become rewarding through predictive processes in the brain's pleasure networks, including dopamine release in the midbrain (Blood and Zatorre, 2001; Gebauer et al., 2012). In this commentary we address the subtle, yet important distinction between two types of “prediction error” that are sometimes conflated in the music neuroscience literature: (i) reward prediction error (RPE) pertaining to (psychological) expectations of how emotionally rewarding a piece of music will be and (ii) prediction error (PE) pertaining to neuronal computation of sensory input relating to the brain's predictions about music itself. Ultimately, “What is the next chord?” (PE) and “How much will I like the next chord?” (RPE) are distinct—potentially orthogonal—questions. While many sources of fundamental pleasure like food, sex, and drugs are readily quantifiable and show a largely monotonic relationship between stimulus amount and pleasure magnitude (until a given saturation point), sources of higher-order pleasures like music cannot be unambiguously quantified (Berridge and Kringelbach, 2008). More music does not in itself imply greater pleasure. Rather, the pleasure potential of music relies on the interplay of prior learning and dynamic changes in stimulus structure over time (Huron, 2006). We propose that predictive coding under the free-energy principle (Friston, 2009)—under which the brain continuously minimizes PE in the interaction with its environment—has the potential to bridge PE and RPE, thus elucidating domain-specific aspects of musical appreciation.
Salimpoor et al. (2015) take noteworthy first steps toward synthesizing the two research literatures on the neurobiology of reward (e.g., their references 2–9) and on musical expectations (e.g., their references 12–42). From a computational perspective, the former relies on reinforcement learning, which sets up computational principles for maximizing reward value, irrespective of music-structural specifics (Schultz, 2013). The latter deals with predictions concerning musical structure and has been modeled using statistical learning and predictive coding (Vuust et al., 2009; Hansen and Pearce, 2014; Vuust and Witek, 2014; Hansen et al., 2016). In predictive coding, PE is neither “positive” nor “negative” per se, but rather strong/weak on a single continuum (Friston and Stephan, 2007). Positive and negative RPE thus seems inconsistent with the mathematical formulation of predictive coding. Friston and colleagues propose that RPE represents mere surface manifestations of more fundamental computations in the brain (Friston et al., 2009). Specifically, rewarding actions are those that minimize the brain's free energy, thus building a stronger and more accurate model of the world. In other words, many types of reinforcement and procedural learning can be reinterpreted as predictive coding and may in fact render the very notion of value redundant (Friston, 2009).
A key claim of Salimpoor et al. (2015) is that “[w]hen listening to previously unheard music, similar-sounding auditory templates may be ‘activated’ to generate expectations of how the new sounds will unfold [i.e., PE]. If the new sounds were better than expected [i.e., RPE], positive PE would result.” Assessing whether music is “structurally-better-than-expected” requires a clear definition of “structurally good.” In music listening, however, expectations more likely pertain to the structure of music (PE) than to its reward value (RPE) (Huron, 2006; Miranda and Ullman, 2007; Hansen and Pearce, 2014; Vuust and Witek, 2014; Hansen et al., 2016). Accordingly, Salimpoor et al.'s notion of valenced PE with respect to structural continuation is problematic because it conflates expectations about experienced pleasure and perceived sounds. Yet, the authors resort to this in their account of the inverted U-shaped relationship between exposure and musical appreciation, claiming that time between hearings increases the leeway for positive PE (Salimpoor et al., 2015, Box 2). In Figure 1 we provide an alternative explanation, without reference to RPE, emphasizing instead how the certainty of the brain's predictions influences the salience of the ensuing PE (Ross and Hansen, 2016) which may in turn affect the level of experienced pleasure. This is thought to be mediated by the sensitivity or gain of neuronal populations in inferior frontal gyrus to feedforward connections from superior temporal gyrus that mediate PE (Dietz et al., 2014).
Figure 1. Predictive coding of musical appreciation with increasing exposure. Understanding the inverted U-curved relationship between exposure and appreciation is a key question in the cognitive neuroscience of music (Salimpoor et al., 2015). Here, it is unadvisable to conflate predictions about musical structure (PE) and its reward value (RPE) (cf. Salimpoor et al., 2015, Box 2). The alternative account presented here is based purely on PE, with no reference to RPE or conflation of the two. Instead, we emphasize contrastive interactions between schematic expectations, pertaining to generalized knowledge about a musical style, and veridical expectations, pertaining to specific knowledge about particular pieces (Huron, 2006). These two types of expectations have distinct neural representations (Miranda and Ullman, 2007). (A) Initially, because unfamiliar music affords a weak predictive model, the gain on error units is set such that prediction error is low in salience (Friston, 2009). With increasing exposure, music is contextualized (e.g., determining key, tempo, meter, instrumentation, duration, form, genre) leading to schematic expectations with higher certainty (Hansen and Pearce, 2014; Hansen et al., 2016). Simultaneously, veridical expectations arise causing potential conflicts with schematic expectations (Huron, 2006). This sharpens the listener's predictive model generating stronger expectations with higher error-unit gain, leading to gradually more salient prediction error (Hansen and Pearce, 2014; Hansen et al., 2016). (B). With increasing levels of exposure (and ultimately over-exposure), the amount of prediction error gradually declines as veridical expectations become increasingly aligned with sensory input and thus are assigned greater relative importance compared to schematic expectations. (C) The combination of increasing certainty-weighting of prediction error (due to gradually higher predictive certainty) and fewer instances of prediction error (due to minimization of free energy) results in an inverted U-shaped trajectory of musical appreciation with increasing levels of exposure. This is consistent with dopamine coding for the precision of prediction error.
Previous studies have found a relationship between reward value and activity in brain structures implicated in positive RPE (Salimpoor et al., 2013). This does, however, not provide causal evidence that musical appreciation is mediated by positive RPE, rather than PE. Moreover, a general focus on RPE circumvents the question of how music evokes pleasure and is assigned reward value in the first place.
So how does dopamine fit into this? Single-cell studies have shown bidirectional coding where changes in dopaminergic activity reflect positive RPE when a reward is greater than expected and negative RPE when it is smaller than expected (Schultz, 2010, 2013). However, empirical evidence indicates that dopamine neurons not only code for expected reward value, but also for the magnitude, timing, probability, and uncertainty of rewards, as well as perceptual salience (Schultz, 2010; Vuust and Kringelbach, 2010). The last possibility, in particular, may relate to PE rather than RPE. Rather than encoding the “prediction error on value,” predictive coding posits that dopamine may encode the “value of prediction error” where value corresponds to precision or incentive salience (Friston and Stephan, 2007; Friston, 2009). For example, one may hypothesize that participants are willing to pay more money for music that they have a strong predictive model for (cf. Salimpoor et al., 2013).
In conclusion, we propose that predictive coding offers a useful framework for understanding the mechanisms that determine when and why music is rewarding. However, it is crucial not to conflate RPE and PE. We regard this as an important distinction that still remains to be adequately studied. To this end, although not the only relevant theory, predictive coding could provide an alternative account of musical reward encapsulated in a general theory of brain function where PE and RPE are treated in a unified manner. In other words, predictive coding of musical structure and its rewarding qualities may be different manifestations of the same underlying computational principles.
Author Contributions
NH and MD: conceived of the initial idea for this commentary. NH: wrote the first draft. NH, MD, and PV: all made substantial contributions to the content, critical revision, and final approval of this work and agree to be held accountable for this.
Funding
Center for Music in the Brain is funded by the Danish National Research Foundation (DNRF117). During part of this work, NH was supported by The Ministry of Culture Denmark and an EliteForsk Travel Grant from The Ministry of Higher Education and Science Denmark. MD is supported by the VELUX Foundation.
Conflict of Interest Statement
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Acknowledgments
The authors would like to acknowledge financial support from the Danish National Research Foundation (PV, NH), The Ministry of Culture Denmark (NH), an EliteForsk Travel Grant from The Ministry of Higher Education and Science Denmark (NH), and the VELUX Foundation (MD).
References
Berridge, K. C., and Kringelbach, M. L. (2008). Affective neuroscience of pleasure: reward in humans and animals. Psychopharmacology 199, 457–480. doi: 10.1007/s00213-008-1099-6
Blood, A. J., and Zatorre, R. J. (2001). Intensely pleasurable responses to music correlate with activity in brain regions implicated in reward and emotion. Proc. Natl. Acad. Sci. U.S.A. 98, 11818–11823. doi: 10.1073/pnas.191355898
Dietz, M. J., Friston, K. J., Mattingley, J. B., Roepstorff, A., and Garrido, M. I. (2014). Effective connectivity reveals right-hemisphere dominance in audiospatial perception: implications for models of spatial neglect. J. Neurosci. 34:5003. doi: 10.1523/jneurosci.3765-13.2014
Friston, K. (2009). The free-energy principle: a rough guide to the brain? Trends Cogn. Sci. 13, 293–301. doi: 10.1016/j.tics.2009.04.005
Friston, K. J., Daunizeau, J., and Kiebel, S. J. (2009). Reinforcement learning or active inference? PLoS ONE 4:e6421. doi: 10.1371/journal.pone.0006421
Friston, K. J., and Stephan, K. E. (2007). Free-energy and the brain. Synthese 159, 417–458. doi: 10.1007/s11229-007-9237-y
Gebauer, L., Kringelbach, M. L., and Vuust, P. (2012). Ever-changing cycles of musical pleasure. Psychomusicol. Music Mind Brain 22, 152–167. doi: 10.1037/a0031126
Hansen, N. C., and Pearce, M. (2014). Predictive uncertainty in auditory sequence processing. Front. Psychol. 5:1052. doi: 10.3389/fpsyg.2014.01052
Hansen, N. C., Vuust, P., and Pearce, M. (2016). “If you have to ask, you'll never know”: effects of specialised stylistic expertise on predictive processing of music. PLoS ONE 11:e0163584. doi: 10.1371/journal.pone.0163584
Huron, D. (2006). Sweet Anticipation: Music and the Psychology of Expectation. Cambridge, MA: MIT Press.
Miranda, R. A., and Ullman, M. T. (2007). Double dissociation between rules and memory in music: an event-related potential study. Neuroimage 38, 331–345. doi: 10.1016/j.neuroimage.2007.07.034
Ross, S., and Hansen, N. C. (2016). Dissociating prediction failure: considerations from music perception. J. Neurosci. 36, 3103–3105. doi: 10.1523/JNEUROSCI.0053-16.2016
Salimpoor, V. N., van den Bosch, I., Kovacevic, N., McIntosh, A. R., Dagher, A., and Zatorre, R. J. (2013). Interactions between the nucleus accumbens and auditory cortices predict music reward value. Science 340, 216–219. doi: 10.1126/science.1231059
Salimpoor, V. N., Zald, D. H., Zatorre, R. J., Dagher, A., and McIntosh, A. R. (2015). Predictions and the brain: how musical sounds become rewarding. Trends Cogn. Sci. 19, 86–91. doi: 10.1016/j.tics.2014.12.001
Schultz, W. (2010). Dopamine signals for reward value and risk: basic and recent data. Behav. Brain Funct. 6:24. doi: 10.1186/1744-9081-6-24
Schultz, W. (2013). Updating dopamine reward signals. Curr. Opin. Neurobiol. 23, 229–238. doi: 10.1016/j.conb.2012.11.012
Vuust, P., and Kringelbach, M. L. (2010). The pleasure of making sense of music. Interdiscipl. Sci. Rev. 35, 166–182. doi: 10.1179/030801810X12723585301192
Vuust, P., Ostergaard, L., Pallesen, K. J., Bailey, C., and Roepstorff, A. (2009). Predictive coding of music-brain responses to rhythmic incongruity. Cortex 45, 80–92. doi: 10.1016/j.cortex.2008.05.014
Keywords: predictive coding, music, reward, pleasure, emotion, dopamine, neuroaesthetics
Citation: Hansen NC, Dietz MJ and Vuust P (2017) Commentary: Predictions and the brain: how musical sounds become rewarding. Front. Hum. Neurosci. 11:168. doi: 10.3389/fnhum.2017.00168
Received: 20 October 2016; Accepted: 21 March 2017;
Published: 05 April 2017.
Edited by:
Arthur M. Jacobs, Freie Universität Berlin, GermanyReviewed by:
Roel M. Willems, Radboud University Nijmegen, NetherlandsDiana Omigie, Max Planck Society (MPG), Germany
Copyright © 2017 Hansen, Dietz and Vuust. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Niels Chr. Hansen, hansen.491@osu.edu