New Ideas in Hearing

Paris, 12 and 13 May 2006

INVITED SPEAKERS: Bill Yost - Dan Ellis - David McAlpine - David Verstraeten - Enrique Lopez-Poveda - Jan Schnupp - Jonathan Fritz - Jonathan Z Simon - Nicol Harper - Philip Joris - Rhodri Cusack - Roy Patterson - Tim Gollisch - Wolfgang Maass
INVITED DISCUSSANTS: Andre Rupp - Bernhard Englitz - Chris Darwin - Chloé Huetz - Chris Sumner - Christian Lorenzi - David Philipona - Emmanuel Dupoux - Fabien Gouyon - Hirokazu Kameoka - Ian M Winter - Jean-Marc Edeline - Jean-Sylvain Liénard - John Culling - Jonathan Laudanski - Jonathan Le Roux - Katrin Krumbholz - Laurent Demany - Lucas Parra - Madhusudana Shashanka - Makio Kashino - Marcel van der Heijden - Maria Chait - Mounya Elhilali - Felix Schmaeschke - Peter Cariani - Peter Heil - Pierre Divenyi - Ray Meddis - Sascha Korl - Shigeki Sagayama - Sidney Frankel - Simon Thorpe - Sophie Deneve - Stefan Launer - Sue Denham - Torsten Dau - Toshio Irino - Volker Hohmann -

Friday 12:		Saturday 13:
9:30 INTRODUCTION 9:50 (50') Wolfgang Maass / Mounya Elhilali 10:40 (30') COFFEE 11:10 (50') David Verstraeten / Volker Hohmann 12:00 (50') Dan Ellis / Chris Darwin 12:50 (80') LUNCH 14:10 (50') Jan Schnupp / Peter Heil 15:00 (50') Enrique Lopez-Poveda / Peter Cariani 15:50 (30') COFFEE 16:20 (50') Tim Gollisch / Sophie Deneve 17:10 (50') Philip Joris / John Culling		9:00 (50') Jonathan Simon / Katrin Krumholz 9:50 (50') Jonathan Fritz / Laurent Demany 10:40 (30') COFFEE 11:10 (50') Nicol Harper / Lucas Parra 12:00 (50') David McAlpine / Ray Meddis 12:50 (80') LUNCH 14:10 (50') Roy Patterson / Torsten Dau 15:00 (50') Rhodri Cusack / Sue Denham 15:50 (20') COFFEE 16:10 (50') Bill Yost / Makio Kashino 17:00 END

• Wolfgang Maass
  Online computing in dynamical systems as a possible paradigm for auditory processing [PDF]
• David Verstraeten
  Speech recognition using reservoir computing
  Liquid State Machines (LSMs) by W. Maass and Echo State Networks (ESNs) by H. Jaeger are two novel and similar computational frameworks, that both use a recurrent neural network as a reservoir into which the inputs are fed and which itself is left untrained. Instead, a linear classifier is applied to the dynamical state of the reservoir - this classifier can be trained quickly and with a guaranteed optimal solution for a certain error metric. Here, we present the work we have done on the application of both instances of reservoir computing - LSMs and ESNs - to the task of recognizing isolated spoken digits by multiple speakers. Our approach differs quite substantially from the traditional setup used in automated speech recognition, i.e. a mel-frequency cepstrum frontend followed by a Hidden Markov Model-based classifier. We use a biologically realistic model of the human cochlea (Lyon's passive cochlea) to preprocess the speech, which is then used as input to either a LSM or an ESN. With this novel setup, we attain surprisingly good results and even outperform state-of-the-art HMM-based recognizers, using rather small reservoirs that require little or no parameter tuning and are apparently quite robust to noise.
• Dan Ellis
  Using Learned Models to Organize Sound Mixtures
  Analyzing sound mixtures into individual waveforms proves very difficult, except in constrained circumstances such as a small number of spatially-compact sources. However, a higher-level task of recognizing simultaneous phrases from a constrained grammar is unexpectedly successful. I will argue that strong expectations, in the form of prior models of source signals, are the key to seemingly impossible source separation problems. The challenge, then, for both computational systems and models of human audition, is how to construct, represent, and deploy these models.
• Jan Schnupp
  Temporal spike pattern codes for complex sounds in primary auditory cortex
  Most previous studies on the representation of natural sounds in auditory cortex have focussed heavily on variations in spike counts. For example, marmoset A1 neurons often respond more vigorously to natural than to time reversed marmoset twitter calls, while neurons recorded in cat A1 show no such selectivity for natural marmoset calls. The functional role of this spike count based selectivity for conspecific calls is, however, unclear. To investigate whether such call selectivity in A1 can arise from experience, we recorded responses to marmoset calls in A1 of naive ferrets, as well as in ferrets that had been trained to recognize natural marmoset calls. We found that training did not induce call selectivity for the trained vocalizations in A1. However, even though ferret A1 neurons were not "call selective" they efficiently represented the vocalizations through temporal pattern codes, and trained animals recognized marmoset twitters with a high degree of accuracy. These temporal patterns needed to be analyzed at timescales of 10-50 ms to ensure efficient decoding. Training led to a substantial increase in the amount of information transmitted by these temporal discharge patterns, but the fundamental nature of the temporal pattern code remained unaltered. To further investigate the role these temporal pattern codes might play in perception, we compared the discrimination performance afforded by these spike pattern codes (a "neurometric" function) to behavioural performance (a psychometric function). Human psychophysical data were collected for a paired same-difference task, in which a natural twitter was presented along with twitters containing locally time-reversed bins of 10, 20, 40 or 80 ms. The listeners ability to distinguish locally time-reversed twitters from unmanipulated sounds was perfect when the reversed time windows were 80 ms wide, but declined dramatically for time windows of 20 ms or less. We also recorded responses of ferret A1 neurons to these twitter stimuli, and used statistical algorithms to determine if the temporal response patterns could discriminate between the natural and the locally time-reversed twitters. We found that the neurometric function derived from the A1 responses matched human psychophysics very well, provided the neural response pattern was analysed at a resolution of 20 ms or finer.
• Enrique Lopez-Poveda
  Auditory-nerve encoding of high-frequency spectral notches: time or place codes?
  High-frequency spectral notches such as those generated by the pinna are useful cues for sound localization. Because their center frequency (7-8 kHz) is beyond the cut-off of phase locking, the common view is that the internal representation of such notches does not involve the timing of action potentials and that, instead, they must be exclusively represented in terms of auditory nerve rate profiles. Recently, we (Alves-Pinto and Lopez-Poveda 2005. J. Acoust. Soc. Am. 118, 2548) have shown that the threshold notch depth for discriminating between a flat-spectrum noise and a noise with a square spectral notch centered at 8 kHz varies nonmonotonically with increasing sound level, and that discrimination is worst for levels around 80 dB SPL. Here, we show that the information in the auditory nerve rate profiles is insufficient to account for this psychophysical result, and that the timing of spikes occurrences is almost certainly used for discriminating between noise bursts with and without high-frequency spectral notches.
• Tim Gollisch
  Spike timing, cascade models, and reverse correlation in the auditory system
  Neural responses in the auditory system are often fast and precise. Their temporal characteristics, for example high precision in spike timing, form essential aspects of the encoding of acoustic signals, which are inherently temporal in nature. A better understanding of the signal processing in auditory systems may therefore be achieved by detailed investigations of the dynamics that shape the temporal structure, sensitivity, and accuracy of neural responses. The analysis of these dynamics with electrophysiological data is complicated by the required high temporal resolution and the multiple processes that occur on similar time scales. Spike jitter, for example, can distort the traditional measurements of temporal receptive fields based on reverse correlation. Similarly, the different filter processes of the sound signal by the mechanics of the ear and by the receptor cell membrane are difficult to disambiguate. To incorporate this complexity of signal processing in the analysis, we use multi-stage cascade models and extend techniques from nonlinear systems analysis to extract the relevant parameters from the electrophysiological data. This results in a generic framework for incorporating different spike timing dynamics, thereby improving receptive field estimates as well as obtaining quantitative models for spike timing. Using the auditory periphery of locusts as a model system, we were able to resolve the sub-millisecond filtering characteristics in the ear and found, for example, that the temporal receptive fields are considerably sharpened when spike jitter is accounted for.
• Philip Joris, Marcel van der Heijden
  Monaural and binaural temporal processing
  The use of submillisecond temporal information is best documented for binaural hearing, at both the perceptual and physiological level. We study the form and extent of monaural temporal information in the auditory nerve; its transformation at the first synaptic level in the central nervous system; and its conversion to a rate code at the level of the olivary complex and inferior colliculus. In reviewing this research, we will touch on important themes and ingredients regarding spike timing: fine-structure vs. envelope coding, monaural and binaural coincidence detection, internal delays, and temporal vs. rate coding. Exquisite temporal coding is also present in pathways that are not part of the binaural system: the significance of these neural responses is unclear.
• Jonathan Simon
  Neural Coding of Multiple Stimulus Features in Auditory Cortex [PDF]
  Determining how neurons encode relevant features of a particular stimulus is an important issue in sensory neuroscience. Experimentally, one might identify a single stimulus feature of interest, and then search in one particular auditory area for neurons that might encode that feature. A more difficult problem is to find encoding schemes for multiple (concurrent) stimulus properties. For instance, natural sounds, including speech, may contain multiple temporal features concurrently, e.g. independent amplitude modulation (AM) and frequency modulation (FM). Neural encoding of either AM or FM alone may be accomplished by modulating the neural firing rate at the stimulus modulation rate, but this simplistic encoding method does not generalize well to multiple concurrent modulations. Using non-invasive magnetoencephalography (MEG) to search all cortical areas simultaneously, and using stimuli that are independently varied in their AM and FM content, we find evidence for two separate neural mechanisms used in human auditory cortex, each of which may encode the two features concurrently.
• Jonathan Fritz
  The Chameleon Neuron - Receptive Field Adaptation to Changing Acoustic Salience
  The conventional wisdom of the A1 cortical receptive field, whose acoustic filter properties were malleable only during an early critical period during development, but were then fixed for the rest of the life of the animal, held sway for many years in auditory neuroscience. This view is changing as we gather new insights into the dynamic and adaptive filter properties of cortical neurons, particularly in the awake, behaving animal (but also by studies in the anesthetized animal showing neural adaptation to a wide range of experience including recent stimulus statistics, or previous classical conditioning or instrumental task learning). Our lab has developed an experimental approach that allows us to view cortical plasticity on-line as the animal meets different behavioral challenges by changing its cortical filters to enhance performance. We have observed clear, predictable task-related changes in STRF shape while the animal performs a variety of acoustic tasks (including single tone and multi-tone detection, and two-tone discrimination) with different tonal targets. These changes can occur quite rapidly, within 1-2 minutes of task onset, and can fade just as quickly after task completion, or in some cases, persist for hours. The same cell can multiplex by differentially changing its receptive field in different task conditions. These results suggest that there is a plasticity algorithm in A1 that can change STRF shape by transforming receptive fields to enhance figure/ground separation, by using a contrast matched filter to filter out the background, while simultaneously forming a sharper image of the salient acoustic target to which the animal is attending. In a sense, the discovery that sensory cortical cells have a chameleon-like nature, is hardly surprising. After all, most animals are generalists rather than specialists; and by necessity, must master and perform a number of different tasks each day and use a wily array of behavioral strategies in order to survive. Therefore, evolution might favor a nimble, dynamic, adaptive brain that could quickly reshape its sensory filter properties and sensori-motor links on a moment-to-moment basis, depending upon the current challenges the animal faces. We will briefly discuss several questions arising from this new view: (a) how does the brain orchestrate the dynamic changes, which are often precise and rapid, across wide expanses of sensory cortex? (b) How do the short-term changes observed over a period of seconds, minutes or hours translate into long-term changes? (c) How are we able to maintain stable percepts when the brain is constantly in flux?
• Nicol Harper
  The optimality approach in the auditory system
  In most of science we examine mechanism, how a system works. But in biology we also have the question of function, what the system does for the organism. By natural selection, systems will evolve towards the optimal arrangement given the needs of the animal and biophysical constraints. Understanding what the optimal system is can be a useful tool for understanding actual biological systems. In the nervous system the most common optimization principles examined have been either maximizing mutual information or minimizing expected coding error, which are related but not equivalent measures. Such optimization principles can be used to provide a unifying scheme for diverse and seemingly complex phenomena such as the range of impulse responses in the auditory nerve (Smith & Lewicki 2006), the diversity to adaptive responses to sound level (Dean et al 2005), and the diversity of population codes for interaural delay found in different animals (Harper and McAlpine 2004). These principles can be powerful enough to make firm predictions across species. We shall show the population coding predictions made from minimizing the expected coding error for interaural delay in the macaque are supported when tested in the awake macaque auditory cortex. [References: Smith E, Lewicki MS (2006) Efficient auditory coding. Nature, 439: 978-82; Dean I, Harper NS, McAlpine D (2005) Neural population coding of sound level adapts to stimulus statistics. Nat. Neurosci. 8:1684-9; Harper NS, McAlpine D (2004) Optimal population coding of an auditory spatial cue. Nature 430:682-6.]
• David McAlpine
  What is the auditory system evolved to do?
• Roy Patterson, Toshio Irino
  Size Matters: How the auditory system produces its scale invariant representation of the message in a sound [PDF] [text]
  When a child and an adult say the same word, it is only the message that is the same. The child has a shorter vocal tract and lighter vocal cords, and as a result, the waveform carrying the message is quite different for the child. The fact that we hear the same message shows that the auditory system 'normalizes' speech sounds for Vocal Tract Length and Glottal Pulse Rate. These normalization mechanisms are crucial for extracting a size invariant version of the message from the sound that carries it from speaker to listener. The vocal apparatus of most mammals is like that of humans, and for animals, size information is particularly important for assessing the sex of an individual and their ability to defend a territory. This talk is about the form of scale information in communication sounds like animal calls, speech, and music, and how the mammalian auditory system parses these sounds to extract the segregate the scale information and produce a size-invariant version of the message.
• Rhodri Cusack
  Auditory perceptual organisation: extracting regularities
• Bill Yost
  Sound Source Perception: A future for Auditory Attention and Memory [PDF]
  Trying to study real world sounds in real world contexts means complex acoustic conditions. That is, sounds that are temporally and spectrally complex covering significant periods of time and broad band widths. We know that the auditory system can process sound in small units of time and in small critical band widths and in many contexts (especially in the laboratory) this is the default mode of processing. Thus, processing sound in complex acoustic environments requires some form of integration across time and spectrum, without losing vital information about the short term and narrow band acoustics. Put another way how are the temporal and spectral pieces of an auditory scene stored and how are they retrieved? This seems to mean knowing more about auditory memory and auditory This seems to mean knowing more about auditory memory and auditory attention than we currently do. I maintain that our field in general has spent little time on these questions and as a result we currently know very little. The literature is full of references to the words attention and memory, but without citing any research, e.g., "at a cocktail party the listener often attends to one sound source and ignores the others." But, few study attention or memory. I will try to present an argument that attention and memory will be of increased interest in the future for those who want to better understand hearing. "at a cocktail party the listener often attends to one sound source and ignores the others." But, few study attention or memory. I will try to present an argument that attention and memory will be of increased interest in the future for those who want to better understand hearing.

---

This workshop is funded by the European Commission FET-Open Coordination Action S2S^2, with support from Phonak and Oticon, as well as by the generous contribution of the participants to their expenses. Thanks to all!

---

[Audition team] [LPP] [DEC] [ENS] [CNRS] [Université Paris 5] [S2S^2]