mcld.co.uk

InterSpeech 2016 was a very interesting conference. I have been to InterSpeech before, yes - but I'm not a speech-recognition person so it's not my "home" conference. I was there specifically for the birds/animals special session (organised by Naomi Harte and Peter Jancovic), but it was also a great opportunity to check in on what's going on in speech technology research.

Here's a note of some of the interesting papers I saw. I'll start with some of the birds/animals papers:

• Localizing Bird Songs Using an Open Source Robot Audition System with a Microphone Array by Suzuki et al. Some really interesting work from a Japanese group I hadn't met before - they use a simple USB multi-microphone device together with source-localisation software, and then use that to recover a multi-species transcript of activity. Potentially very useful.
• Recognition of Multiple Bird Species Based on Penalised Maximum Likelihood and HMM-Based Modelling of Individual Vocalisation Elements by Jancovic and Kokuer - I like their modulated-sinusoid detector - here it's incorporated into a system for multi-species recognition in polyphonic audio. Good to see this being developed. It'd be great if others could make use of this technique (I don't think source code is available...).
• Bird Song Synthesis Based on Hidden Markov Models by Bonada et al - this is work Jordi Bonada developed while on a research visit to our lab (C4DM), with a colleague of mine Rob Lachlan. Good probabilistic generation of birdsong needs to be developed, you see, because at the moment a lot of bird behaviour experiments use a small number of WAV files as play back, and the unnatural repetitiveness that results from this makes me uncomfortable! Hence we need work like this on flexible bird sound synthesis.
• Cost Effective Acoustic Monitoring of Bird Species by Ciira wa Maina. A neat paper about a prototype 'full solution', little Raspberry Pi sound recorders with a straightforward single-species sound detector.
• Feature Learning and Automatic Segmentation for Dolphin Communication Analysis by Kohlsdorf et al. I don't know what audio tracking methods dolphin researchers currently favour, but this is an interesting highly-scalable method based on feature-learning in spectrogram data.
• Robust Detection of Multiple Bioacoustic Events with Repetitive Structures by Frank Kurth. To be honest I don't grok this yet - it's a kind of enhanced autocorrelation, but I'm not sure what leads to it being said to be robust.
• A Real-Time Parametric General-Purpose Mammalian Vocal Synthesiser by Roger K Moore: a PureData patch for general-purpose mammal sound synthesis. Fun, and maybe useful for someone out there working on mammal sounds.

That's not all the bird/animal papers, sorry, just the ones I have comments about.

And now a sampling of the other papers that caught my interest:

• Retrieval of Textual Song Lyrics from Sung Inputs by Anna Kruspe. Nice to see work on aligning song lyrics against audio recordings - it's something that the field of MIR is in need of. The example application here is if you sing a few words, can a system retrieve the right song audio from a karaoke database?
• The auditory representation of speech sounds in human motor cortex - this journal article has some of the amazing findings presented by Eddie Chang in his fantastic keynote speech, discovering the way phonemes are organised in our brains, both for production and perception.
• Today's Most Frequently Used F0 Estimation Methods, and Their Accuracy in Estimating Male and Female Pitch in Clean Speech by Sofia Strömbergsson. This survey is a great service for the community. The general conclusion is that Praat's pitch detection is really among the best off-the-shelf recommendations (for speech analysis, here - the evaluation hasn't been done for non-human sounds!).
• Supervised Learning of Acoustic Models in a Zero Resource Setting to Improve DPGMM Clustering by Heck et al - "zero-resource" speech analysis is interesting to me because it could be relevant for bird sounds. "Zero resource" means analysing languages for which we have no corpora or other helpful data available - all we have is audio recordings. (Sounds familiar?) In this paper the authors used some adaptation techniques to improve a method introduced last year based on unsupervised nonparametric clustering.
• Speech reductions cause a de-weighting of secondary acoustic cues by Varnet et al: a study of some niche aspects of human listening. Through tests of people listening to speech examples in noise they found that people's use of secondary cues - i.e. clues that help to distinguish one phoneme from another, which clues are embedded elsewhere in the word than the phoneme itself - changes according to the nature of the stimulus. Yet more evidence that perception is an active, context-sensitive process etc.

One thing you won't realise from my own notes is that InterSpeech was heavily dominated by deep learning. Convolutional neural nets (ConvNets), recurrent neural nets (RNNs), they were everywhere. Lots of discussion about connectionist temporal classification (CTC) - some people say it's the best, some people say it requires too much data to train properly, some people say they have other tricks so they can get away without it. It will be interesting to see how that discussion evolves. However, many of the other deep-learning based papers were much of a muchness: lots of people use a ConvNet or an RNN and, as we all know, in many cases they can get good results. They apply these to many tasks in speech technology. However, in many cases there was application without a whole lot of insight. That's the way the state of the art is at the moment, I guess. Therefore, many of my most interesting moments at InterSpeech were deep-learning-less :) see above.

(Also, I had to miss the final day, to catch my return flight. Wish I'd been able to go to the VAD and Audio Events session, for example.)

Another aspect of speech technology is the emphasis on public data challenges - there are lots of them! Speech recognition, speaker recognition, language recognition, distant speech recognition, zero-resource speech recognition, de-reverberation... Some of these have been running for years and the dedication of the organisers is worth praising. Useful to check in on how these things are organised, as we develop similar initiatives in general and natural sound scene analysis.