Speech technology for language learning

In collaboration with our industrial partners, we develop computer-assisted pronunciation training tools for Dutch learners of English.

In all areas of life there is a growing demand for people who speak languages well. Yet pronunciation and language training require a lot of practice, with exercises and feedback tailored to the specific needs of each individual learner. Most traditional language classes cannot deliver that level of support, and as a result, computer-assisted language learning has become an area of great interest to the general public.

Our project concentrates on three themes: visualisation of pronunciation (and errors); utterance verification (speech recognition); and pronunciation quality assessment.

Further details:

  • Project website: ITSLanguage
  • Principal Investigator: Thomas Hain, Department of Computer Science, University of Sheffield
  • Collaborators: ITSLanguage BV, The Netherlands
  • Dates: Nov 2012 – Feb 2014 (part-time)

Selected publications

  • M. Nicolao, A. V. Beeston, and T. Hain, “Automatic assessment of English learner pronunciation using discriminative classifiers,” in 40th IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Brisbane, Australia, 19-24 Apr, 2015, pp. 5351-5355.
    [BibTeX] [paper]
    @inproceedings{Nicolao:2015icassp,
    author = {Nicolao, M. and Beeston, A. V. and Hain, T.},
    title = {{Automatic assessment of English learner pronunciation using discriminative classifiers}},
    booktitle = {{40th IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)}},
    pages = {5351-5355},
    year = {2015},
    address = {Brisbane, Australia, 19-24 Apr},
    paper = {http://staffwww.dcs.shef.ac.uk/people/M.Nicolao/site/publications_files/pron_assess.pdf},
    month = apr
    }

  • A. V. Beeston, M. Nicolao, and T. Hain, “Pairwise audio comparison for visualisation of mispronunciation,” in UK Speech Conference, Cambridge, UK. Poster. 17-18 Sep, 2013.
    [BibTeX] [poster]
    @inproceedings{Beeston:2013it,
    author = {Beeston, Amy V and Nicolao, Mauro and Hain, Thomas},
    title = {{Pairwise audio comparison for visualisation of mispronunciation}},
    booktitle = {{UK Speech Conference}},
    year = {2013},
    address = {Cambridge, UK. Poster. 17-18 Sep},
    poster = {http://staffwww.dcs.shef.ac.uk/people/A.Beeston/dl/Poster_ukspeech2013_pronunciation.pdf},
    month = sep
    }

ITSLanguage (annotation tool 2013)

ITSLanguage (analysis diagram 2013)