Is Dialog Intelligibility the New Loudness?

The Intelligibility Problem

Fast forward to today, and now the largest source of audio related complaints to broadcasters and streamers is dialog intelligibility.

“Dialogue intelligibility remains the top complaint in broadcast audio — and it’s getting worse. A recent Xperi survey reports that 84% of consumers have difficulty understanding dialogue in TV shows and films.” [3]. “The issue of poor speech intelligibility has ranked number one on the ‘negative hit-list’ of viewer complaints for quite some time.” [4].

Also, some tests have shown that poor dialog intelligibility negatively affects the “stickiness” of a show (the likelihood that a viewer will watch to the end), and therefore, ultimately, the revenue of the broadcaster/streamer.

Intelligibility Measurement

There is not currently one universally preferred standard for measuring dialog intelligibility. In recent years an algorithm developed by Fraunhofer IDMT is gaining acceptance, called the “Listening Effort Meter” (LE Meter). [5]

The Fraunhofer LE meter uses a Deep Neural Network to identify individual phonemes in an audio clip, with the confidence of the identification leading to a measure of Listening Effort (the more confident, the less effort, and the more intelligible) [6]. This method has proven to be largely language agnostic, and has been validated against human subject listening tests.

Several software and hardware manufacturers have, or are working on, including the LE meter in commercial products (including Steinberg, NUGEN Audio, Telos and RTW).

Intelligibility Control

But the question remains whether some form of dialog intelligibility measure should become a QC check for broadcasters / streamers, or should it become a recommended measure to aid program makers?

In either case there are questions about what the most useful presentation of the raw LE meter readings should be. The LE meter produces a reading many times per second (every 30ms), and these values can vary a lot over a short time frame, making them hard to read. So, as with loudness measurements, some sort of windowing, smoothing and / or ballistics is likely to be required.

Also, for a QC check, ideally there would be a single “yes / no” flag for each program, answering the question “is the dialog intelligibility ok in this program” (or a single number to represent “how good is the dialog intelligibility in this program”).

It is not obvious how this single number for a program should be derived from the raw data. Various statistical measures could be used, but there are some situations when they may not give a full enough picture. One could use the mean or median average of the readings, or the upper or lower quartile. These statistical measures could be taken on the raw LE meter data, or the smoothed data (which may give different results).

However, it is not hard to imagine situations where these might not give the information we really need.

For example, consider a program with excellent dialog intelligibility throughout, except for one 5 minute scene with plot-critical dialog in a noisy environment, which was very hard to understand. In this case, an “average” intelligibility score may well read as being fine, but it might be considered that this should have been a QC fail as viewers may get turned off by the one hard-to-understand scene.

Finding solutions for this sort of problem remains a matter for future research.