Spirent begins speech-intelligibility testing on LTE voice, plans to test MCPTT in the future

Spirent Communications has done initial testing of LTE telephony voice in recognized public-safety noise scenarios and plans to continue such tests as technologies like 3GPP-standard mission-critical-push-to-talk (MCPTT) services are introduced in the market, according to company officials.

Testing of LTE voice is common in the commercial marketplace, but Spirent Communications has decided to test LTE devices and determine their usability in the kind of noisy environments in which public-safety personnel frequently have to work—for instance, near a chainsaw or in a nightclub.

And, instead of simply measuring voice quality in terms of (MOS) scores via the POLQA method used by carriers, Spirent Communications also is conducting tests that determine a devices speech intelligibility via the ABC-MRT16 algorithm. This is the same testing method that was used by NIST/PSCR researchers to test the speech intelligibility of digital LMR radios in fireground environments, according to Eric Sinclair, Spirent Communications’ senior manager of business development for public safety and mission-critical services. In fact, the same noise environments that were used in those PSCR tests on LMR device are being used by Spirent, he said.

“PSCR talked about their algorithm for speech intelligibility and how that was a much better algorithm to detect the speech in emergency situations,” Sinclair said during an interview with IWCE’s Urgent Communications. “These are the exact same noises. We basically followed their guidelines and implemented the same noises and the same algorithms.”

Sinclair said that Spirent Communications tried to use the same methodology to mimic the PSCR speech-intelligibility testing, but he noted that there could be some slight differences—for instance, the Spirent test labs in today may use more speakers than were used in the PSCR testing years ago.

Otherwise, “they [the testing results] should align, in theory,” Sinclair said.

In its initial tests, Spirent tested the “native dialer” telephony capabilities—not voice over LTE (VoLTE) services–of four unidentified LTE handsets with noise-cancelling technology, at least some of which are approved FirstNet devices, Sinclair said.

“We realized that, in terms of speech intelligibility, the differences were pretty big,” he said. “And that’s one thing right now that the industry is not testing for—speech intelligibility. The conclusion of the study is that we should move into that direction and test a little bit more before we announce those devices.”

In particular, the study noted that some devices that the MOS-based POLQA carrier-quality scores did not necessarily translate into speech-intelligibility scores, Sinclair said. As an example, one device with poor MOS scores in noisy environments registered some of the best results when tested on a speech-intelligibility basis, he said.

“The current way of testing speech quality—using POLQA, for example—is not really adequate,” Sinclair said.

Sinclair said that Spirent Communications plans to conduct the same voice-quality tests on MCPTT offerings when they become available, and the company may consider testing some LMR devices to provide a point of comparison to public-safety entities. AT&T officials have announced plans to introduce MCPTT offerings from multiple vendors to FirstNet users by the end of the year.

Saul Einbinder, Spirent Communications’ vice president of venture development, said he visited the Spirent lab In Frederick, Md., during the testing and was surprised that all of the LTE devices were able to transmit understandable voice signals in such noisy environments.

“When I was down in the room in Frederick in the performance lab, it’s incredibly difficult to hear what’s going on, as a human,” Einbinder said during an interview with IWCE’s Urgent Communications. “When I look at the scores, they’re remarkably good. There’s definitely a story about some devices not being nearly as good as others, but what I personally got excited about was just how good some of the scores are in the midst of this K12 saw that’s cutting through steel.

“What I come away thinking is, ‘How does the industry get to a point where it’s very, very good much more often?’

One key is for such testing to become a priority for carriers that serve the public-safety community, according to Einbinder.

They have the ability to drive manufacturers to do this type of testing to make the services better for everybody,” he said. “To me, that’s the most interesting thing. There’s a lot of room to grow and to get better.

“There’s a proof point that we can get better. I think, if the carriers embrace their longstanding desire to be more than a pipe, that’s the role that they would play here that would really be profound for the industry.”

Ken Rehbehn, IHS Markits analyst and a volunteer firefighter in Virginia, said that the kind of public-safety-specific testing being done by Spirent is an important step for first responder to adopt LTE-based technologies.

“We went through this trauma with the land mobile radio, where the early codecs that translated voice into digital were subpar,” Rehbehn said. “It had a tremendous impact on the confidence of the users and the takeup of the systems. It was addressed, but it took a lot of work.

“As we have LTE devices coming out, it’s important that those LTE devices provide high-quality sound, in a variety of environments.”

Given the additional spectrum and higher-level codec that broadband technologies like LTE can use, Rehbehn said it is logical to believe that LTE voice technologies would deliver higher-quality voice when devices can access a broadband network, but this needs to be demonstrated.

“I think there should be an advantage to broadband, in terms of voice quality,” Rehbehn said. “You have a better channel to work with, as far as the amount of information that can be sent, in comparison to a Project 25 system. That’s a help to the codecs.

“But you still have to test.”

Detailed information about Spirent Communications’ initial public-safety LTE testing can be read in a white paper that is available via a free download at https://www.spirent.com/assets/wp/wp_speech-intelligibility-evaluation-highlights-differences-between-lte-public-safety-devices.