https://urgentcomm.com/wp-content/themes/ucm_child/assets/images/logo/footer-logo.png
  • Home
  • News
  • Multimedia
    • Back
    • Multimedia
    • Video
    • Podcasts
    • Galleries
  • Commentary
    • Back
    • Commentary
    • Urgent Matters
    • View From The Top
    • All Things IWCE
    • Legal Matters
  • Resources
    • Back
    • Resources
    • Events
    • Webinars
    • White Papers
    • Reprints & Reuse
  • IWCE
    • Back
    • IWCE
    • Conference
    • Special Events
    • Exhibitor Listings
    • Premier Partners
    • Floor Plan
    • Exhibiting Information
    • Register for IWCE
  • About Us
    • Back
    • About Us
    • Contact Us
    • Advertise
    • Terms of Service
    • Privacy Statement
    • Cookies Policy
  • Related Sites
    • Back
    • American City & County
    • IWCE
    • Light Reading
    • IOT World Today
    • Mission Critical Technologies
    • Microwave/RF
    • T&D World
    • TU-Auto
  • In the field
    • Back
    • In the field
    • Broadband Push-to-X
    • Internet of Things
    • Project 25
    • Public-Safety Broadband/FirstNet
    • Virtual/Augmented Reality
    • Land Mobile Radio
    • Long Term Evolution (LTE)
    • Applications
    • Drones/Robots
    • IoT/Smart X
    • Software
    • Subscriber Devices
    • Video
  • Call Center/Command
    • Back
    • Call Center/Command
    • Artificial Intelligence
    • NG911
    • Alerting Systems
    • Analytics
    • Dispatch/Call-taking
    • Incident Command/Situational Awareness
    • Tracking, Monitoring & Control
  • Network Tech
    • Back
    • Network Tech
    • Interoperability
    • LMR 100
    • LMR 200
    • Backhaul
    • Deployables
    • Power
    • Tower & Site
    • Wireless Networks
    • Coverage/Interference
    • Security
    • System Design
    • System Installation
    • System Operation
    • Test & Measurement
  • Operations
    • Back
    • Operations
    • Critical Infrastructure
    • Enterprise
    • Federal Government/Military
    • Public Safety
    • State & Local Government
    • Training
  • Regulations
    • Back
    • Regulations
    • Narrowbanding
    • T-Band
    • Rebanding
    • TV White Spaces
    • None
    • Funding
    • Policy
    • Regional Coordination
    • Standards
  • Organizations
    • Back
    • Organizations
    • AASHTO
    • APCO
    • DHS
    • DMR Association
    • ETA
    • EWA
    • FCC
    • IWCE
    • NASEMSO
    • NATE
    • NXDN Forum
    • NENA
    • NIST/PSCR
    • NPSTC
    • NTIA/FirstNet
    • P25 TIG
    • TETRA + CCA
    • UTC
Urgent Communications
  • NEWSLETTER
  • Home
  • News
  • Multimedia
    • Back
    • Video
    • Podcasts
    • Galleries
  • Commentary
    • Back
    • All Things IWCE
    • Urgent Matters
    • View From The Top
    • Legal Matters
  • Resources
    • Back
    • Events
    • Webinars
    • White Papers
    • Reprints & Reuse
    • UC eZines
  • IWCE
    • Back
    • Conference
    • Special Events
    • Exhibitor Listings
    • Floor Plan
    • Exhibiting Information
    • Register for IWCE
  • About Us
    • Back
    • About Us
    • Contact Us
    • Advertise
    • Terms of Service
    • Privacy Statement
    • Cookies Policy
  • Related Sites
    • Back
    • American City & County
    • IWCE
    • Light Reading
    • IOT World Today
    • Mission Critical Technologies
    • Microwave/RF
    • T&D World
    • TU-Auto
  • newsletter
  • In the field
    • Back
    • Internet of Things
    • Broadband Push-to-X
    • Project 25
    • Public-Safety Broadband/FirstNet
    • Virtual/Augmented Reality
    • Land Mobile Radio
    • Long Term Evolution (LTE)
    • Applications
    • Drones/Robots
    • IoT/Smart X
    • Software
    • Subscriber Devices
    • Video
  • Call Center/Command
    • Back
    • Artificial Intelligence
    • NG911
    • Alerting Systems
    • Analytics
    • Dispatch/Call-taking
    • Incident Command/Situational Awareness
    • Tracking, Monitoring & Control
  • Network Tech
    • Back
    • Cybersecurity
    • Interoperability
    • LMR 100
    • LMR 200
    • Backhaul
    • Deployables
    • Power
    • Tower & Site
    • Wireless Networks
    • Coverage/Interference
    • Security
    • System Design
    • System Installation
    • System Operation
    • Test & Measurement
  • Operations
    • Back
    • Critical Infrastructure
    • Enterprise
    • Federal Government/Military
    • Public Safety
    • State & Local Government
    • Training
  • Regulations
    • Back
    • Narrowbanding
    • T-Band
    • Rebanding
    • TV White Spaces
    • None
    • Funding
    • Policy
    • Regional Coordination
    • Standards
  • Organizations
    • Back
    • AASHTO
    • APCO
    • DHS
    • DMR Association
    • ETA
    • EWA
    • FCC
    • IWCE
    • NASEMSO
    • NATE
    • NXDN Forum
    • NENA
    • NIST/PSCR
    • NPSTC
    • NTIA/FirstNet
    • P25 TIG
    • TETRA + CCA
    • UTC
acc.com

content


Benchmarking tools enhance wireless test and measurement for competitive analysis

Benchmarking tools enhance wireless test and measurement for competitive analysis

Benchmarking tools are not only a means of rectifying specific network problems and ensuring ongoing quality. They also provide documentation of competitive
  • Written by Urgent Communications Administrator
  • 1st April 1998

Benchmarking tools are not only a means of rectifying specific network problems and ensuring ongoing quality. They also provide documentation of competitive advantage over similar wireless services.

Competitive benchmarking tools are a new addition to the test and measurement arsenals of wireless carriers. These tools allow carriers to measure their quality of service (QOS) against competing cellular, personal communications services (PCS) and enhanced specialized mobile radio (ESMR) carriers in their geographic markets.

What is competitive QOS? The importance of competitive QOS has grown as the wireless industry has matured and new licenses have been granted or auctioned to aggressive new carriers. QOS will separate the winners from the losers in the competitive wireless industry.

Before the turn of the millennium, high-quality audio performance from wireless systems will become a competitive necessity, something that subscribers will take for granted. Operators who fail to deliver acceptable audio quality performance-as measured by audio quality and other metrics-will rapidly lose customers to any of several competitors, be they traditional cellular carriers or the new breed of PCS, ESMR, wireless local loop (WLL) and mobile satellite service (MSS) providers.

The subscriber’s ear is the final judge of any wireless telephone system. Customers do not care about network technology and engineering issues, such as time-division multiple-access (TDMA) vs. code-division multiple-access (CDMA). They just care about performance. They will have every incentive to switch competing wireless service providers, technology notwithstanding, in the same way that they already change their landline long-distance providers.

This means that wireless operators will have to increasingly focus on subscriber- perceptions of quality. They will need to concentrate their efforts on areas of competitive weakness, such as the incidence of noise and distortions due to radio frequency (RF) phenomena. Quality-related systems test, measurement and optimization activities will grow in importance.

QOS is a composite metric made up of several call-related factors that contribute directly to end-user satisfaction.

Call initiation – When the subscriber hits “send,” does the call go through? There are three possible results from hitting the “send” key: * no-service areas – The call does not go through, and the subscriber has to wait until he or she reaches another coverage area. * blocked calls – The call does not go through, and the subscriber has to hit the “send” key a second time. * good initiations – The call goes through.

Call access – Assuming a good initiation, how long did it take for the system to grant the subscriber access to a voice channel? This is not a critical factor unless access times exceed 15 seconds, at which point the subscriber starts to wonder if the phone is working properly.

Call quality – Once the call has gone through, how clear, intelligible and distortion-free is the audio transmitted between both ends of the connection? Graded on an MOS (mean opinion score) basis, there are five possible scores: 5 = excellent (no distortion detected). 4 = good (a little distortion detected; not annoying to the listener). 3 = fair (some distortion; slightly annoying). 2 = poor (significant distortion; annoying). 1 = bad (almost unintelligible; very annoying).

Call completion – Assuming that audio quality is good enough (and does not annoy callers to the point where they abandon the call in frustration), was the call completed successfully? A call can end for the subscriber in two ways: * normal end – The call ends on demand, when the caller presses the “end” key or replaces “on-hook.” * dropped calls – The system drops the call before the customer is ready, such as through faulty hand-offs.

Each carrier (indeed, each subscriber) will apply different weightings to these various QOS factors, depending on background, expectations, applications and the current state of wireless technologies.

The quality bar on all of these parameters will only get higher over time. Customers have come to expect continual improvements in wireless system performance in voice quality, due to technological advances and fierce competition. As wireless systems mature, customers will expect wireless voice quality on a par with traditional wireline telephone service.

Troubleshooting QOS problems QOS and user satisfaction not only depend on the expectations of subscribers, but also on the intricate, end-to-end network infrastructure that supports a call. The perceived quality of audio transmissions in a wireless network depends on the performance of every system and component between the people speaking, including every switch, trunk, line, cross-connect and radio-frequency (RF) channel. It also depends on the quality of the subscribers’ mobile terminals and how those phones are used.

When analyzing the quality of a given voice call, one must consider the technical environment specific to that call. Most calls placed from or to cellphones involve connections with the landline public switched telephone network (PSTN), as shown in Figure 1 on page 16.

Audio quality problems can be introduced at any of the following nodes or circuits: * landline phone. * inside wiring-including jacks, horizontal cabling, punchdown blocks and cross-connects-connecting the landline phone to an office private branch exchange (PBX). * PBX, including trunks connecting to the PSTN. * PSTN, including trunks connecting to the cellular operator’s mobile telephone switching office (MTSO). * MTSO. * base station controller (BSC) or mobile system controller (MSC). * voice coder. * base station. * RF channel. * cellphone.

On this end-to-end call configuration, the only components typically under the wireless service provider’s control are the MTSO, BSC or MSC, voice coder, base station, intranetwork trunk routes and, to a lesser extent, the RF channel. Service providers have little control over the type or quality of terminal equipment employed by their subscribers or over the subscriber’s usage patterns or habits.

In most operational wireless networks, performance of infrastructure hardware-the MTSO, coder, trunks and base stations-is fairly stable. The basic network architecture is modified slowly over time, usually in a tightly controlled fashion subject to thorough testing and quality assurance. Consequently, infrastructure establishes a baseline quality to which customers, theoretically, usually adjust their expectations.

Isolating RF channel QOS problems The one QOS factor that fluctuates constantly is the RF channel. Its reliability is affected by dynamic atmospheric, morphologic, geologic, hydrologic and traffic conditions present between fixed base stations and roving users. Fading, multipath, interference, noise and hand-offs can degrade audio quality and cause it to fluctuate widely sometimes within a given phone call. One could argue that the bulk of users’ reported audio quality problems on wireless networks can be attributed to dynamic RF conditions that were not foreseen or addressed adequately in the network design.

Many audio problems are caused or aggravated by RF factors and are commonly experienced in calls between landline and wireless telephone systems. The problems include: fading, static, noise, distortion, cross-talk, interference, clipping, echo and delay.

Wireless service providers usually address RF-related audio problems through regular modifications to their RF network design, such as adding base stations and repeaters, subdividing cells and sectors, and changing the tilt and beamwidth of transmitters.

No wireless carrier can claim to have eliminated RF-related QOS deficiencies from every hill, dale and office park in its market. Carriers do their best to maximize coverage, keep unwanted interference at bay, and minimize bit-error rates and dropped handoffs, but RF links are notoriously trouble-plagued. Bedeviled by tricky terrain, volatile weather conditions, and the ever-shifting landscape of fixed and mobile obstructions, RF propagation modeling is an inexact science-an engineer’s rolling headache of network remodeling, retesting and reconfiguration. Each wireless carrier focuses on addressing a “Swiss cheese” pattern of weak zones in its system while searching for similar deficiencies in competitors’ networks that can be called to a customer’s attention.

Competitive benchmarking tools Into this intensifying dogfight step the competitive benchmarking tools. To design our tool system, Benchmarker, we identified three fundamental tasks the system would have to perform: emulation, comparison and analysis.

*Emulate the subscriber – The benchmarking tool should automatically place and receive wireless phone calls; take QOS measurements on those calls (initiation, access, quality and completion factors); geolocate QOS measurements with a navigation technology, such as GPS; and log measurements with navigation coordinates.

*Compare access protocols, bands, and service providers – The tool should place calls and take QOS measurements on all wireless access protocols, bands, and service providers in a given geographic market. This would enable “apples-to-apples” QOS comparisons across different service providers implementing different technologies. It should be possible to place multiple calls concurrently from the same mobile benchmarking tool to as many as eight wireless service providers, reflecting the number of available cellular, PCS, and ESMR (and, soon, mobile satellite) service providers available in a given geographic market. A full-fledged competitive benchmarking tool should support all of the following common digital and analog wireless protocols: AMPS, IS-54B, IS-136 (800MHz and 1.9 GHz), IS-95 (800MHz and 1.9 GHz), GSM (900MHz, 1.8GHz and 1.9GHz) and Iden.

*Analyze the data – The tool should come with software that produces detailed management level reports, statistical graphs and location plots. Using these outputs to identify competitive strengths and weaknesses, a wireless carrier can then use the engineering data collected simultaneously with the competitive data to locate and resolve problems quickly.

Benchmarking tool architecture A full-featured, competitive benchmarking tool requires mobile, landline, data collection and audio quality components.

*Mobile unit – This unit, typically installed in a “drive-test” vehicle, consists of multiple portable wireless phones and scan receivers covering all protocols and bands to be benchmarked; a navigation system, such as GPS or Travelpilot, to correlate measurement data with latitude and longitude coordinates; and a portable computer to set up measurement sessions, autodial the unit’s cellphones, store collected QOS metric data and view data as it is being collected. Special mounting hardware is needed to seat the unit’s chassis, and all portable phones, in the vehicle’s trunk. The mobile unit draws power through a direct connection to the vehicle’s battery.

*Landline unit – This unit, usually installed at a carrier’s network operations center, MSC or MTSO, consists of landline phones that automatically (via computer scripts) dial or answer calls to and from the cellphones in the mobile unit. This simulates the typical mobile-to-landline configuration of most wireless calls. Whereas the mobile unit collects QOS data on the downlink, the landline unit collects corresponding data on the uplink of a call. Time synchronization (“handshaking”) between the landline and mobile units is necessary to merge and correlate the resulting data to produce a composite portrait of wireless QOS.

*Data collection, analysis, post-processing and reporting – This requires software with geographic information system (GIS) capabilities. As previously discussed, this software should support collection, correlation, analysis and display of QOS data from uplink and downlink QOS data.

*Audio quality measurement – Both the mobile and the landline units should contain audio quality measurement modules, usually based on digital signal processing (DSP) technology, that produce MOS-like ratings of downlink and uplink audio quality on calls.

Audio quality assessment technology LCC’s audio quality assessment module, called Auryst (aural analyst) assesses acceptable voice quality measurements in both digital and analog wireless systems based on validated user audio perception. The module measures wireless network voice quality as it would be perceived by the average listener. The module generates audio quality scores (AQS) on a scale of 1 to 5, as previously discussed, that are highly correlated (>95%) to known differential MOS (DMOS) data validated by Comsat Laboratories, which performs listener evaluations.

The audio quality module is based on an LCC-developed model (patent pending) of auditory perception that is derived from wavelet decomposition theory and accounts for auditory nerve-firing rates. The model uses critical band analysis and focuses on elements of speech and sound that are critical to user perception, such as frequency sensitivity, time resolution, masking phenomenon and intensity.

The audio quality module uses built-in reference speech samples-male and female-that are transmitted through the wireless system to establish a standard of acceptable voice quality. Speech samples are of phonetically balanced Harvard sentence-pairs to ensure that the audio quality measurements are taken on a representative phonemic sampling. Using high-speed DSP technology and a custom filter bank, the module can analyze and quantify distortion between reference and impaired samples in real time. Alternately, this information can be collected for mapping, post-processing, analysis and reporting in a Windows-based, post-processing application.

The module is used to generate “source-referenced” AQS metrics, which are analogous to source-reference DMOS. Source-reference AQS measures RF- and coder-impaired speech against “clean,” pre-coder source speech. This is useful for characterizing the quality, including coder- and RF-related impairments, of calls on a single wireless system. Figure 2 on page 26 illustrates how the module computes an AQS.

AQS is measured in simplex mode, on either the forward or reverse channel, at any point in time during measurement. In addition to AQS, the audio quality module measures various follow-call parameters and call statistics, such as the incidence of call setup failures, busy signals, dropped calls, hand-off exceptions and completions.

The audio quality measurement module is designed to work with the leading digital and analog wireless protocols. RF engineers can receive and merge uplink and downlink measurements for audio quality, thereby supporting assessment of the baseline quality of an entire system. Moreover, engineers can use the technology to compare the quality of their service with that of competitors-regardless of which technology the competition uses.

The Benchmarker system measures audio quality on as many as eight digital or analog networks simultaneously. The landline audio quality measurement module performs uplink quality measurements on tones received from the mobile unit, while the mobile devices measure downlink audio in similar fashion, by measuring audio or tones transmitted from the landline unit.

Test calls can be initiated and answered automatically by either the landline or mobile unit. The landline unit module is capable of unattended, script-driven operation for each analyzer that controls calling and answering, and it can be located for call termination at a mobile switching center or at a standard public-switched telephone network (PSTN) line interface. The landline unit supports an interface to the analog North American PSTN access loops and to international digital phone systems.

Ongoing benchmarking Competitive benchmarking is an important activity that is of common interest to engineering, marketing, and executive-level personnel. Benchmarking is often used to set engineering goals and guide network expansion and optimization activities. Benchmarking must be based on valid methodologies that ensure that resultant data is clear and easy to interpret by all parties. To work through the complexities of a benchmarking operation, carriers should expect the benchmarking provider to offer tools, methodology and training.

*Benchmarking tool – The tool must support all of the carrier’s benchmarking requirements, as discussed above, and be easy to install, use, administer and maintain by engineering staff. Ideally, the tool should output data to RF post-processing and analysis packages that carrier personnel are already using.

*Methodology development – Without appropriate data collection and analysis procedures, the competitive benchmarking results obtained can be extremely misleading. Benchmarking procedures must mesh with the carrier’s system engineering and field operations and be capable of being carried out consistently, over time and across markets, by carrier personnel and engineering consultants. Better results can be obtained from benchmarking tools when they are bundled with consulting services that show the carrier’s engineering staff how to make the most effective use of the equipment, analyze the benchmark data, and use those analyses to modify and optimize their systems.

*Documentation, training, and support – Benchmarking tools and operational procedures must be presented to carrier staff through intensive, upfront training. The provider must be able to supply ongoing remedial training and in-service support. Well-organized, easy-to-read course materials, user manuals, application notes and service bulletins should be provided.

Effectively implementing a complex measurement system requires a carrier to rethink how it monitors its RF networks and to reconfigure support systems accordingly. The measurement system should include a landline unit, a mobile unit, and data collection, analysis, post-processing and reporting software with GIS capabilities.

Conclusion Competitive benchmarking solutions provide the hard QOS data demanded by today’s maturing, competitive wireless industry. With this data, wireless operators can perform intelligent optimization of their networks. Instead of simply looking for problem areas in their networks, they can look for areas of competitive weakness. In this way they can better prioritize which network problems need to be addressed and better answer the question of “How good is good enough?”

Supplying the data “ammunition” for quality “shoot-outs” between rival carriers, these tools are catalysts for raising industry-wide QOS levels to new heights. Subscribers-and the industry as a whole-can only benefit from this renewed attention to service quality.

Kobielus and Woessner are product managers for LCC International, McLean VA. Email: kobielus_james@lcc.com or woessner_andrew@lcc.com.

Tags: content

Related


  • Photo gallery: 2014 Communications Marketing Conference (CMC) in Tucson
    This photo gallery features scenes from the 2014 Communications Marketing Conference (CMC) in Tucson, Ariz.
  • Top 5 Stories - Week of Sept. 22
    Top 5 Stories - Week of Sept. 22
    Here’s a look at the most popular stories on IWCE’s Urgent Communications from the last week.
  • RCA plans to expand this year's Technical Symposium
    RCA plans to expand this year's Technical Symposium
    Recent presentations, given by both startups and major corporations, have included fractal antennas, LTE, emergency communications, "invisibility cloaks" using RF, disabling IEDs with RF and tropospheric ducting.
  • Newscan: Securing the Internet of Things is quite a challenge
    Newscan: Securing the Internet of Things is quite a challenge
    Also: EWA requests dismissal of 900 MHz applications; TIA names tech and policy priorities for 2014; IJIS Institute names Shumate Award winner; App makes bus waits more tolerable; a Blackberry comeback may be in the offing.

Leave a comment Cancel reply

To leave a comment login with your Urgent Comms account:

Log in with your Urgent Comms account

Or alternatively provide your name, email address below:

Your email address will not be published. Required fields are marked *

Related Content

  • Newscan: Texting 911 becomes an option for some Pennylvania cell-phone users
  • Newscan: Senator wants inventory of government spectrum
  • CMA conducts 40th conference this week
  • Newscan: Missouri county may have to drop 911 if sales-tax hike fails

Commentary


Public safety needs a better way to triage emergency calls

13th January 2021

In challenging year, working with public safety to move FirstNet forward

30th December 2020

Communications solutions must evolve quickly to meet needs of a changing world

31st October 2020
view all

Events


UC Ezines


IWCE 2019 Wrap Up

13th May 2019
view all

Twitter


UrgentComm

Become a Thought Leader at IWCE 2021! We are looking for dynamic speakers to present case studies and technical tal… twitter.com/i/web/status/1…

26th January 2021
UrgentComm

RT @IWCEexpo: 📆 Mark Your Calendars: IWCE will be returning to Las Vegas this September and registration is slated to open in April 📆 Wa…

15th January 2021
UrgentComm

RT @IWCEexpo: ⚡FLASH SALE: Don't miss this exclusive offer! Passes to #IBFVirtual are now 50% off with code TWITTER50. Take advantage of th…

6th November 2020
UrgentComm

Get ready for part 2 of "Ensuring Public Safety Emergency Communications" next week! @PCTEL_inc will explore… twitter.com/i/web/status/1…

3rd November 2020
UrgentComm

Over the past few months, we’ve seen the world transform, and it's clear that cities will be affected in the long-t… twitter.com/i/web/status/1…

27th October 2020
UrgentComm

Florida state & local agencies subscribing to the Statewide Law Enforcement Radio System (SLERS) will be able to co… twitter.com/i/web/status/1…

26th October 2020
UrgentComm

Tune in to @slacorp CEO Josh Lober as he explains how the company has fully integrated its #PTT application to work… twitter.com/i/web/status/1…

26th October 2020
UrgentComm

.@SierraWireless announced the commercial availability of the AirLink MG90 platform, which they tout as the first m… twitter.com/i/web/status/1…

26th October 2020

Newsletter

Sign up for UrgentComm’s newsletters to receive regular news and information updates about Communications and Technology.

Expert Commentary

Learn from experts about the latest technology in automation, machine-learning, big data and cybersecurity.

Business Media

Find the latest videos and media from the market leaders.

Media Kit and Advertising

Want to reach our digital and print audiences? Learn more here.

DISCOVER MORE FROM INFORMA TECH

  • American City & County
  • IWCE
  • Light Reading
  • IOT World Today
  • Mission Critical Technologies
  • Microwave/RF
  • T&D World
  • TU-Auto

WORKING WITH US

  • About Us
  • Contact Us
  • Events
  • Careers

FOLLOW Urgent Comms ON SOCIAL

  • Privacy
  • CCPA: “Do Not Sell My Data”
  • Cookies Policy
  • Terms
Copyright © 2021 Informa PLC. Informa PLC is registered in England and Wales with company number 8860726 whose registered and Head office is 5 Howick Place, London, SW1P 1WG.
This website uses cookies, including third party ones, to allow for analysis of how people use our website in order to improve your experience and our services. By continuing to use our website, you agree to the use of such cookies. Click here for more information on our Cookie Policy and Privacy Policy.
X