Healthy VoIP Nets�Part III�Quality of Service: Subjective Evaluation and MOS
The quality of real-time communications is ultimately a subjective calculation; nonetheless, we try to measure it.
In our two previous tutorials on VoIP network management, we first looked at the five functional areas of network management, and secondly at some classic architectures that have traditionally been deployed for managing mainframe, WAN and LAN environments.
However, systems that were designed to support legacy environments often require some degree of modification before they can effectively support a converged network. In the case of network management, we must go beyond the processing environment (which might be measuring CPU utilization or disk storage capacity), the WAN environment (which might be concerned with link capacities) or the LAN environment (which might be counting collisions on the Ethernet) and address the key issue that we are dealing with real time communication.
And at the end of that communication link is a humannot a host computerwho likely has some very specific expectations of how their communication service should operate. Let's review those requirements and where they came from.
High end-to-end reliability:
The Bell System provided its customers with over a century of ultra-reliable (the famous "five nines") service prior to its divestiture in 1984. The many regional, national and international wireline carriers continued that tradition, while also implementing many new technologies, such as fiber to the home, which expanded the network's usefulness.
On the wireless side, many of the commercials that you see on television make claims such as 'the network with the fewest dropped calls' which provides further evidence that reliabilitynot just costis still very much on consumers' minds.
Thus, the net manager must understand that the end users may not really know (or care) about the new technology, but they do care about the quality of service, and anything less than the level they have grown accustomed to will be deemed unsatisfactory.
Mitigating packet network complexities:
Circuit switched networks, which were the norm for public telephone networks for decades, solve many of the challenges that humans require for effective communications. These include the sequential (i.e. first infirst out) delivery of the information, and a fixednot variableend-to-end delay.
Packet networks, by their nature, add three key impairments that may degrade this communication. These include: packet delay or latency, which is highly dependent upon the processing characteristics of the subsystems, such as codecs, that are deployed in the VoIP network; packet jitter, which is a variance in the arrival rates of packets coming from the same source; and packet loss, when some of the packets from the original packet stream do not make it to their intended destination. Thus, a network management tool that might have been perfectly adequate for the LAN or WAN may not have the measurement capabilities for these additional parameters that are key for VoIP network performance.
To borrow from the old phrase "beauty is in the eye of the beholder" we could say that "call quality is in the ear of listener." But since my ear processes audio information differently than yours, some measurement standards are required.
In 1994, the International Telecommunications UnionTelecommunication Standardization Sector, or ITU-T, published the E.800 Recommendation, titled Terms and Definitions Related to Quality of Service and Network Performance Including Dependability http://www.itu.int/rec/T-REC-E.800/en. In that document, Quality of Service (QoS) is defined as follows:
"The collective effect of service performance, which determine the degree of satisfaction of a user of the service."
Note that these few words carry a strong message, as the ultimate factor is whether that end user is satisfied with how the network is operating. In other words, all your management theories and systems go out the window if VoIP operation does not meet the performance criteria of the end users.
In 1996, the ITU-T published Recommendation P.800, Methods for Subjective Determination of Transmission Quality, which outlined a number of human-listener-based tests to determine voice call quality http://www.itu.int/rec/T-REC-P.800/en. These include the Conversation-Opinion Test, which is intended to reproduce (in a laboratory) the actual service conditions experienced by telephone customers; a Listening-Opinion Test, which evaluates less restrictive speech material, such as short sentences from the newspaper; and the Quantal-Response Detectability Test, which considers the effects that interfering signals, such as echo and crosstalk, have on the listener.
This standard also gave the industry the phrase Mean Opinion Score, or MOS, a five-point scale that measured the listener's rating of a speech sample, with 5 being Excellent and 1 being Bad. This work was further clarified with the 2006 publication of P.800.1 Mean Opinion Score (MOS) Terminology http://www.itu.int/rec/T-REC-P.800.1/en. In most circles, "Toll Quality" is achieved with a MOS score of 4.0 or greater.
But MOS tests are not the only story. The ITU-T has developed other tests for voice quality measurement, which we will explore in our next tutorial.
Copyright Acknowledgement: © 2007 DigiNet Corporation ®, All Rights Reserved
Mark A. Miller, P.E., is President of DigiNet Corporation®, a Denver-based consulting engineering firm. He is the author of many books on networking technologies, including Voice over IP Technologies, and Internet Technologies Handbook, both published by John Wiley & Sons.