Digital Regulation Platform
«
»
«
»

The relation between quality of service and quality of experience

25.08.2020

Factors influencing the quality of experience

Quality of experience (QoE) is now taken to be “the degree of delight or annoyance of the user of an application or service,” following ITU-T Recommendation P.10/G.100 (ITU-T 2017). Conventionally it is regarded as being determined by “influencing factors” of the following kinds, which are discussed in more detail in Qualinet White Paper (Qualinet 2013):

The range of factors here is wide enough to suggest that QoE would include accessibility, availability, and affordability. These are human-related, system-related, and context-related factors (respectively) and are also the normal concerns of universal service. Yet, in practice work on QoE does not deal much with them: it deals more with new services for the affluent than with old services for the disadvantaged.

Nonetheless a service that is unavailable can hardly be said to be offer its users much “delight.” Indeed coverage and reliability are relevant to quality of service (QoS), let alone QoE; here QoS is “the totality of characteristics of a telecommunications service that bear on its ability to satisfy stated and implied needs of the user of the service” (ITU-T 2017). Coverage is often assessed fairly easily, through checking the presence of cables or the strengths of radio signals. As users are interested in calls, messages, data streams, and data files, not signal strengths, it should be defined in terms of these interests of users and included in QoS. Likewise, reliability can be included in QoS.

Some other topics might appear to be excluded from QoE and QoS because they are not mentioned among the influencing factors. Among these topics are the correctness of applications (whether systems would produce the intended results in the absence of transmission errors) and the security of systems (whether systems would provide the results to, and only to, the intended people). QoE is perhaps concerned with these topics only to the extent that user feelings might be affected by knowledge, beliefs, or suspicions about them. For instance, for online language translation systems the speed of translation might influence the QoE but the accuracy of translation would not. Yet there are exceptions to these apparent exclusions from QoE: money transfer services would not offer acceptable experiences if they debited accounts wrongly or admitted unauthorized access to accounts. Here, as in some other areas, regulation of the application is closer to user concerns than regulation of the network.

Assessments of the quality of experience

QoE assessments can be subjective or objective. Subjective assessments determine quality by asking human assessors (but not by asking all end users at the point of use). Objective assessments estimate quality from values obtained by making measurements of the systems being assessed.

Subjective assessments require human assessors, each of whom rates each experience, usually on a five-point scale; the average (or “mean”) of the ratings is then taken to be the mean opinion score (MOS). The assessments might be performed in the laboratory or in the field, where they have the following advantages and disadvantages:

Even the fullest possible subjective assessments of systems ignore the feelings and environments of people in places and at times other than those of the assessors.

Objective assessments do not require human assessors, so they are generally cheaper and faster than subjective ones. They usually involve making measurements of the systems being assessed, in order to extract values that can be combined in mathematical models to provide mean opinion scores. Their accuracy depends on whether the models are realistic enough to determine perceived quality much as human assessors would on average. Typically, the models are constructed on the basis of theories of perception, empirical studies of influencing factors, and machine learning from subjective assessments. They do not take account of all system-related influencing factors, let alone all human-related and context-related influencing factors. Such models are now available in ITU-T recommendations for voice, video and multimedia applications under various conditions, such as adaptive streaming and progressive downloading to televisions and smartphones of multimedia by over-the-top (OTT) and managed content providers (ITU-T 2020).

Objective assessments can be performed in the laboratory or in the field, but they can also be performed in the network. They have the following advantages and disadvantages:

Objective assessments are also possible without making measurements of the systems being assessed. Instead values, based on prior knowledge, are assumed and combined in a model to provide mean opinion scores. This technique is useful when networks are being planned.

QoS assessments often consider just the influencing factors that can be affected by network operators and content providers. Then, in terms of ITU-T Recommendation G.1000 (ITU-T 2001):

Measurements can of course be made without then being combined in a model. In fact, this is typical of QoS measurements currently. For instance, the delay and the packet loss ratio in an IP network are typically quoted separately, though they could be combined with assumptions about some influencing factors to provide an MOS for phone call conversations according to ITU-T Recommendation G.107 (ITU-T 2015). Quoting them separately lets operators concentrate the improvements in network performance on one of them and can be more useful if they affect different services to different extents.

Objective assessments of QoS should be calibrated against subjective assessments by large numbers of users. These subjective assessments might be regarded as measuring QoE. However, they are often performed in the laboratory with users who are compensated in some way for their time; they are then not even representative of the environments of users, let alone of the feelings of one particular user.

Much work on QoE standards development relies on assessments in the laboratory. It might then result in products, composed of hardware and software for performing objective assessments that match subjective assessments on average. At that point, the products and the associated standards can become of interest to operators, who might install them for performing assessments in the field or in the network.

References

ITU-T. 2001. Communications Quality of Service: A Framework and Definitions. ITU-T Recommendation G.1000. https://www.itu.int/ITU-T/recommendations/rec.aspx?id=5597&lang=en.

ITU-T. 2015. The E-model: a Computational Model for Use in Transmission Planning. ITU-T Recommendation G.107. https://www.itu.int/ITU-T/recommendations/rec.aspx?id=12505&lang=en.

ITU-T. 2017. Vocabulary for Performance, Quality of Service and Quality of Experience. ITU-T Recommendation P.10/G.100. https://www.itu.int/ITU-T/recommendations/rec.aspx?id=13408&lang=en.

Qualinet. 2013. Qualinet White Paper on Definitions of Quality of Experience (2012). European Network on Quality of Experience in Multimedia Systems and Services (COST Action IC 1003). Version 1.2. March 12, 2013. http://www.qualinet.eu/images/stories/QoE_whitepaper_v1.2.pdf.

Last updated on: 28.08.2020