Han Hee Song
ABSTRACT
In large-scale IPTV systems, it is essential to maintain high service quality while providing a wider variety of service features than typical traditional TV. Thus service quality assessment systems are of paramount importance as they monitor the user-perceived service quality and alert when issues occurs. For IPTV systems, however, there is no simple metric to represent user-perceived service quality and Quality of Experience (QoE). Moreover, there is only limited user feedback, often in the form of noisy and delayed customer calls. Therefore, we aim to approximate the QoE through a selected set of performance indicators in a proactive (i.e., detect issues before customers reports to call centers) and scalable fashion.
In this paper, we present a service quality assessment framework, Q-score, which accurately learns a small set of performance indicators most relevant to user-perceived service quality, and proactively infers service quality in a single score. We evaluate Q-score using network data collected from a commercial IPTV service provider and show that Q-score is able to predict 60% of the service problems that are reported by customers with 0.1% false positives. Through Q-score, we have (i) gained insight into various types of service problems causing user dissatisfaction, including why users tend to react promptly to sound issues while late to video issues; (ii) identified and quantified the opportunity to proactively detect the service quality degradation of individual customers before severe performance impact occurs; and (iii) observed possibility to allocate customer care workforce to potentially troubling service areas before issues break out.
- Methods for subjective determination of transmission quality. ITU-T Rec. P.800, 1998.Google Scholar
- One-way transmission time. ITU-T Rec. G.114, 2003.Google Scholar
- Objective perceptual multimedia video quality measurement in the presence of a full reference. ITU-T Rec. J.247, 2008.Google Scholar
- Perceptual audiovisual quality measurement techniques for multimedia services over digital cable television networks in the presence of a reduced bandwidth reference. ITU-T Rec. J.246, 2008.Google Scholar
- R. Abreu, A. G. 0002, P. Zoeteweij, and A. J. C. van Gemund. Automatic software fault localization using generic program invariants. In SAC, 2008. Google ScholarDigital Library
- R. Abreu, P. Zoeteweij, and A. J. C. van Gemund. Spectrum-based multiple fault localization. In ASE, 2009. Google ScholarDigital Library
- P. Bahl, R. Chandra, A. Greenberg, S. Kandula, D. A. Maltz, and M. Zhang. Towards highly reliable enterprise network services via inference of multi-level dependencies. In Sigcomm, 2007. Google ScholarDigital Library
- L. Breslau, D. Estrin, K. Fall, S. Floyd, J. Heidemann, A. Helmy, P. Huang, S. McCanne, K. Varadhan, Y. Xu, and H. Yu. Advances in network simulation. IEEE Computer, 33(5), 2000. Google ScholarDigital Library
- S. Chatterjee and A. S. Hadi. Influential observations, high leverage points, and outliers in linear regression. Statistical Science, Vol. 1:379--416, 1986.Google ScholarCross Ref
- I. Cohen, J. S. Chase, M. Goldszmidt, T. Kelly, and J. Symons. Correlating instrumentation data to system states: A building block for automated diagnosis and control. In OSDI, 2004. Google ScholarDigital Library
- A. E. Hoerl and R. W. Kennard. Ridge regression: Biased estimation for nonorthogonal problems. Technometrics, Vol. 12, No. 1:55--67, 1970.Google ScholarCross Ref
- G. Holzmann. Design and Validation of Computer Protocols. Prentice-Hall, 1991. Google ScholarDigital Library
- Y. Jin, N. G. Duffield, A. Gerber, P. Haffner, S. Sen, and Z.-L. Zhang. Nevermind, the problem is already fixed: proactively detecting and troubleshooting customer dsl problems. In CoNEXT, page 7, 2010. Google ScholarDigital Library
- S. Kandula, D. Katabi, and J.-P. Vasseur. Shrink: A tool for failure diagnosis in IP networks. In MineNet, 2005. Google ScholarDigital Library
- S. Kandula, R. Mahajan, P. Verkaik, S. Agarwal, J. Padhye, and P. Bahl. Detailed diagnosis in enterprise networks. In ACM SIGCOMM, 2009. Google ScholarDigital Library
- R. R. Kompella, J. Yates, A. Greenberg, and A. C. Snoeren. IP fault localization via risk modeling. In NSDI, 2005. Google ScholarDigital Library
- T.-L. Lin, S. Kanumuri, Y. Zhi, D. Poole, P. Cosman, and A. R. Reibman. A versatile model for packet loss visibility and its application in packet prioritization. IEEE Transactions on Image Processing, to appear, 2010. Google ScholarDigital Library
- A. Mahimkar, Z. Ge, A. Shaikh, J. Wang, J. Yates, Y. Zhang, and Q. Zhao. Towards automated performance diagnosis in a large IPTV network. In ACM SIGCOMM, 2009. Google ScholarDigital Library
- A. Mahimkar, H. H. Song, Z. Ge, A. Shaikh, J. Wang, J. Yates, Y. Zhang, and J. Emmons. Detecting the performance impact of upgrades in large operational networks. In ACM SIGCOMM, 2010. Google ScholarDigital Library
- A. Mahimkar, J. Yates, Y. Zhang, A. Shaikh, J. Wang, Z. Ge, and C. T. Ee. Troubleshooting chronic conditions in large IP networks. In ACM CoNEXT, 2008. Google ScholarDigital Library
- J. S. Moore and M. Kaufmann. Some key research problems in automated theorem proving for hardware and software verification. In RACSAM, 2004.Google Scholar
- Perceptual evaluation of video quality. 2011. http://www.pevq.org.Google Scholar
- M. Pinson and S. Wolf. Comparing subjective video quality testing methodologies. In SPIE Video Communications and Image Processing Conference, pages 8--11, 2003.Google Scholar
- T. Qiu, J. Feng, Z. Ge, J. Wang, J. Xu, and J. Yates. Listen to me if you can: Tracking user experience of mobile network on social media. In IMC, 2010. Google ScholarDigital Library
- K. Shen, C. Stewart, C. Li, and X. Li. Reference-driven performance anomaly identification. In SIGMETRICS, 2009. Google ScholarDigital Library
- M. Steinder and A. Sethi. Increasing robustness of fault localization through analysis of lost, spurious, and positive symptoms. In Infocom, 2002.Google ScholarCross Ref
- S. Tao, J. Apostolopoulos, and R. Guérin. Real-time monitoring of video quality in ip networks. In Proceedings of the international workshop on Network and operating systems support for digital audio and video, NOSSDAV'05, pages 129--134, New York, NY, USA, 2005. ACM. Google ScholarDigital Library
- S. Tao, J. Apostolopoulos, and R. Guérin. Real-time monitoring of video quality in ip networks. IEEE/ACM Trans. Netw., 16:1052--1065, October 2008. Google ScholarDigital Library
- M. Tariq, A. Zeitoun, V. Valancius, N. Feamster, and M. Ammar. Answering what-if deployment and configuration questions with WISE. In SIGCOMM, 2008. Google ScholarDigital Library
Index Terms
- Q-score: proactive service quality assessment in a large IPTV system
Recommendations
QoE assessment model for multimedia streaming services using QoS parameters
This paper proposes a video QoE (Quality of Experience) assessment model which can assess video quality of experience with only QoS (Quality of Service) parameters and their relative importance at network layer. Since network or service providers can ...
SQUAD: a spectrum-based quality adaptation for dynamic adaptive streaming over HTTP
MMSys '16: Proceedings of the 7th International Conference on Multimedia SystemsThe application-layer based control loops of dynamic adaptive streaming over HTTP (DASH) make video bitrate selection a complex problem. In this work, we review and present new insights into the challenges of DASH rate adaptation. We identify several ...
A QoE Management System for Ubiquitous IPTV Devices
UBICOMM '09: Proceedings of the 2009 Third International Conference on Mobile Ubiquitous Computing, Systems, Services and TechnologiesThe use of the triple-play services is growing up very quick in the recent years. This has come by the hand of the higher bandwidth offered in broadband access by the service providers. IPTV is the service that has been experienced more request, ...
Comments