Xiao Zhang
Skip Abstract Section
Abstract
Non-intrusive and sensitive measurement of users' stress is very crucial for computers to dynamically understand and respond to users' mental status while users are naturally interacting with them, such as context-aware reminding, smart assistant, health monitoring etc. Compared to others, physiological measures are known as more sensitive and reliable ones. However, most of the current physiological methods need explicit and obtrusive sensors that deter natural human-computer interactions. In this study, we propose a photoplethysmogram-based mental stress measuring method through infrared touchscreen, which, from the best of our knowledge, is the first to integrate with the touch modality to enable natural, sensitive and reliable measurement. We designed and conducted two user experiments with touch-and-hold mode and tap mode respectively. By using person-independent classifiers to handle the photoplethysmographic parameters, the results reached a 97% and 87% average stress recognition accuracy for static test and interaction test respectively.
- John Allen. 2007. Photoplethysmography and its application in clinical physiological measurement. Physiological Measurement 28, 3 (2007), R1.Google Scholar
Cross Ref
- Jorn Bakker, Mykola Pechenizkiy, and Natalia Sidorova. 2011. What's Your Current Stress Level? Detection of Stress Patterns from GSR Sensor Data. In Proceedings of the 2011 IEEE 11th International Conference on Data Mining Workshops (ICDMW '11). IEEE Computer Society, Washington, DC, USA, 573--580. Google ScholarDigital Library
- M Biondi and A Picardi. 1999. Psychological stress and neuroendocrine function in humans: the last two decades of research. Psychotherapy 8 Psychosomatics 68, 3 (1999), 114--150.Google Scholar
- Wolfram Boucsein. 1992. Electrodermal Activity. Springer, Boston, US. 316--317 pages.Google Scholar
- F Bousefsaf, C Maaoui, and A Pruski. 2014. Remote detection of mental workload changes using cardiac parameters assessed with a low-cost webcam. Computers in Biology 8 Medicine 53, C (2014), 154. Google ScholarDigital Library
- A. V. J. Challoner. 1979. Photoelectric plethysmography for estimating cutaneous blood flow. Non-invasive physiological measurements 1 (1979), 125--151.Google Scholar
- K. Chariot, J. Cornolo, J. V. Brugniaux, J. P. Richalet, and A. Pichon. 2009. Interchangeability between heart rate and photoplethysmography variabilities during sympathetic stimulations. PHYSIOLOGICAL MEASUREMENT 30, 12 (DEC 2009), 1357--1369.Google Scholar
- Hsiu Sen Chiang. 2015. ECG-based Mental Stress Assessment Using Fuzzy Computing and Associative Petri Net. Journal of Medical 8 Biological Engineering 35, 6 (2015), 833--844.Google ScholarCross Ref
- Hiroshi Chigira, Atsuhiko Maeda, and Minoru Kobayashi. 2011. Area-based Photo-plethysmographic Sensing Method for the Surfaces of Handheld Devices. In Proceedings of the 24th Annual ACM Symposium on User Interface Software and Technology (UIST '11). ACM, New York, NY, USA, 499--508. Google ScholarDigital Library
- Yoorim Choi, Minjung Kim, and Chungyoon Chun. 2014. Measurement of occupants' stress based on electroencephalograms (EEG) in twelve combined environments. Building 8 Environment 88 (2014), 65--72.Google Scholar
- C. P. Chua and C. Heneghan. 2006. Continuous blood pressure monitoring using ECG and finger photoplethysmogram. In 2006 28TH ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY, VOLS 1-15. IEEE Engn Med 8 Biol Soc, IEEE, 345 E 47TH ST, NEW YORK, NY 10017 USA, 4128+. 28th Annual International Conference of the IEEE-Engineering-in-Medicine-and-Biology-Society, New York, NY, AUG 30-SEP 03, 2006.Google Scholar
- S Cohen, D Janickideverts, and G. E. Miller. 2007. Psychological stress and disease. Jama 298, 14 (2007), 1685.Google ScholarCross Ref
- Sheldon Cohen, Ronald C. Kessler, and Lynn Underwood Gordon. 1995. Strategies for measuring stress in studies of psychiatric and physical disorders. Oxford University Press, New York, NY, US, 3--26.Google Scholar
- D. F. Dinges, R. L. Rider, J Dorrian, E. L. Mcglinchey, N. L. Rogers, Z Cizman, S. K. Goldenstein, C Vogler, S Venkataraman, and D. N. Metaxas. 2005. Optical computer recognition of facial expressions associated with stress induced by performance demands. Aviation Space 8 Environmental Medicine 76, 6 Suppl (2005), B172.Google Scholar
- Mohamed Elgendi. 2012. Standard Terminologies for Photoplethysmogram Signals. Curr Cardiol Rev 8, 3 (2012), 215--219.Google ScholarCross Ref
- H. S. Goldstein and R Edelberg. 1997. A plethysmographic method for demonstrating the response specificity of the oral vascular bed. Psychophysiology 34, 1 (1997), 124.Google ScholarCross Ref
- John H. Hansen and Sanjay Patil. 2007. Speech Under Stress: Analysis, Modeling and Recognition. In Speaker Classification I, Christian Müller (Ed.). Springer-Verlag, Berlin, Heidelberg, Chapter Speech Under Stress: Analysis, Modeling and Recognition, 108--137. Google ScholarDigital Library
- J Hashimoto, K Chonan, Y Aoki, T Nishimura, T Ohkubo, A Hozawa, M Suzuki, M Matsubara, M Michimata, and T Araki. 2002. Pulse wave velocity and the second derivative of the finger photoplethysmogram in treated hypertensive patients: their relationship and associating factors. Journal of Hypertension 20, 12 (2002), 2415--22.Google ScholarCross Ref
- J. Healey and R. Picard. 2000. SmartCar: detecting driver stress. In Proceedings 15th International Conference on Pattern Recognition. ICPR-2000, Vol. 4. 218--221 vol.4. Google ScholarDigital Library
- J. A. Healey and R. W. Picard. 2005. Detecting stress during real-world driving tasks using physiological sensors. IEEE Transactions on Intelligent Transportation Systems 6, 2 (2005), 156--166. Google ScholarDigital Library
- Matthew Hotopf, Paul J. Rosch, and Carole Hart. 2002. Psychological Stress And Cardiovascular Disease. Bmj 325, 7359 (2002), 1237--1246.Google Scholar
- R. Khosrowabadi, C. Quek, K. K. Ang, S. W. Tung, and M. Heijnen. 2011. A Brain-Computer Interface for classifying EEG correlates of chronic mental stress. In The 2011 International Joint Conference on Neural Networks. 757--762.Google Scholar
- E LabbÃl, N Schmidt, J Babin, and M Pharr. 2007. Coping with stress: the effectiveness of different types of music. Applied Psychophysiology 8 Biofeedback 32, 3--4 (2007), 163--168.Google Scholar
- S. Lee, H. Shin, and C. Hahm. 2016. Effective PPG sensor placement for reflected red and green light, and infrared wristband-type photoplethysmography. In 2016 18th International Conference on Advanced Communication Technology (ICACT). 1--1.Google Scholar
- Iulia Lefter, Gertjan J. Burghouts, and LÃl'on J. M. Rothkrantz. 2016. Recognizing Stress Using Semantics and Modulation of Speech and Gestures. IEEE Transactions on Affective Computing 7, 2 (2016), 162--175.Google ScholarCross Ref
- Wenhui Liao, Weihong Zhang, Zhiwei Zhu, and Qiang Ji. 2005. A Real-Time Human Stress Monitoring System Using Dynamic Bayesian Network. In 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Workshops. 70--70. Google ScholarDigital Library
- S. Lu, H. Zhao, K. Ju, K Shin, M Lee, K Shelley, and K. H. Chon. 2008. Can photoplethysmography variability serve as an alternative approach to obtain heart rate variability information? Journal of Clinical Monitoring 8 Computing 22, 1 (2008), 23--29.Google ScholarCross Ref
- Yongqiang Lyu, Xiaomin Luo, Jun Zhou, Chun Yu, Congcong Miao, Tong Wang, Yuanchun Shi, and Ken-ichi Kameyama. 2015. Measuring Photoplethysmogram-Based Stress-Induced Vascular Response Index to Assess Cognitive Load and Stress. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI '15). ACM, New York, NY, USA, 857--866. Google ScholarDigital Library
- Yuka Maeda, Masaki Sekine, and Toshiyo Tamura. 2011. The Advantages of Wearable Green Reflected Photoplethysmography. Journal of Medical Systems 35, 5 (2011), 829--834. Google ScholarDigital Library
- D. McDuff, S. Gontarek, and R. Picard. 2014. Remote measurement of cognitive stress via heart rate variability. In 2014 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society. 2957--2960.Google Scholar
- Daniel J. McDuff, Javier Hernandez, Sarah Gontarek, and Rosalind W. Picard. 2016. COGCAM: Contact-free Measurement of Cognitive Stress During Computer Tasks with a Digital Camera. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI '16). ACM, New York, NY, USA, 4000--4004. Google ScholarDigital Library
- J. P. Niskanen, M. P. Tarvainen, P. O. Rantaaho, and P. A. Karjalainen. 2004. Software for advanced HRV analysis. Computer Methods 8 Programs in Biomedicine 76, 1 (2004), 73.Google Scholar
- T. G. Pickering, R. B. Devereux, G. D. James, W Gerin, P Landsbergis, P. L. Schnall, and J. E. Schwartz. 1996. Environmental influences on blood pressure and the role of job strain. Journal of Hypertension Supplement Official Journal of the International Society of Hypertension 14, 5 (1996), 179--85.Google Scholar
- Ming Zher Poh, Daniel J. Mcduff, and Rosalind W. Picard. 2011. Advancements in Noncontact, Multiparameter Physiological Measurements Using a Webcam. IEEE transactions on bio-medical engineering 58, 1 (2011), 7.Google Scholar
- Acharya U Rajendra, Joseph K Paul, N Kannathal, C. M. Lim, and J. S. Suri. 2006. Heart rate variability: a review. Medical 8 Biological Engineering 8 Computing 44, 12 (2006), 1031--1051.Google Scholar
- Rodrigo Siqueira Reis, Adriano Akira Ferreira Hino, and Ciro RomÃl'lio Rodriguez AÃśez. 2010. Perceived Stress Scale. Journal of Health Psychology 15, 1 (2010), 107--14.Google ScholarCross Ref
- K. s. Yoo and W. h. Lee. 2011. Mental stress assessment based on pulse photoplethysmography. In 2011 IEEE 15th International Symposium on Consumer Electronics (ISCE). IEEE, Singapore, Singapore, 323--326.Google ScholarCross Ref
- S. C. Segerstrom and G. E. Miller. 2004. Psychological stress and the human immune system: a meta-analytic study of 30 years of inquiry. Psychological Bulletin 130, 4 (2004), 601--630.Google ScholarCross Ref
- Cornelia Setz, Bert Arnrich, Johannes Schumm, Roberto La Marca, and Ulrike Ehlert. 2010. Discriminating stress from cognitive load using a wearable EDA device. IEEE Transactions on Information Technology in Biomedicine A Publication of the IEEE Engineering in Medicine 8 Biology Society 14, 2 (2010), 410--417. Google ScholarDigital Library
- Nandita Sharma and Tom Gedeon. 2012. Objective measures, sensors and computational techniques for stress recognition and classification. Computer Methods 8 Programs in Biomedicine 108, 3 (2012), 1287. Google ScholarDigital Library
- Harry A Silber, Jeffrey C Trost, Peter V Johnston, W Lowell Maughan, Nae-Yuh Wang, Edward K Kasper, Thomas R Aversano, and David E Bush. 2012. Finger photoplethysmography during the Valsalva maneuver reflects left ventricular filling pressure. American journal of physiology. Heart and circulatory physiology 302, 10 (May 2012), H2043--7.Google Scholar
- X. F. Teng and Y. T. Zhang. 2007. Theoretical study on the effect of sensor contact force on pulse transit time. IEEE Transactions on Biomedical Engineering 54, 8 (2007), 1490.Google ScholarCross Ref
- J.H.M. Tulen, P. Moleman, H.G. van Steenis, and F. Boomsma. 1989. Characterization of stress reactions to the Stroop Color Word Test. Pharmacology Biochemistry and Behavior 32, 1 (1989), 9--15.Google ScholarCross Ref
- Ingun Ulstein, Torgeir Bruun Wyller, and Knut Engedal. 2007. High score on the Relative Stress Scale, a marker of possible psychiatric disorder in family carers of patients with dementia. International Journal of Geriatric Psychiatry 22, 3 (2007), 195--202.Google ScholarCross Ref
- Jing Zhai, A. B. Barreto, C. Chin, and Chao Li. 2005. Realization of stress detection using psychophysiological signals for improvement of human-computer interactions. In Proceedings. IEEE SoutheastCon, 2005. 415--420.Google Scholar
- Xiao Zhang, Yongqiang Lyu, Xin Hu, Ziyue Hu, Yuanchun Shi, and Hao Yin. 2018. Evaluating Photoplethysmogram as a Real-time Cognitive Load Assessment during Game Playing. International Journal of Human Computer-Interaction (2018). To appear.Google Scholar
Index Terms
- Touch Sense: Touch Screen Based Mental Stress Sense
Recommendations
To touch or not to touch?: comparing 2D touch and 3D mid-air interaction on stereoscopic tabletop surfaces
SUI '13: Proceedings of the 1st symposium on Spatial user interactionRecent developments in touch and display technologies have laid the groundwork to combine touch-sensitive display systems with stereoscopic three-dimensional (3D) display. Although this combination provides a compelling user experience, interaction with ...
Pen + touch = new tools
UIST '10: Proceedings of the 23nd annual ACM symposium on User interface software and technologyWe describe techniques for direct pen+touch input. We observe people's manual behaviors with physical paper and notebooks. These serve as the foundation for a prototype Microsoft Surface application, centered on note-taking and scrapbooking of ...
Air+touch: interweaving touch & in-air gestures
UIST '14: Proceedings of the 27th annual ACM symposium on User interface software and technologyWe present Air+Touch, a new class of interactions that interweave touch events with in-air gestures, offering a unified input modality with expressiveness greater than each input modality alone. We demonstrate how air and touch are highly complementary: ...
Comments