Skip to main content
Springer Nature Link
Log in

Understanding Drivers’ Physiological Responses in Different Road Conditions

  • Conference paper
  • First Online:
HCI in Mobility, Transport, and Automotive Systems (HCII 2022)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13335))

Included in the following conference series:

  • 2062 Accesses

Abstract

Although the driver’s emotion has been studied in the different driving environments (such as city and highway), understanding what eye metrics and facial expressions correspond to specific emotion and behavior based on subjective and biosensor data to study emotion in depth is not well researched in previous studies. Using an eye-integrated human-in-the-loop (HTIL) simulation experiment, we studied how drivers’ facial expressions and ocular measurements relate to emotions. We found that the driving environment could significantly affect drivers’ emotions, which is evident in their facial expressions and eye metrics data. In addition, such outcomes provide knowledge to human-computer-interaction (HCI) practitioners on designing emotion recognition systems in cars to have a robust understanding of the drivers’ emotions and help progress multimodal emotion recognition.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
€34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
EUR 29.95
Price includes VAT (Germany)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
EUR 85.59
Price includes VAT (Germany)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
EUR 106.99
Price includes VAT (Germany)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

') var buybox = document.querySelector("[data-id=id_"+ timestamp +"]").parentNode var buyingOptions = buybox.querySelectorAll(".buying-option") ;[].slice.call(buyingOptions).forEach(initCollapsibles) var buyboxMaxSingleColumnWidth = 480 function initCollapsibles(subscription, index) { var toggle = subscription.querySelector(".buying-option-price") subscription.classList.remove("expanded") var form = subscription.querySelector(".buying-option-form") var priceInfo = subscription.querySelector(".price-info") var buyingOption = toggle.parentElement if (toggle && form && priceInfo) { toggle.setAttribute("role", "button") toggle.setAttribute("tabindex", "0") toggle.addEventListener("click", function (event) { var expandedBuyingOptions = buybox.querySelectorAll(".buying-option.expanded") var buyboxWidth = buybox.offsetWidth ;[].slice.call(expandedBuyingOptions).forEach(function(option) { if (buyboxWidth buyboxMaxSingleColumnWidth) { toggle.click() } else { if (index === 0) { toggle.click() } else { toggle.setAttribute("aria-expanded", "false") form.hidden = "hidden" priceInfo.hidden = "hidden" } } }) } initialStateOpen() if (window.buyboxInitialised) return window.buyboxInitialised = true initKeyControls() })()

Institutional subscriptions

Similar content being viewed by others

References

  1. Tavakoli, A., Balali, V., Heydarian, A.: A Multimodal Approach for Monitoring Driving Behavior and Emotions (2020)

    Google Scholar 

  2. Kleinginna, P.R., Kleinginna, A.M.: A categorized list of motivation definitions, with a suggestion for a consensual definition. Motiv. Emot. 5(3), 263–291 (1981). https://doi.org/10.1007/BF00993889

    Article  Google Scholar 

  3. Pittermann, J., Pittermann, A., Minker, W.: Emotion recognition and adaptation in spoken dialogue systems. Int. J. Speech Technol. 13(1), 49–60 (2010). https://doi.org/10.1007/s10772-010-9068-y

    Article  Google Scholar 

  4. Leahu, L., Schwenk, S., Sengers, P.: Subjective objectivity: negotiating emotional meaning. In: Proceedings of the 7th ACM Conference on Designing Interactive Systems, pp. 425–434 (2008)

    Google Scholar 

  5. Picard, R.W., Klein, J.: Computers that recognise and respond to user emotion: theoretical and practical implications. Interact. Comput. 14(2), 141–169 (2002)

    Article  Google Scholar 

  6. Cohn, J.F.: Foundations of human computing: facial expression and emotion. In: Proceedings of the 8th International Conference on Multimodal Interfaces, pp. 233–238 (2006)

    Google Scholar 

  7. Martis, J.E.: Effective emotion recognition of expressions from facial features. 5(06), 4–7 (2017)

    Google Scholar 

  8. Geiger, A., Brandenburg, E., Stark, R.: Natural virtual reality user interface to define assembly sequences for digital human models. Appl. Syst. Innov. 3(1), 15 (2020)

    Article  Google Scholar 

  9. Sedenberg, E., Wong, R., Chuang, J.: A window into the soul: biosensing in public. arXiv preprint arXiv:1702.04235 (2017)

  10. Liu, L., et al.: Deep learning for generic object detection: a survey. arXiv 2018. arXiv preprint arXiv:1809.02165 (2019)

  11. Martinez-Conde, S., Macknik, S.L., Hubel, D.H.: The role of fixational eye movements in visual perception. Nat. Rev. Neurosci. 5(3), 229–240 (2004)

    Article  Google Scholar 

  12. Baujon, J., Basset, M., Gissinger, G.L.: Visual behaviour analysis and driver cognitive model. In: Proceedings of the 3rd IFAC Workshop on Advances in Automotive Control, Karlsruhe, Germany, pp. 47–52 (2001)

    Google Scholar 

  13. Brackstone, M., Waterson, B.: Are we looking where we are going? An exploratory examination of eye movement in high-speed driving. In: Proceedings of the 83rd Transportation Research Board Annual Meeting, vol. 2, p. 602 (2004)

    Google Scholar 

  14. Yan, Y., Yuan, H., Wang, X., Xu, T., Liu, H.: Study on driver’s fixation variation at entrance and inside sections of tunnel on highway. Adv. Mech. Eng. 7(1), 273427 (2015)

    Article  Google Scholar 

  15. Nadal, M., Munar, E., Marty, G., Cela-Conde, C.J.: Visual complexity and beauty appreciation: explaining the divergence of results. Empirical Stud. Arts 28(2), 173–191 (2010)

    Article  Google Scholar 

  16. Chan, M., Singhal, A.: Emotion matters: Implications for distracted driving. Saf. Sci. 72, 302–309 (2015)

    Article  Google Scholar 

  17. Zhang, W., Zhang, X., Feng, Z., Liu, J., Zhou, M., Wang, K.: The fitness-to-drive of shift-work taxi drivers with obstructive sleep apnea: an investigation of self-reported driver behavior and skill. Transp. Res. Part F: Traffic Psychol. Behav. 59, 545–554 (2018)

    Article  Google Scholar 

  18. Eherenfreund-Hager, A., Taubman-Ben-Ari, O., Toledo, T., Farah, H.: The effect of positive and negative emotions on young drivers a simulator study. Transp. Res. Part F: Traffic Psychol. Behav. 49, 236–243 (2017)

    Google Scholar 

  19. Du, N., et al.: Examining the effects of emotional valence and arousal on takeover performance in conditionally automated driving. Transp. Res. Part C: Emerg. Technol. 112, 78–87 (2020)

    Article  Google Scholar 

  20. Hedlund, J., Simpson, H.M., Mayhew, D.R.: Summary of proceedings and recommendations. In: International Conference on Distracted Driving. The Traffic Injury Research Foundation, The Canadian Automobile Association, Ottawa (2006)

    Google Scholar 

  21. Nyström, M., Holmqvist, K.: An adaptive algorithm for fixation, saccade, and glissade detection in eyetracking data. Behav. Res. Methods 42(1), 188–204 (2010). https://doi.org/10.3758/BRM.42.1.188

    Article  Google Scholar 

  22. Ekman, P., Friesen, W.V.: Facial Action Coding System: Investigator’s Guide. Consulting Psychologists Press, Palo Alto (1978)

    Google Scholar 

  23. Steyer, R., Schwenkmezger, P., Notz, P., Eid, M.: MDMQ questionnaire (English version of MDBF). Jena: Friedrich-Schiller-Universität Jena, Institut für Psychologie, Lehrstuhl für Methodenlehre und Evaluationsforschung (2014). https://www.metheval.uni-jena.de/mdbf.php. Accessed 4 Apr 2016

  24. Meinlschmidt, G., et al.: Smartphone-based psychotherapeutic micro-interventions to improve mood in a real-world setting. Front. Psychol. 7, 1112 (2016)

    Article  Google Scholar 

  25. Hinz, A., Daig, I., Petrowski, K., Brähler, E.: Die stimmung in der deutschen bevölkerung: referenzwerte für den mehrdimensionalen befindlichkeitsfragebogen MDBF. PPmP-Psychother. Psychosom. Med. Psychol. 62(02), 52–57 (2012)

    Article  Google Scholar 

  26. Park, J., Abdel-Aty, M., Yina, W., Mattei, I.: Enhancing in-vehicle driving assistance information under connected vehicle environment. IEEE Trans. Intell. Transp. Syst. 20(9), 3558–3567 (2018)

    Article  Google Scholar 

  27. Lynch, B.K.: Designing qualitative research by catherine marshall an Gretchen B. Rossman. Issues Appl. Linguist. 1(2), 1–9 (1990)

    Article  Google Scholar 

  28. Schutte, N.S., Malouff, J.M., Thorsteinsson, E.B., Bhullar, N., Rooke, S.E.: A meta-analytic investigation of the relationship between emotional intelligence and health. Pers. Individ. Differ. 42(6), 921–933 (2007)

    Article  Google Scholar 

  29. Guarnera, M., Hichy, Z., Cascio, M.I., Carrubba, S.: Facial expressions and ability to recognize emotions from eyes or mouth in children. Eur. J. Psychol. 11(2), 183 (2015)

    Article  Google Scholar 

  30. Li, J., Jin, K., Zhou, D., Kubota, N., Zhaojie, J.: Attention mechanism-based CNN for facial expression recognition. Neurocomputing 411, 340–350 (2020)

    Article  Google Scholar 

  31. Hassib, M., Braun, M., Pfleging, B., Alt, F.: Detecting and influencing driver emotions using psycho-physiological sensors and ambient light. In: Lamas, D., Loizides, F., Nacke, L., Petrie, H., Winckler, M., Zaphiris, P. (eds.) Human-Computer Interaction – INTERACT 2019, vol. 11746, pp. 721–742. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-29381-9_43

    Chapter  Google Scholar 

  32. Mesken, J.: Determinants and consequences of drivers’ emotions. Stichting Wetenschappelijk Onderzoek Verkeersveiligheid SWOV (2006)

    Google Scholar 

  33. Remington, R.W.: Attention and saccadic eye movements. J. Exp. Psychol.: Hum. Percept. Perform. 6(4), 726 (1980)

    Google Scholar 

  34. Murphy-Chutorian, E., Trivedi, M.M.: Head pose estimation and augmented reality tracking: an integrated system and evaluation for monitoring driver awareness. IEEE Trans. Intell. Transp. Syst. 11(2), 300–311 (2010)

    Article  Google Scholar 

  35. Sahayadhas, A., Sundaraj, K., Murugappan, M.: Detecting driver drowsiness based on sensors: a review. Sensors 12(12), 16937–16953 (2012)

    Article  Google Scholar 

  36. Mashko, A.: Subjective methods for assessment of driver drowsiness. Acta Polytech. CTU Proc. 12, 64–67 (2017)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Industrial and Manufacturing Systems Engineering Department, University of Missouri, Columbia, MO, USA

    Sara Mostowfi & Jung Hyup Kim

Authors
  1. Sara Mostowfi
    View author publications

    Search author on:PubMed Google Scholar

  2. Jung Hyup Kim
    View author publications

    Search author on:PubMed Google Scholar

Corresponding author

Correspondence to Jung Hyup Kim .

Editor information

Editors and Affiliations

  1. Technische Universität Ilmenau, Ilmenau, Germany

    Heidi Krömker

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Mostowfi, S., Kim, J.H. (2022). Understanding Drivers’ Physiological Responses in Different Road Conditions. In: Krömker, H. (eds) HCI in Mobility, Transport, and Automotive Systems. HCII 2022. Lecture Notes in Computer Science, vol 13335. Springer, Cham. https://doi.org/10.1007/978-3-031-04987-3_15

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-04987-3_15

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-04986-6

  • Online ISBN: 978-3-031-04987-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics