Abstract
We developed a new method to link designer emotions with corresponding designer activities while using computer-aided design (cad) software. Our method employs automated facial emotion detection software and cursor tracking. We applied this method via an experiment with nine participants, each working with the same synchronously collaborative cad platform, and assigned a series of cad tasks in one of two distinct working styles: single participants working by themselves and paired participants working together. We analyzed and compared trends in emotion for these two working styles. Pairs, on average per person, experienced higher levels of emotion (measured as joy, sadness, anger, contempt, fear, and surprise) than individuals. We linked occurrences of each emotional response to their antecedent activities in the cad environment (navigating the model tree, sketching in the graphics area, making selections in the feature menu, and communicating using the chat window). Using a logistic regression analysis, we revealed statistically significant trends linking emotions and cad events, and we found that some emotions are more likely to occur with certain designer actions in the cad software. The method and conclusions presented in this paper allow us to better understand designer emotions in traditional and collaborative cad, which link to the established relationships between emotion and designer satisfaction, creativity, performance, and other outcomes increasingly valued by engineering designers and managers in virtually collaborative environments.
Issue Section:
Design Theory and Methodology
References
1.
Demirbilek
, O.
, and Sener
, B.
, 2003
, “Product Design, Semantics and Emotional Response
,” Ergonomics
, 46
(13/14
), pp. 1346
–1360
. 10.1080/001401303100016108742.
Damen
, N.
, and Toh
, C.
, 2019
, “Designing for Trust: Understanding the Role of Agent Gender and Location on User Perceptions of Trust in Home Automation
,” ASME J. Mech. Des.
, 141
(6
), p. 061106
. 10.1115/1.40422233.
Bao
, Q.
, Burnell
, E.
, Hughes
, A. M.
, and Yang
, M. C.
, 2019
, “Investigating User Emotional Responses to Eco-Feedback Designs
,” ASME J. Mech. Des.
, 141
(2
), p. 021103
. 10.1115/1.40420074.
Sas
, C.
, and Zhang
, C.
, 2010
, “Do Emotions Matter in Creative Design?
” DIS ‘10: Proceedings of the 8th ACM Conference on Designing Interactive Systems
, Aarhus, Denmark
, pp. 372
–375
.5.
National Academies of Sciences, Engineering, and Medicine
, 2017
, Information Technology and the U.S. Workforce: Where Are We and Where Do We Go from Here?
The National Academies Press
, Washington, DC
.6.
Bailey
, D. E.
, Leonardi
, P. M.
, and Barley
, S. R.
, 2012
, “The Lure of the Virtual
,” Organ. Sci.
, 23
(5
), pp. 1485
–1504
. 10.1287/orsc.1110.07037.
Balijepally
, V.
, Mahapatra
, R.
, Nerur
, S.
, and Price
, K. H.
, 2009
, “Are Two Heads Better Than One for Software Development? The Productivity Paradox of Pair Programming
,” MIS Q.
, 33
(1
), pp. 91
–118
. 10.2307/206502808.
Baas
, M.
, De Dreu
, C. K. W.
, and Nijstad
, B. A.
, 2008
, “A Meta-Analysis of 25 Years of Mood-Creativity Research: Hedonic Tone, Activation, or Regulatory Focus?
,” Psychol. Bull.
, 134
(6
), pp. 779
–806
. 10.1037/a00128159.
Barrett
, L. F.
, Lewis
, M.
, and Haviland-Jones
, J. M.
, 2016
, Handbook of Emotions
, 4th ed., Guilford Publications
, New York
.10.
Calvo
, R. A.
, and D’Mello
, S.
, 2010
, “Affect Detection: An Interdisciplinary Review of Models, Methods, and Their Applications
,” IEEE Trans. Affect. Comput.
, 1
(1
), pp. 18
–37
. 10.1109/T-AFFC.2010.111.
Ekman
, P.
, 1992
, “Are There Basic Emotions?
,” Psychol. Rev.
, 99
(3
), pp. 550
–553
. 10.1037/0033-295X.99.3.55012.
Lindquist
, K. A.
, Wager
, T. D.
, Kober
, H.
, Bliss-Moreau
, E.
, and Barrett
, L. F.
, 2012
, “The Brain Basis of Emotion: A Meta-Analytic Review
,” Behav. Brain Sci.
, 35
(03
), pp. 121
–143
. 10.1017/S0140525X1100044613.
Ekman
, P.
, and Heider
, K. G.
, 1988
, “The Universality of a Contempt Expression: A Replication
,” Motiv. Emot.
, 12
(3
), pp. 303
–308
. 10.1007/BF0099311614.
Barrett
, L. F.
, Adolphs
, R.
, Marsella
, S.
, Martinez
, A. M.
, and Pollak
, S. D.
, 2019
, “Emotional Expressions Reconsidered: Challenges to Inferring Emotion From Human Facial Movements
,” Psychol. Sci. Public Interest
, 20
(1
), pp. 1
–68
. 10.1177/152910061983293015.
Côté
, S.
, 1999
, “Affect and Performance in Organizational Settings
,” Curr. Dir. Psychol. Sci.
, 8
(2
), pp. 65
–68
. 10.1111/1467-8721.0001616.
Barsade
, S. G.
, and Gibson
, D. E.
, 2007
, “Why Does Affect Matter in Organizations?
,” Acad. Manage. Perspect.
, 21
(1
), pp. 36
–59
. 10.5465/amp.2007.2428616317.
Kaplan
, S.
, Bradley
, J. C.
, Luchman
, J. N.
, and Haynes
, D.
, 2009
, “On the Role of Positive and Negative Affectivity in Job Performance: A Meta-Analytic Investigation
,” J. Appl. Psychol.
, 94
(1
), pp. 162
–176
. 10.1037/a001311518.
Amabile
, T. M.
, Barsade
, S. G.
, Mueller
, J. S.
, and Staw
, B. M.
, 2005
, “Affect and Creativity at Work
,” Adm. Sci. Q.
, 50
(3
), pp. 367
–403
. 10.2189/asqu.2005.50.3.36719.
Niklas
, C. D.
, and Dormann
, C.
, 2005
, “The Impact of State Affect on Job Satisfaction
,” Eur. J. Work Organ. Psychol.
, 14
(4
), pp. 367
–388
. 10.1080/1359432050034888020.
Koriat
, A.
, 2012
, “When Are Two Heads Better Than One and Why?
” Science
, 336
(6079
), pp. 360
–363
. 10.1126/science.121654921.
Allen
, N. J.
, and Hecht
, T. D.
, 2004
, “The ‘Romance of Teams’: Toward an Understanding of Its Psychological Underpinnings and Implications
,” J. Occup. Organ. Psychol.
, 77
(4
), pp. 439
–461
. 10.1348/096317904259646922.
Behoora
, I.
, and Tucker
, C. S.
, 2015
, “Machine Learning Classification of Design Team Members’ Body Language Patterns for Real Time Emotional State Detection
,” Des. Stud.
, 39
, pp. 100
–127
. 10.1016/j.destud.2015.04.00323.
Bezawada
, S.
, Hu
, Q.
, Gray
, A.
, Brick
, T.
, and Tucker
, C.
, 2017
, “Automatic Facial Feature Extraction for Predicting Designers’ Comfort With Engineering Equipment During Prototype Creation
,” ASME J. Mech. Des.
, 139
(2
), p. 021102
. 10.1115/1.403542824.
Hu
, W.-L.
, Booth
, J. W.
, and Reid
, T.
, 2017
, “The Relationship Between Design Outcomes and Mental States During Ideation
,” ASME J. Mech. Des.
, 139
(5
), p. 051101
. 10.1115/1.403613125.
Barsade
, S. G.
, and Knight
, A. J.
, 2012
, “Group Affect
,” Curr. Dir. Psychol. Sci.
, 21
(2
), pp. 119
–123
. 10.1177/096372141243835226.
Liu
, Y.
, Ritchie
, J. M.
, Lim
, T.
, Kosmadoudi
, Z.
, Sivanathan
, A.
, and Sung
, R. C. W.
, 2014
, “A Fuzzy Psycho-Physiological Approach to Enable the Understanding of an Engineer’s Affect Status During CAD Activities
,” CAD Comput. Aided Des.
, 54
, pp. 19
–38
. 10.1016/j.cad.2013.10.00727.
Sivanathan
, A.
, Lim
, T.
, Ritchie
, J.
, Sung
, R.
, Kosmadoudi
, Z.
, and Liu
, Y.
, 2013
, “The Application of Ubiquitous Multimodal Synchronous Data Capture in CAD
,” Comput. Des.
, 59
, pp. 176
–191
. 10.1016/j.cad.2013.10.00128.
Ekman
, P.
, and Friesen
, W.
, 1978
, “Facial Signs of Emotional Experience
,” J. Pers. Soc. Psychol.
, 39
(6
), pp. 1123
–1134
.29.
Kleck
, R. E.
, Vaughan
, R. C.
, Cartwright-Smith
, J.
, Vaughan
, K. B.
, Colby
, C. Z.
, and Lanzetta
, J. T.
, 1976
, “Effects of Being Observed on Expressive, Subjective, and Physiological Responses to Painful Stimuli
,” J. Pers. Soc. Psychol.
, 34
(6
), pp. 1211
–1218
. 10.1037/0022-3514.34.6.121130.
Tassinary
, L.
, Cacioppo
, J.
, and Geen
, T.
, 1989
, “A Psychometric Study of Surface Electrode Placements for Facial Electromyographic Recording: I. The Brow and Cheek Muscle Regions
,” Psychophysiology
, 26
(1
), pp. 1
–16
. 10.1111/j.1469-8986.1989.tb03125.x31.
Sayette
, M. A.
, Cohn
, J. F.
, Wertz
, J. M.
, Perrott
, M. A.
, and Parrott
, D. J.
, 2001
, “A Psychometric Evaluation of the Facial Action Coding System for Assessing Spontaneous Expression
,” J. Nonverbal Behav.
, 25
(3
), pp. 167
–185
. 10.1023/A:101067110978832.
Ekman
, P.
, and Rosenberg
, E.
, 1997
, What the Face Reveals: Basic and Applied Studies of Spontaneous Expression Using the Facial Action Coding System
, Oxford University Press
, New York
.33.
Senechal
, T.
, McDuff
, D.
, and El Kaliouby
, R.
, 2015
, “Facial Action Unit Detection Using Active Learning and an Efficient Non-Linear Kernel Approximation
,” Proceedings of the IEEE International Conference on Computer Vision Workshop
, Santiago, Chile
, pp. 10
–18
.34.
Anil
, J.
, and Suresh
, L. P.
, 2016
, “Literature Survey on Face and Face Expression Recognition
,” Proceedings of the IEEE International Conference on Circuit, Power and Computing Technologies (ICCPCT)
, Kumaracoil, India
, pp. 1
–6
.35.
McDuff
, D.
, El Kaliouby
, R.
, and Picard
, R. W.
, 2012
, “Crowdsourcing Facial Responses to Online Videos
,” IEEE Trans. Affect. Comput.
, 3
(4
), pp. 456
–468
. 10.1109/T-AFFC.2012.1936.
Chen
, X.
, Gao
, S.
, Yang
, Y.
, and Zhang
, S.
, 2012
, “Multi-Level Assembly Model for Top-Down Design of Mechanical Products
,” CAD Comput. Aided Des.
, 44
(10
), pp. 1033
–1048
. 10.1016/j.cad.2010.12.00837.
Horváth
, I.
, and Vroom
, R. W.
, 2015
, “Ubiquitous Computer Aided Design: A Broken Promise or a Sleeping Beauty?
,” CAD Comput. Aided Des.
, 59
, pp. 161
–175
. 10.1016/j.cad.2014.10.00638.
Wu
, D.
, Terpenny
, J.
, and Schaefer
, D.
, 2017
, “Digital Design and Manufacturing on the Cloud: A Review of Software and Services
,” Artif. Intell. Eng. Des. Anal. Manuf. AIEDAM
, 31
(1
), pp. 104
–118
. 10.1017/S089006041600030539.
Ostergaard
, K. J.
, and Summers
, J. D.
, 2009
, “Development of a Systematic Classification and Taxonomy of Collaborative Design Activities
,” J. Eng. Des.
, 20
(1
), pp. 57
–81
. 10.1080/0954482070149965440.
Maynard
, M. T.
, and Gilson
, L. L.
, 2014
, “The Role of Shared Mental Model Development in Understanding Virtual Team Effectiveness
,” Group. Organ. Manage.
, 39
(1
), pp. 3
–32
. 10.1177/105960111347536141.
McComb
, C.
, Cagan
, J.
, and Kotovsky
, K.
, 2017
, “Optimizing Design Teams Based on Problem Properties: Computational Team Simulations and an Applied Empirical Test
,” ASME J. Mech. Des.
, 139
(4
), p. 041101
. 10.1115/1.403579342.
Red
, E.
, Holyoak
, V.
, Jensen
, C. G.
, Marshall
, F.
, Ryskamp
, J.
, and Xu
, Y.
, 2010
, “v-CAx: A Research Agenda for Collaborative Computer-Aided Applications
,” Comput. Aided Des. Appl.
, 7
(3
), pp. 1
–15
. 10.3722/cadaps.2010.387-40443.
Red
, E.
, Marshall
, F.
, Weerakoon
, P.
, and Jensen
, C. G.
, 2013
, “Considerations for Multi-User Decomposition of Design Spaces
,” Comput. Aided Des. Appl.
, 10
(5
), pp. 803
–815
. 10.3722/cadaps.2013.803-81544.
Briggs
, J. C.
, Hepworth
, A. I.
, Stone
, B. R.
, Coburn
, J. Q.
, Jensen
, C. G.
, and Red
, E.
, 2015
, “Integrated, Synchronous Multi-User Design and Analysis
,” ASME J. Comput. Inf. Sci. Eng.
, 15
(3
), p. 031002
. 10.1115/1.402980145.
Hepworth
, A. I.
, Tew
, K.
, Nysetvold
, T.
, Bennett
, M.
, and Greg Jensen
, C.
, 2014
, “Automated Conflict Avoidance in Multi-User CAD
,” Comput. Aided Des. Appl.
, 11
(2
), pp. 141
–152
. 10.1080/16864360.2014.84607046.
Eves
, K.
, Salmon
, J.
, Olsen
, J.
, and Fagergren
, F.
, 2018
, “A Comparative Analysis of Computer-Aided Design Team Performance With Collaboration Software
,” Comput. Aided Des. Appl.
, 15
(4
), pp. 476
–487
. 10.1080/16864360.2017.141964947.
Stone
, B.
, Salmon
, J. L.
, Hepworth
, A. I.
, Red
, E.
, Killian
, M.
, La
, A.
, Pedersen
, A.
, and Jones
, T.
, 2017
, “Methods for Determining the Optimal Number of Simultaneous Contributors for Multi-User CAD Parts
,” Comput. Aided Des. Appl.
, 14
(5
), pp. 610
–621
. 10.1080/16864360.2016.127357848.
Phadnis
, V. S.
, Leonardo
, K. A.
, Wallace
, D. R.
, and Olechowski
, A. L.
, 2019
, “An Exploratory Study Comparing CAD Tools and Working Styles for Implementing Design Changes
,” ICED19 22nd International Conference on Engineering Design
, Delft, Netherlands
, pp. 1383
–1392
.49.
Wynn
, D. C.
, and Eckert
, C. M.
, 2017
, Perspectives on Iteration in Design and Development
, Springer
, London
, 28
(2
).50.
Nguyen
, P.
, Nguyen
, T. A.
, and Zeng
, Y.
, 2018
, “Segmentation of Design Protocol Using EEG
,” Artif. Intell. Eng. Des. Anal. Manuf. AIEDAM
, 33
(1
), pp. 11
–23
.51.
Mcduff
, D.
, El Kaliouby
, R.
, Senechal
, T.
, Amr
, M.
, Cohn
, J. F.
, and Picard
, R.
, 2013
, “Affectiva-MIT Facial Expression Dataset (AM-FED): Naturalistic and Spontaneous Facial Expressions Collected in-the-Wild
,” Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops
, Portland, OR
, June 23–28
.52.
Kulke
, L.
, Feyerabend
, D.
, and Schacht
, A.
, 2020
, “A Comparison of the Affectiva iMotions Facial Expression Analysis Software With EMG for Identifying Facial Expressions of Emotion
,” Front. Psych.
, 11
, pp. 1
–10
. 10.3389/fpsyg.2020.0032953.
Qu
, F.
, Wang
, S. J.
, Yan
, W. J.
, and Fu
, X.
, 2016
, “CAS(ME)2: A Database of Spontaneous Macro-Expressions and Micro-Expressions
,” International Conference on Human-Computer Interaction
, Toronto, Canada
, pp. 48
–59
.54.
Lopez-Rincon
, A.
, 2019
, “Emotion Recognition Using Facial Expressions in Children Using the NAO Robot
,” CONIELECOMP 2019—2019 International Conference on Electronics, Communications and Computers
, Puebla, Mexico
, pp. 146
–153
.55.
Spaulding
, S.
, and Breazeal
, C.
, 2019
, “Frustratingly Easy Personalization for Real-Time Affect Interpretation of Facial Expression
,” 2019 8th International Conference on Affective Computing and Intelligent Interaction, ACII 2019
, pp. 531
–537
.56.
Elfenbein
, H. A.
, 2007
, “Emotion in Organizations: A Review and Theoretical Integration
,” Acad. Manage. Annals
, 1
(1
), pp. 315
–386
. 10.5465/07855981257.
Coburn
, J. Q.
, Salmon
, J. L.
, and Freeman
, I.
, 2018
, “Effectiveness of an Immersive Virtual Environment for Collaboration With Gesture Support Using Low-Cost Hardware
,” ASME J. Mech. Des.
, 140
(4
), p. 042001
. 10.1115/1.403900658.
Ruzzante
, S. W.
, and Bilton
, A. M.
, 2018
, “Agricultural Technology in the Developing World: A Meta-Analysis of the Adoption Literature
,” Proceedings of the ASME 2018 International Design Engineering Technical Conferences and Computers and Information in Engineering Conference IDETC/CIE 2018
, Quebec City, Canada
, Aug. 26–29
.59.
Moen
, E. L.
, Fricano-Kugler
, C. J.
, Luikart
, B. W.
, and O’Malley
, A. J.
, 2016
, “Analyzing Clustered Data: Why and How to Account for Multiple Observations Nested Within a Study Participant?
,” PLoS One
, 11
(1
), pp. 1
–17
. 10.1371/journal.pone.014672160.
Grandey
, A.
, 2008
, Handbook of Organizational Behavior
, J.
Barling
, and C.
Cooper
, eds., Sage
, London
.61.
Pourtois
, G.
, Schettino
, A.
, and Vuilleumier
, P.
, 2013
, “Brain Mechanisms for Emotional Influences on Perception and Attention: What is Magic and What is Not
,” Biol. Psychol.
, 92
(3
), pp. 492
–512
. 10.1016/j.biopsycho.2012.02.00762.
Oh
, J.
, and Sundar
, S. S.
, 2015
, “How Does Interactivity Persuade ? An Experimental Test of Interactivity on Cognitive Absorption, Elaboration, and Attitudes
,” J. Commun.
, 65
(2
), pp. 213
–236
. 10.1111/jcom.1214763.
Sundar
, S. S.
, Bellur
, S.
, Oh
, J.
, Jia
, H.
, and Kim
, H. S.
, 2016
, “Theoretical Importance of Contingency in Human-Computer Interaction: Effects of Message Interactivity on User Engagement
,” Commun. Res.
, 43
(5
), pp. 595
–625
. 10.1177/009365021453496264.
Häggman
, A.
, Tsai
, G.
, Elsen
, C.
, Honda
, T.
, and Yang
, M. C.
, 2015
, “Connections Between the Design Tool, Design Attributes, and User Preferences in Early Stage Design
,” ASME J. Mech. Des.
, 137
(7
), p. 071408
. 10.1115/1.403018165.
Robertson
, B. F.
, and Radcliffe
, D. F.
, 2009
, “Impact of CAD Tools on Creative Problem Solving in Engineering Design
,” CAD Comput. Aided Des.
, 41
(3
), pp. 136
–146
. 10.1016/j.cad.2008.06.00766.
Fixson
, S. K.
, and Marion
, T. J.
, 2012
, “Back-Loading: A Potential Side Effect of Employing Digital Design Tools in New Product Development
,” J. Prod. Innov. Manage.
, 29
(S1
), pp. 140
–156
. 10.1111/j.1540-5885.2012.00959.x67.
Schon
, D. A.
, 2016
, The Reflective Practitioner: How Professionals Think in Action
, Routledge
, London
.68.
Buolamwini
, J.
, and Gebru
, T.
, 2018
, “Gender Shades: Intersectional Accuracy Disparities in Commercial Gender Classification
,” Proceedings of the 1st Conference on Fairness, Accountability and Transparency, PMLR
, New York
, Feb. 23–24
.69.
Lopez
, C.
, and Tucker
, C.
, 2019
, “Mining Facial Keypoint Data: The Quest Toward Personalized Engineering Applications,” Emotional Engineering
, Vol. 7
, S.
Fukuda
, ed., Springer
, Cham, Switzerland
, pp. 97
–112
.70.
Chen
, M.
, Anderson
, J. R.
, and Sohn
, M.-H.
, 2001
, “What Can a Mouse Cursor Tell Us More? Correlation of Eye/Mouse Movements on Web Browsing
,” CHI'01
, Seattle, WA
, CHI,pp. 281
–282
.71.
Dering
, M. L.
, Tucker
, C. S.
, and Kumara
, S.
, 2018
, “An Unsupervised Machine Learning Approach to Assessing Designer Performance During Physical Prototyping
,” ASME J. Comput. Inf. Sci. Eng.
, 18
(1
), p. 011002
. 10.1115/1.403743472.
Bonebright
, D. A.
, 2010
, “40 Years of Storming: A Historical Review of Tuckman’s Model of Small Group Development
,” Hum. Resour. Dev. Int.
, 13
(1
), pp. 111
–120
. 10.1080/1367886100358909973.
Mcstay
, A.
, 2020
, “Emotional AI, Soft Biometrics and the Surveillance of Emotional Life: An Unusual Consensus on Privacy
,” Big Data Soc.
, 7
(1
), pp. 1
–12
. 10.1177/2053951720904386Copyright © 2020 by ASME
You do not currently have access to this content.
