The Psychophysiology Primer: A Guide to Methods and a Broad Review with a Focus on Human–Computer Interaction

B. Cowley, M. Filetti, K. Lukander, J. Torniainen, A. Henelius, L. Ahonen, O. Barral, I. Kosunen, T. Valtonen, M. Huotilainen, N. Ravaja, and G. Jacucci

Abstract

Digital monitoring of physiological signals can allow computer systems to adapt unobtrusively to users, so as to enhance personalised ‘smart’ interactions. In recent years, physiological computing has grown as a research field, and it is increasingly considered in diverse applications, ranging from specialised work contexts to consumer electronics. Working in this emerging field requires comprehension of several physiological signals, psychophysiological states or ‘indices’, and analysis techniques. The resulting literature encompasses a complex array of knowledge and techniques, presenting a clear challenge to the practitioner. We provide a foundational review of the field of psychophysiology to serve as a primer for the novice, enabling rapid familiarisation with the core concepts, or as a quick-reference resource for advanced readers. We place special emphasis on everyday human–computer interface applications, drawing a distinction from clinical or sports applications, which are more commonplace. The review provides a framework of commonly understood terms associated with experiential constructs and physiological signals. Then, 12 short and precisely focused review sections describe 10 individual signals or signal sources and present two technical discussions of online data fusion and processing. A systematic review of multimodal studies is provided in the form of a reference table. We conclude with a general discussion of the application of psychophysiology to human–computer interaction, including guidelines and challenges.

Countering Countermeasures: Detecting Identity Lies by Detecting Conscious Breakthrough

Howard Bowman , Marco Filetti, Abdulmajeed Alsufyani, Dirk Janssen, Li Su

Abstract

One major drawback of deception detection is its vulnerability to countermeasures, whereby participants wilfully modulate their physiological or neurophysiological response to critical guilt-determining stimuli. One reason for this vulnerability is that stimuli are usually presented slowly. This allows enough time to consciously apply countermeasures, once the role of stimuli is determined. However, by increasing presentation speed, stimuli can be placed on the fringe of awareness, rendering it hard to perceive those that have not been previously identified, hindering the possibility to employ countermeasures. We tested an identity deception detector by presenting first names in Rapid Serial Visual Presentation and instructing participants to lie about their own identity. We also instructed participants to apply a series of countermeasures. The method proved resilient, remaining effective at detecting deception under all countermeasures.

In the Figure: difference waves for all participants that applied a specific countermeasure against our Concealed Information Test(CIT). Clear difference can be seen for all participants, despite their attempt to apply countermeasures.

Subliminal Salience Search Illustrated: EEG Identity and Deception Detection on the Fringe of Awareness

Howard Bowman, Marco Filetti, Dirk Janssen, Li Su, Abdulmajeed Alsufyani, Brad Wyble

Abstract:

We propose a novel deception detection system based on Rapid Serial Visual Presentation (RSVP). One motivation for the new method is to present stimuli on the fringe of awareness, such that it is more difficult for deceivers to confound the deception test using countermeasures. The proposed system is able to detect identity deception (by using the first names of participants) with a 100% hit rate (at an alpha level of 0.05). To achieve this, we extended the classic Event-Related Potential (ERP) techniques (such as peak-to-peak) by applying Randomisation, a form of Monte Carlo resampling, which we used to detect deception at an individual level. In order to make the deployment of the system simple and rapid, we utilised data from three electrodes only: Fz, Cz and Pz. We then combined data from the three electrodes using Fisher’s method so that each participant was assigned a single p-value, which represents the combined probability that a specific participant was being deceptive. We also present subliminal salience search as a general method to determine what participants find salient by detecting breakthrough into conscious awareness using EEG.

In the figure: ERP responses for the four critical items presented in the experiment. The concealed item (Probe) the pretented item (Fake) and the two Irrelevants. The Probe elicits a very visible response, even though participants were attempting to conceal this item. This is a type of concealed information test (CIT).