Remember me
Noise is present in most work environments, including emissions from machines and devices, irrelevant speech from colleagues, and traffic noise. Although it is generally accepted that noise below the permissible exposure limits does not pose a considerable risk for auditory effects like hearing impairments. Yet, noise can have a direct adverse effect on cognitive performance (non-auditory effects like workload or stress). Under certain circumstances, the observable performance for a task carried out in silence compared to noisy surroundings may not differ. One possible explanation for this phenomenon needs further investigation: individuals may invest additional cognitive resources to overcome the distraction from irrelevant auditory stimulation. Recent developments in measurements of psychophysiological correlates and analysis methods of load-related parameters can shed light on this complex interaction. These objective measurements complement subjective self-report of perceived effort by quantifying unnoticed noise-related cognitive workload. In this review, literature databases were searched for peer-reviewed journal articles that deal with an at least partially irrelevant “auditory stimulation” during an ongoing “cognitive task” that is accompanied by “psychophysiological correlates” to quantify the “momentary workload.” The spectrum of assessed types of “auditory stimulations” extended from speech stimuli (varying intelligibility), oddball sounds (repeating short tone sequences), and auditory stressors (white noise, task-irrelevant real-life sounds). The type of “auditory stimulation” was related (speech stimuli) or unrelated (oddball, auditory stressor) to the type of primary “cognitive task.” The types of “cognitive tasks” include speech-related tasks, fundamental psychological assessment tasks, and real-world/simulated tasks. The “psychophysiological correlates” include pupillometry and eye-tracking, recordings of brain activity (hemodynamic, potentials), cardiovascular markers, skin conductance, endocrinological markers, and behavioral markers. The prevention of negative effects on health by unexpected stressful soundscapes during mental work starts with the continuous estimation of cognitive workload triggered by auditory noise. This review gives a comprehensive overview of methods that were tested for their sensitivity as markers of workload in various auditory settings during cognitive processing.
Keywords: Cognition, measure, noise, task, workload
How to cite this article:
Noise is the proportion of the perceived soundscape that is irrelevant to the listener’s goals.[1],[2] For that reason, noise in the context of occupational safety and health, public health, and environmental protection is often defined as unwanted sound that can cause disturbance, annoyance, impairment, or damage (see DIN 1320). Sound consists of superimposed acoustic waves stemming from several sound sources. Only some of these superimposed acoustic waves might be relevant to the listeners’ goals at a time. However, the listener might not be aware that a particular momentarily irrelevant sound source will become relevant (like signaling danger[3]). If the goals of the listener change, auditory information judged to be noise, may become a relevant signal. This observation highlights the importance of permanent auditory vigilance. Accordingly, the auditory perceptual system has certain peculiarities unseen in other modalities.[4],[5] The human visual system allows averting the gaze from a visual object or closing the eyelids. In contrast, auditory sensory information cannot be suppressed in such a manner. Instead, attentional resources need to be directed to another sound stream to implicitly ignore an unwanted auditory stream.[6] Still, the irrelevant sensory information is processed to a varying degree, even if it is intended to be suppressed. Moreover, the probability that the suppressed stream dominates conscious attention is still elevated.[7] Therefore, the information filtered out as noise can be best described by its unfavorable effect on cognition by distraction.[1] The auditory cognition constantly monitors and segregates the auditory information. This filtration process comes at a particular cognitive workload.[8],[9],[10]
Mental work and the underlying processes are particularly prone to additional cognitive workload. The preservation of the momentary cognitive state is a prerequisite for cognitive activities.[11] The processing of information can be disrupted by mind wandering (internally induced[12]) and unexpected sensory input (externally induced[13]). The externally induced change in the cognitive state can be caused by occupational noises. Noise, even below the permissible sound pressure levels and at a short exposure duration, lowers concentration, often triggers stress, and may require additional workload to maintain the current work performance.[11] Many studies find evidence that the individual perception in a subjective rating of noise sounds is negative or detrimental to the perceived psychological wellbeing.[14],[15],[16],[17]
At work, the employee behaves goal-oriented: consequently, available cognitive resources are focused on work tasks. Thus, the prevention of auditory disturbances during this focused process plays a crucial role in the design of the workplace. Work tasks can be fulfilled at a lower cognitive workload if the workplace is designed ergonomically.[18] One aspect of acoustic ergonomics is the soundscape: noise is among the most frequent and annoying factors in many workplaces.[19] For instance, machine noise from close-by production areas, conversations of colleagues (density of office space), traffic sound from an open window (density of urban space), and the neighbors’ children playing in the corridor (home office) − the sources seem endless and are partly interwoven with the core job duties. For example, in a pedagogic setting, a teacher in a classroom needs to focus attention on a selected pupil’s voice, even in the presence of irrelevant speech. This scenario underlines how job characteristics may co-occur with certain stressful acoustic settings. The mental processing of information is becoming the dominant element of many professions.[20] In digitalizing societies, mental work is gradually superseding more and more tasks.[21] Regulators have identified noise at work as a health issue and invested time in the evaluation, prevention, and protective measures.[22] Effects of noise on the employee are classified as either auditory or non-auditory.[23] Auditory effects of noise relate to noise-induced hearing impairment; non-auditory effects include, among others, annoyance, stress, and effects on work and task performance. The scope of this review is non-auditory effects of noise.
If the noise is consciously perceived as disturbing, the impact of noise adversely affecting task performance can be prevented by adapting the workspace: for example, reducing noise emissions at the source by selecting low-noise work equipment, reshaping the workspace by taking room acoustics measures tailored to the work activity, organizational measures leading to spatial or temporal separation from the noise source to reduce noise exposure and, as a last resort, wearing hearing protection. According to European occupational health and safety regulations, employers shall assess risks to the safety and health of employees at workplaces and take measures to avoid or reduce the risks. In terms of noise, that means employers bear the responsibility of evaluating and minimizing the risk of the worker related to activities exposed to noise. If necessary, the employer shall measure the level of noise that workers are exposed to and take action to eliminate the risks arising from exposure to the noise source (see Directive 2003/10/EC of the European Parliament and of the European Council[24]). Nevertheless, noise below the permissible exposure limits was also found to subconsciously affect cognition,[25] which might chronically accumulate into negative health effects. In other studies, where no effect can be measured on the performance and the subjectively experienced workload,[26],[27] researchers assumed that the cognitive system activates a compensatory strategy that invests additional workload in overcoming noise-related demands.
To expand the scientific knowledge as a prerequisite for improving occupational health and safety regulations, especially with regard to non-auditory noise effects, it would be helpful to have an unambiguous marker that can objectively track the actually invested workload. Until now, the effects of noise on a primary cognitive task have been evaluated mostly with retro- and introspective diagnostic instruments and task performance comparisons.[28],[29] Besides detrimental effects on task performance, non-auditory effects also comprise the noise-induced workload. The self-reported individual perception through questionnaires is a powerful tool that allows cheap and fast insights into the experienced cognitive workload. For example, the NASA, N. (1986). Task load index (tlx) v. 1.0 manual. NASA, NASA-Ames Research Center Moffett Field is an instrument that allows ratings of the perceived workload during a just finished task.[30] However, limitations of a subjective rating scale are not only the inconsistencies in individual judgments, depending on emotional states, memory capacity, and willingness to cooperate. Also, the current workload cannot be monitored online during the processed task itself.[31] In contrast, objective measurements that correlate with (neuro-) physiological markers that represent activities in the cognitive system have the potential to track noise-related workload online. In this way, the unchanged task performance (and subjective reports) between a silence and noise condition can be explained better.
The advances in the required sensor technology (e.g., miniaturization, price reduction, mobility[32]), mobile applications,[33] and development of software for complex mathematical analysis pipelines (e.g., regression, classification, networks) drive the assessment of cognitive workload in objectively measurable units.[34] In the digital world, the quantification of human resources allows for better workforce planning and prevention of work-related diseases. The scope of the review is to identify measurement methods, including sensitive parameters, and describe the acoustics settings they are most commonly used in. In this fashion, scientists can combine the appropriate measure in their respective occupational investigations for sensitivity tests.
A broad spectrum of measuring techniques has been found to be sensitive to the momentary cognitive workload in different instruments such as pupillometry,[35] electroencephalogram (EEG),[36] functional near-infrared spectroscopy (fNIRS),[37] and electrocardiogram (ECG).[38] However, those objective measurement methods are often used in a controlled laboratory setting, sometimes within an experimental paradigm that contextualizes the findings and limits generalizability. The attributability of specific proportions of cognitive workload (task- or noise-related) is a major research area that needs further investigation. In this review, we offer a starting point on which measurement technique is applicable and in which combination it might be administered to cover realistic work settings. That is, the employee is usually engaged in a primary cognitive task covering already some of the available cognitive resources (task-related workload). If during this ongoing cognitive process, noise is present, the listener may have to invest more effort to maintain performance on the primary cognitive task. This review has a particular focus on the objective measurement methods that allow the quantification of noise-related workload in the presence of an already demanding primary cognitive task. Several reviews deal with the impact of short- and long-term exposure to noise on chronic health.[39],[40],[41] However, we examine here publications with the acute, non-auditory effects of noise on cognitive workload and the respective measurability.
Since the amount of mental work is increasing in the work-life of many employees, the facilitation of safe work environments is fundamental for future acoustic ergonomics. The prevention of noise effects helps improve the quality of work, the personal experience of work, and, subsequently, psychological health (technical rules for the ordinance on workplaces[42]). In this paper, we review the literature of the last 10 years on the quantification of noise-related workload during an ongoing mental task. Along the following four dimensions that establish the search string, we investigated which measurement methods are being used and showed effects on which specific parameters. We aim to describe qualitatively in a comprehensive review of empirical findings regarding the objective measure of workload (1. measurement dimension) in adverse auditory conditions (2. acoustic dimension) during an ongoing cognitive process (3. cognitive task dimension), which generates mental workload (4. workload dimension).
MethodThe research question can be answered from many viewpoints (e.g., speech-perception effort, general noise stress, auditory split attention task). We adopted a reduced form of the PRISMA framework to synthesize the evidence.[43] Due to the heterogeneity in the screened approaches chosen in the publications examined, not all items of the PRISMA framework could be addressed. Our methodological approach consisted of developing a search string, the reproducible search requests in representational literature databases, a technical and human filtration process, and the classification of the findings within the prescribed four dimensions. Ultimately, the connection within and between the dimensions can be observed in publication-based categories, allowing statements in which the combination of auditory stimulation, cognitive task, and measurement methods effects can be expected. In this way, conclusions upon the clusters of evidence can be made regarding which measurement methods should be used in which setting.
Search string design
The search string was developed in a discussion by two researchers based on expert knowledge and refined by strategies that help prevent bias. The design was supported by learnings from bibliographic methods, such as formulated by Page et al.[44] In addition, the risk of bias was minimized by carefully selecting terms with neutral meanings concerning the expected effect.
Since we were interested in publications satisfying all four dimensions (acoustic, cognitive task, workload, measurement), we had to specify search terms for each dimension. As independent variables, the acoustic environment and the cognitive task are defined. The workload and the measurement dimensions contain dependent variables. As shown in [Table 1], the dimensions of cognitive task and measurement were split into two. In this hierarchical order, the search engine needed to satisfy all dimensions with at least one included search term each. The search terms animal*, child* should not appear in the final selected publication. The search engines usually committed themselves only to the title, abstract, and the (author selected) keywords, but that differed inconsistently between databases. The full text is not regarded in this primary step.
Databases
Seven literature databases (Web of Science, Scopus, EBSCO, ILO, JSTOR, PubMed, SAGE) were queried for the search string in German and English language on March 8, 2021 between 11 am and 1 pm. The timespan of considered publications extended from 2010 to 2021. Depending on the syntactic commands the literature database required (e.g., AND, OR, NOT), the above search terms were transformed and translated in the fitting format to appease the search field logic. For example, in the “web of science” the exact search string looked like this in a shortened form (“…” symbolize the remaining search terms found in [Table 1]):
TS=((acoustic* OR “background speech” OR …) AND (cognit* OR mental) AND (arith* OR attend* OR attention* OR …) AND (allocat* OR arous* OR …) AND (assess* OR activit* OR correlat* OR evaluat* OR …) AND (*adrenaline* OR biofeedback …)) NOT TS=(animal* OR child*)
Filtration
The filtration process consisted of the removal of the duplicates (technical and human assistance), the screening of the titles, the screening of the abstracts, and finally, the full-text screening. The topical inclusion criteria of the screening process were based on the cardinal dimensions that define the search string (acoustic, cognitive task, workload, measurement). Exclusions could happen along all stages of the screening steps (title/abstract/full-text). Only peer-reviewed journal articles were considered that had been published in a journal with an impact factor available on Scopus. The general quality of the article was rated to meet “good scientific practice” in terms of depth of description and study design.
Extraction
From eligible publications, relevant information was extracted in an abbreviated form to allow for an overview and evidence synthesis. That included information on…
the sample (sample size, gender distribution, age, occupation, exclusion criteria, exclusion count, location, count of experimental groups, experimental setting, session duration)the auditory stimulation (number of acoustic conditions, parameter modulated, description of sound type, duration, and position relative to the cognitive task)the cognitive task (number of cognitive tasks, name of the task, the tested cognitive system, test name [if applicable], difficulty levels per task, performance parameter)the moderator variables (general moderators [e.g., personality], subjective rating, name of the subjective effort scale, subscales)the measurement methods (number of measurement methods, name of the method, company and product name, measurement type, analyzed parameter, time of measurement relative to the cognitive task, invasivenessthe results (summary thereof, effect size)the discussion (summary thereof)The whole process was executed by a researcher and, after that, discussed critically in a group of five researchers of the same institute and three external senior scientists. Finally, the resulting data extraction table was further analyzed and aggregated to reveal fundamental insights into the presented evidence. The data extraction table can be found in Appendix 1.
Categorization
Based on the results identified during the extraction process, three dominating categories could be identified within the publications. Often the auditory stimuli used in the publications can be conflated within a single category because the underlying design is similar. Different task-relevant auditory stimuli were applied in combination with noise in the acoustic dimension: many of the speech-related studies used various corpora of text, sentences, and words that are eventually masked by noise or irrelevant speech. Evidence-based on speech stimuli was condensed into a single category in the deployed information reduction process (C1–C3, see results). The three categories were set up to establish a model of the evidence, allowing conclusions within the categories along the four analyzed dimensions. In this way, we could overcome a fragmented insight and get the whole picture of the heterogeneous publication sample.
ResultsFiltration
During the search in the selected databases, 5479 publications were accumulated. Two thousand and three hundred fifteen publications were identified as duplicates and removed (42%). The titles of the remaining 3164 publications were screened for the thematic focus, which was not satisfied for 2279 publications (72%). The thematic focus of a publication had to fulfill the four dimensions that were introduced above. During the next screening of the abstracts of 885 publications, 730 were, again, thematically unfit and removed (82%). At this stage, the journal required a journal impact factor released in the Scopus network. In the last stage of the filtration process, the full text of 155 publications was screened, and the eligibility was rated. Based on a thematic focus, 84 publications were eligible for data extraction and consequently screened for the included evidence and experimental details. Seventy one publications were removed after full-text screening (46%), see [Figure 1]. The data extraction table can be accessed in Appendix 1.
Figure 1 Filtration process. Remaining publications after each step of filtration.The analysis of the bibliographic properties of the selected 84 publications revealed a dominance of the search term “listening” in the title of the publication. Further, a growing number of publications were published in the considered time slot from 2010 (1 publication) to 2020 (11 publications). The most frequent journals where the publications were published in were “Ear and Hearing” (9 publications, 11%), “Hearing research” (5 publications, 6%), and “Trends in hearing” (4 publications, 5%). The literature databases Scopus (49 publications) and Web of Science (21 publications) contributed the most publications finally considered.
All publications consisted of intervention studies that controlled different experimental conditions on a selected sample of study participants. The size of the samples varied around M = 34 participants, SD = 18 with, if stated, around M = 18 females, SD = 18. In the following analysis, the statements are based solely on information from the publications that provided them. Publications without information stated are not included in the conclusions. The participant sample was in most cases stemming from the Netherlands (17 publications, 20%), USA (9 publications, 11%), Germany (6 publications, 71%), China (4 publications, 5%), or France (4 publications, 5%). Most of the studies were situated in a laboratory (67 publications, 80%) or were executed in a simulation environment (14 publications, 17%), and in-field studies summed to 3 publications (4%).
The cardinal dimension in this review is the acoustic dimension. We are interested in how the cognitive workload is modulated in the presence of task-irrelevant sound. Therefore, studies were considered that include both partially and completely task-irrelevant sounds. In this fashion, it is possible to use evidence from a wider set of publications that may eventually shed light on this complex interaction. Most studies had two separate acoustic conditions (28 publications, 33%), 3 (27 publications, 32%), 1 (16 publications, 19%) acoustic condition(s), 4 (9 publications, 11%), 5 (2 publications, 2%), 17 (1 publication, 1%), and 30 (1 publication, 1%). In 42 publications (50%), a silent condition was introduced as a baseline condition. That means, besides the 16 publications with a single acoustic condition (19%), another 26 publications presented studies without a silent condition (31%). The silent condition was defined by the absence of the otherwise present auditory stimuli. This condition might differ from a more controlled silent condition, as it is available in acoustic research laboratories only. The sound pressure level of the non-silent acoustic conditions in the studies varies between 35 and 80 dB(A).
Categories identified within the acoustic dimension
Within the acoustic dimension, three broad categories could be identified. Each publication would be allocated to one of the three categories based on the used auditory stimulus and experimental paradigm: “listening effort” (C1), “auditory stressor” (C2), and “oddball” (C3). These three categories describe which auditory stimulation and experimental paradigm were included in the specific publication. A publication in the “listening effort” category (C1) contains to-be-remembered masked speech, see publications.[45],[46],[47],[48],[49],[50],[51],[52],[53],[54],[55],[56],[57],[58],[59],[60],[61],[62],[63],[64],[65],[66],[67],[68],[69],[70],[71],[72],[73],[74],[75],[76],[77],[78],[79],[80],[81],[82],[83],[84] The “auditory stressor” category (C2) contained a stimulus independent of the cognitive task and contained various auditory materials, see publications.[85],[86],[87],[88],[89],[90],[91],[92],[93],[94],[95],[96],[97],[98],[99],[100],[101],[102],[103],[104],[105],[106],[107],[108],[109],[110],[111],[112],[113],[114],[115],[116],[117] The “oddball” category (C3) contained a task-independent stimulus that followed the oddball experiment design (repetitive short sounds with a break in the pattern), see publications.[118],[119],[120],[121],[122],[123],[124],[125],[126],[127] The subsequent analysis of evidence is performed within the categories C1 to C3, see [Figure 2].
Figure 2 “The fibers of the chord”: schematic of the topical review process.In the following section, each category (C1–C2) is described along with the four dimensions: the acoustic dimension, the cognitive task dimension, the workload dimension, and the measurement dimension. In this way, it is possible to grasp which measurement method (and parameter) was combined with which acoustic settings and which cognitive tasks. The following table gives an overview of the three categories and the four dimensions, [Table 1]. The cells of the table are intended to help researchers to select the appropriate combination for their own research question, as it has been used in the literature. A detailed description is eventually given in the following paragraphs.
Category 1 (C1) “Listening Effort”
In the “listening effort” research, the workload stems from the challenges in the intelligibility of the acoustic speech signals. Listening effort emerges when mental resources are deliberately allocated to overcome acoustic obstacles while listening to speech.[128]. Again, in an occupational setting, a teacher in the classroom must withstand a specific workload when listening to a voice of a selected pupil (pupil A, male pitch, location x) that is masked by the voice of another pupil (pupil B, female pitch, location y). In this situation, the auditory stimulus is partially “irrelevant” (pupil B) and can be labeled as noise. Therefore, the listener must exert selective attention to direct attentional resources onto the target voice (pupil A). The relevant signal (pupil A) is necessarily processed to extract the targeted linguistic information. In this scenario, the auditory system can filter the interfering speech based on spatial cues (location) and voice cues (male/female pitch), among others. In this vein, the perception of speech is regarded as a cognitive task that is dependent on working memory capacity and linguistic capabilities.[129],[130] Therefore, the processing intensity varies by the external, acoustic factors (e.g., speech degradation by noise) and internal cognitive factors (e.g., intelligence).
Examples for the listening effort category are publications with the titles: “Cognitive processing workload during listening is reduced more by decreasing voice similarity than by increasing spatial separation between target and masker speech,”[45] “Listening effort during sentence processing is increased for non-native listeners: A pupillometry study,”[47] and “Impact of stimulus-related factors and hearing impairment on listening effort as indicated by pupil dilation.”[48]
C1 “Listening Effort”: acoustic dimension
The category “listening effort,” C1, is a so-called mono-modal category because the experiment is limited to one modality. The speech perception in the listening effort category represents a cognitive task, and the masker sound represents the noise in the acoustic dimension. In both dimensions, the acoustic dimension, and cognitive task dimension are taking place in the auditory modality. However, the following two identified categories (i.e., auditory stressor, C2, and oddball, C3) might rely on two modalities at the same time, for example, the visual, for the cognitive task, and the auditory modality, for the noise. For that reason, both categories are bi-modal since the noise is partially independent of the cognitive task. Furthermore, both categories cover different aspects of occupational settings, like speech-related or text-processing tasks. If a listening effort study dealt with hearing impaired and non-native speaker samples, only the evidence from the normal hearing and native speaker sample was considered in the following analysis.
In the listening effort category, C1, the acoustic conditions (from 1 to 30 possible conditions) systematically manipulated the signal-to-noise ratio (SNR) to influence the intelligibility of the presented target speech. By changing the target and masker voice differences (e.g., gender, spatial, mono/binaural, single/multi-talker), the SNR was adopted. In some studies, the masker’s voice was replaced by a non-speech masker (white noise, stationary noise, pink noise, fluctuating noise), speech-shaped noise, and noise-vocoded speech. In other cases, the masker was replaced by multi-talker babble (e.g., four-talker, two-talker, five-talker, 10-talker, 30-talker) noise or a recording from a complex soundscape (e.g., café, open-plan office, swimming pool). The masker sound usually starts before the target speech and ends shortly after. During the retention and recall phase, no noise sound is presented in almost all cases.
C1 “Listening Effort”: cognitive task dimension
The task for the participant in the listening effort paradigm is often to repeat (recall and verbalize) the whole or parts of the sentence, keyword(s), and single words from the just presented target speech stimulus material. Different cognitive functions are engaged in such a speech reception threshold task (e.g., memory, attention, sensory, linguistics processes). The response could be verbal, by keyboard/computer interface (multiple-choice), or visual closeness tests. The cognitive task is to perceive the target speech in a masked stimulus, to memorize the relevant part of the stimulus, and finally to recall and respond. Consequently, both the noise and the cognitive task are hearing-based only without activating other modalities like vision. The auditory stimulus is partly relevant and partly irrelevant.
C1 “Listening Effort”: workload dimension
The induced cognitive workload (e.g., processing workload, listening effort, stress, demand, mental resource allocation) is estimated by task performance (correctly perceived target speech) and is doubled by objective measurements like pupillometry (mean/maximal peak amplitude), EEG, ECG, EDG, endocrinological makers (salivary cortisol), fNIRS, and fMRI (measurement dimension), see below in the measurement dimension. In addition, some studies investigated moderator variables like cognitive abilities, memory span tests, working memory capacity, need for recovery, reading span score, tiredness, fatigue (fatigue visual analog scale), perceived subjective workload (e.g., NASA TLX), listening effort, and linguistic closure abilities.
C1 “Listening Effort”: measurement dimension
The cognitive load is often investigated between conditions with modulated intelligibility and quantified by one measure. Always, the measure is accompanied by a behavioral parameter (performance).
The studies showed that the pupil dilation (amplitude, slope, latency of amplitude) was modulated with the listening effort: the lower the intelligibility, the higher the amplitude of the dilation of the pupil.[45–56,58–61,64–66, 69, 71, 74, 75, 77, 79-81] Often such effects were measured in a fixed retention period (“offline”) following the stimulus presentation and, as well, during the presentation period itself (online). No such effect was found in one pupillometry study.[73] EEG-based publications reported time- and frequency-based effects for late positive potential components, P3, N1, N2,[57] theta and alpha band power,[66] N1, N2, and P3,[70] and power analysis.[83]
Endocrinological markers in the saliva (cortisol, CgA) did not reveal any effects.[64] Cardiovascular activity studies found effects for the pre-ejection-period reactivity,[62] mixed results for the heart-rate variability,[78],[82] but no effect for high-frequency heart-rate variability.[63] The effects on skin conductance (EDG, EDA) were somewhat mixed.[63],[82],[131] Hemodynamic studies reported in regard to our research question unclear effects for the HbO2 concentration in the prefrontal cortex, measured by fNIRS,[68],[132] and various activations in other brain regions, measured by fMRI.[84]
Category 2 (C2) “Auditory Stressor”
Attentional resources are competed for by incoming sensory information because they are limited. Attention refers, in this view, to the depth of processing: auditory stimuli are processed only until the cognitive system can decide with a certain probability that the auditory stimulus is relevant. Thus, irrelevant stimuli are less processed than relevant stimuli for which more memory capacity is reserved,[133] informational streams are prioritized,[134] and conscious perception and transformation of stimuli are more likely.[135]
An “auditory stressor” is a stimulus that is irrelevant to the current primary cognitive task. Still, the auditory stressor cannot be blatantly ignored but is processed to a certain extent. In this category, all content of the auditory sensory information is distractive and includes no information necessary for handling the primary cognitive task (or signal a threat). In this case, the participant needs to focus available mental resources only on the cognitive task.
Examples of the auditory stressor category are publications with titles such as: “Quantifying the effects of external factors on individual performance,”[85] “Effects of low-intensity noise from aircraft or from the neighborhood on cognitive learning and electrophysiological stress responses,”[86] and “Increased anxiety induced by listening to unpleasant music during stress exposure is associated with reduced blood pressure and ACTH responses in healthy men.”[87]
C2 “Auditory Stressor”: acoustic dimension
Studies with an auditory stimulus categorized as an “auditory stressor” often contained a silence condition. In the remaining conditions, low-noise, silent white noise, light/pleasant music, nature sounds, and safe sound conditions are contrasted against high-noise, industrial noise, rock/unpleasant music, ambient noise, and threat sound conditions. Like the oddball stimulus (see below C3), the auditory stressor was presented during different stages of the primary task. However, the oddball stimulus is somewhat artificial compared to the real-life sounds used in the auditory stressor studies. Often the auditory stressor is presented for a longer time (e.g., up to around 15 min), during which several cognitive tasks must be executed.
C2 “Auditory Stressor”: cognitive task dimension
The cognitive tasks that played a role in these publications reach from the text-processing task (listening, reading, processing, writing), Stroop task, mental arithmetic task, proactive interference, response inhibition, flanker task, logics, recall/shifting/updating memory/attention, problem-solving, (simulated) car driving, n-/2-back task, delayed response task, gambling, sustained attention to response task (SART), a complex control task, stress test, deliberated eye-movement task, simulated surgery task, and object tracking task. Such tasks were used to test various cognitive systems (attention, memory, concentration, among others) in applied settings found at work. Sometimes, different difficulty levels (e.g., easy, medium, arduous task) of the cognitive tasks were applied and factored eventually with the acoustic conditions (e.g., silent, loud 1, loud 2). As a result, the cognitive task’s performance parameters varied strongly (e.g., hit rate, accuracy, reaction time).
C2 “Auditory Stressor”: workload dimension
The workload was varied with different levels of difficulty in the cognitive tasks. The load necessary to shield a cognitive process against noise potentially interacts with the mental workload stemming from the primary task. However, the cognitive tasks evaluated here are usually not in the auditory modality but are dominated by another modality (e.g., visual). That means that the cognitive processes in this category might block the auditory stimulus as a whole, and not just partially, as it was in the listening effort category, C1.
C2 “Auditory Stressor”: measurement dimension
In the auditory stressor category, the heterogeneity of the applied measurement technologies was larger than in the listening effort and oddball categories. Different measurement methods were applied: behavioral,[85] cardiovascular,[86, 87, 90–92, 94, 99, 102–105, 107, 108, 112–114] skin conductance,[105],[108],[113],[115],[116] endocrinology,[87],[89] EEG,[88],[92],[95],[96],[98],[106],[112] fNIRS,[91],[94],[99],[100],[112] pupillometry,[91],[94],[117] fMRI,[93],[98],[101] and eye-tracking.[97],[100],[109],[110],[111]
In all studies, the presence of noise affected at least one parameter of each measurement method. Several different movement types of a manually controlled computer mouse for behavioral measurements,[85] blood pressure,[87],[112] heart-rate variability (HRV),[86],[90],[94],[102],[107],[114] heart rate,[94] RR-intervals[99] for cardiovascular activity, fluctuations in skin conductance,[86],[108],[113],[115],[116] ACTH for endocrinology,[87] N400,[88] prefrontal cortex N100,[95] feedback-related negativity,[96] P300, steady-state visual potentials[106] for EEG, O2Hb in fNIRS,[91],[94],[99],[100] pupil dilation,[94],[109],[117] respiration rate,[92],[103] frequency ratio of HR,[92] the power of the frequency bands,[92],[96] default mode network in fMRI[93] and frontal and parietal areas,[98],[101] the gaze shifts in the eye-tracking,[97],[110],[111] and urinary cortisol.[102]
No effects were found in publications for the cortisol level, norepinephrine response,[89] inter-beat-intervals,[103],[104],[105] and eye movements.[109]
Category 3 (C3) “Oddball”
The oddball paradigm is a well-established auditory paradigm to study the fundamentals of auditory processing. An oddball stimulus consists of a repeating sequence of, usually, two types of sounds: frequent “standard sounds” that are similar or equal and infrequent or unpredictable (or “odds”) “deviant sounds” that differ from the standard sounds perceptibly. Besides the insights into the fundamental structure of the auditory system, the paradigm allows for studying a process called attention capture.[136] This way, it is possible to measure subjects’ susceptibility to distractions from an irrelevant stimulus, like the deviating oddball sound. The oddball paradigm can be realized as an active and passive response format.[137] Either a deliberate response by the participant is required (active oddball, like, verbal response, button-press, among others) or an automatic or reflexive response from (passive oddball: e.g., EEG, pupil dilation) is recorded during the passive perception (non-responding) of the oddball stimulus (difference between standards and deviants) − or mixtures of both is applied.
Therefore, the passive oddball paradigm can function as a secondary task, which is combinable with various cognitive tasks. The attentional processes driving the oddball deviant detection occupy the remaining cognitive resource that is available besides the workload of the primary task. In this scenario, the oddball sounds are irrelevant to the primary task and can be staged at different phases of ongoing cognitive processes.
Examples for the oddball category are publications with the titles like: “Auditory task irrelevance: a basis for inattentional deafness,”[123] “Disruption in neural phase synchrony is related to identification of inattentional deafness in real‐world setting,”[118] and “Event-related potentials associated with auditory attention capture in younger and older adults.”[120]
C3 “Oddball”: acoustic dimension
The considered oddball does not consist of several different conditions but a single one. However, the oddball paradigm can be used easily in real-life settings like an aircraft cabin or movie screening as a dual-task. The oddball sounds can be either single frequency sounds or sounds from the environment like a barking dog or a ringing bell, among others. Most considered publications do not require an active response. The time point the oddball paradigm is applied varies between the studies depending on the primary cognitive task.
C3 “Oddball”: cognitive task dimension
The oddball paradigm was combined with the following primary cognitive tasks: n-back task, verb-generation-task, aircraft piloting, movie watching, mental imagery tasks, speech-in-noise task, visuomotor tasks, and steering tasks. The primary task produced a cognitive workload that was eventually accompanied by the oddball task’s cognitive workload. The “oddball task” is to detect the deviant sound, which requires the auditory system to form a prediction about incoming sensory information.
C3 “Oddball”: workload dimension
Both the primary and secondary tasks evoke a workload that usually was not analyzed separately. Instead, the difference between the standard and deviant patterns was monitored. In this way, as a secondary task, the oddball task covers the supposed “remaining” cognitive workload. Once no more cognitive load capacity is available, the performance in the deviant detection of the oddball task decreases. The investigated cognitive system referred to dual-/multi-tasking, (visual, auditory, and cross-modal) attention, shielding, goal-directed behavior, (short-term) working memory, executive functions, decision making, perceptual load, general attentional resources. In addition, some studies applied moderator variables to investigate the subjectively perceived effort (NASA TLX, Stanford sleepiness scale).
C3 “Oddball”: measurement dimension
The oddball paradigm is a classic EEG experiment.[118],[119],[120],[121],[122],[123],[124],[125] One study considered was Magnetoencephalography-based (MEG).[126] In the EEG experiments, analysis can be based on a time-domain analysis (event-related potential, ERP) or frequency-domain analysis (power spectrum). For the ERPs, amplitudes in defined time-slots relative to the auditory event can be investigated: mismatch negativity (MMN), deviant-related negativity (DRN), late positive potential (LPP), P3a, P300, N1, N2, and early/late/frontal P3.
In the majority of the studies, effects can be detected for at least one of the investigated ERPs even in the absence of an active response,[119],[120],[121],[122],[123],[124],[125],[126],[127] as well as for the MEG in the auditory steady-state response.[126] Similarly, the power analysis reveals effects in the phase reset, alpha, beta, theta band, power ratios, and the general “brain state.” [118],[121],[122],[125]
DiscussionThe estimation of the current cognitive workload in humans is complex. However, measuring cognitive workload stemming from two cognitive processes (one task- and one noise-related process) that are potentially partially interconnected (modality general) and partially separate (modality specific) is an even more arduous endeavor. Psycho- and neurophysiological correlates of cognitive workload, like ERPs,[138] variations in cardiovascular activity,[139] and dilations of the pupil,[140] have received attention from research communities for different purposes. Objective measures are often used for understanding fundamental psychological functions.[141] As well, the measurements are capable of predicting the performance in cognitive tasks and estimating the current workload in various situations.[142] In this vein, correlates reveal to some extent the involvement of a particular cognitive process when confronted with a certain type of task. Nowadays, it is possible to record and monitor changes in the cognitive state with a sensor unobtrusively.[143] In that case, the worker can be immersed in a task without distractions from subjective effort reports. Thus, the well-established subjective report of cognitive workload can be partially replaced and extended with a myriad of data from objective measurement methods introduced above. The recorded parameters represent specific aspects of the activity of the mind (e.g., neuronal activity, hemodynamic blood flow, muscle contractions). Additionally, data from body sensors have a high granularity (sampling rate) and can be monitored during the task processing in real-time.
However, the question that we raised is whether such measurement methods can be used as a marker for cognitive workload in situations with and without noise. The reviewed results indicate that the described measurement methods can detect noise-related effects. In the following, we want to structure the challenges formulated in the literature to help develop measurement systems for occupational safety evaluations (i.e., combinations of sensors). We propose occupational scenarios in which the findings can be further studied in-depth. Finally, an integrated measurement system of several sensors relies on the selection of suitable sensory channels (e.g., EEG and PD) that have demonstrated sensitivity to some extent and benefit from each other when combined. A limitation of this review is the absence of a quantitative sensitivity comparison between objective measures used. The sensitivity should be investigated in future studies with an experimental setting that is identical to all tested objective measures, whic
Comments (0)