Abstract

A negative link between media multitasking and sustained attention has been proposed; yet, whether such a link exists remains hotly debated as previous studies found mixed effects. The present study seeks to evaluate the size of this effect taking into account possible variations due to how media multitasking is measured, how sustained attention is assessed, and the origin of the samples. Using an established and a novel, shortened measure for media multitasking, 924 participants were recruited through three different platforms (Amazon Mechanical Turk, Prolific, and university students). In addition to questionnaire- and task-based assessments for sustained attention, impulsivity and sensation seeking were also assessed to further qualify behavioral problems associated with media multitasking. The findings establish a negative link between media multitasking and sustained attention of a medium effect size, whether questionnaires (r = .20) or a task-based measure (r = .21) are used. Importantly, the findings support the notion that previous differences across studies can be at least partly attributed to the choice of media multitasking measure as well as differences across samples.

Keywords: media multitasking, attention, digital media, sample differences

Funding: This work was funded by the Office of Naval Research Grant N00014-20-1-2074 and the H2020 European Research Council—BrainPlay— ERC-2018-SyG to Daphne Bavelier. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the article.

Disclosures: The authors have no conflicts of interest to disclose.

Authors Contributions: Daphne Bavelier, Kenneth Rioja, and Susanne E. Baumgartner designed the study. Kenneth Rioja collected all data. Kenneth Rioja and Sezen Cekic carried statistical analyses and wrote the first draft of the Method and Result sections. Susanne E. Baumgartner wrote the first draft of the introduction and Discussion sections. All coauthors contributed to the final article. Kenneth Rioja and Sezen Cekic share equal first authorship. Daphne Bavelier and Susanne E. Baumgartner share equal last authorship.

Data Availability: Data, analytic methods, and study materials are publicly available on Open Science Framework: https://osf.io/nt7de/

Acknowledgments: The authors thank F. Joessel for sharing her knowledge about the gradual-onset continuous performance task, P. E. Joessel for the implementation of this task online, and B. Bediou for his constant help on R and about the Media Multitasking Index.Funding: This work was funded by the Office of Naval Research Grant N00014-20-1-2074 and the H2020 European Research Council—BrainPlay—ERC-2018-SyG to Daphne Bavelier. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the article.

Correspondence concerning this article should be made to Kenneth Rioja, Faculté de Psychologie et des Sciences de l’Éducation, Université de Genève, Boulevard du Pont d’Arve 40, CH-1211 Geneva 4, Switzerland [email protected] E. Baumgartner, Amsterdam School of Communication Research, University of Amsterdam, Nieuwe Achtergracht 166, 1018 WV Amsterdam, the Netherlands [email protected]

Watch a brief video abstract:

Recent years have seen an upsurge in studies investigating the associations between media multitasking and a variety of cognitive functions, such as sustained attention, executive functioning, working memory, and impulsivity (for reviews and meta-analyses, see Uncapher et al., 2016; Uncapher & Wagner, 2018; van der Schuur et al., 2015; Wiradhany & Koerts, 2019; Wiradhany & Nieuwenstein, 2017). Media multitasking is the simultaneous use of two or more media types (e.g., van der Schuur et al., 2015). For example, watching an online video and at the same time sending a message to a friend via social media would be a typical form of media multitasking. While multitasking in everyday life is not new (think of driving and talking, or eating your breakfast and reading the newsprint), the frequency of media multitasking has been greatly increased since digital technologies have invaded our lives. In fact, in the current digital and mobile media landscape, media multitasking has become a very common, if not the most common, form of media use (Ettinger & Cohen, 2020). Media multitasking has triggered the interest of researchers after a seminal study by Ophir et al. (2009) that has found evidence for different cognitive profiles of heavy versus light media multitaskers (LMM). Following this study, these associations have been further investigated by a variety of laboratories, with mixed results. Whereas many studies found negative associations between the amount of media multitasking and various measures of attention, other studies did not find these associations (Uncapher & Wagner, 2018; Wiradhany & Nieuwenstein, 2017).

Recent meta-analyses (e.g., Wiradhany & Nieuwenstein, 2017) and reviews on the relationship between media multitasking and attention and related constructs (Uncapher & Wagner, 2018; van der Schuur et al., 2015) concluded that these varying findings might, among other reasons, be due to sample as well as measurement heterogeneity. For example, Wiradhany and Nieuwenstein (2017) concluded in their meta-analysis that, across all studies, effect sizes were weak and might become negligible when accounting for small-study effects. This raises the concern that these findings might be driven by a few small-sample studies. In addition, Uncapher and Wagner (2018) suggested that the variance in findings might be due to heterogeneity in the measures used. Due to the fast-changing nature of the media environment, studies adapted the original version of the media multitasking measure by Ophir et al. (2009) to reflect those changes in media activities (Madore et al., 2020; Pea et al., 2012). Thus, what constitutes media multitasking might differ across studies.

Finally, studies differed in their use of outcome measures, with some studies using cognitive tasks to assess attention (e.g., Cain & Mitroff, 2011; Cardoso-Leite et al., 2016; Ophir et al., 2009; Ralph et al., 2015; Seddon et al., 2018), and others mainly relying on self-reports (Baumgartner, van der Schuur, et al., 2017; Ralph et al., 2014; Uncapher et al., 2016). Interestingly, studies employing self-reports for measuring attention have been suggested to show more consistent negative relations with media multitasking than studies employing performance-based measures (Baumgartner et al., 2014; Parry et al., 2021; Wiradhany & Koerts, 2019).

Thus, despite a growing body of research, there is no consensus yet on how consistent the link is between media multitasking and attention. Attention is a multifaceted construct and we focus here on sustained attention, or the ability to maintain focus for several seconds to minutes, in the face of distractions. This form of attention, especially as measured by continuous performance tasks, has been one of the main focus when it comes to the impact of media multitasking on attention (e.g., Madore et al., 2020; Ophir et al., 2009; Wiradhany & Nieuwenstein, 2017). Next to sustained attention, we focus here on two additional previously established correlates of media multitasking, sensation seeking (e.g., Sanbonmatsu et al., 2013; Shin et al., 2019) and impulsivity (e.g., Baumgartner, Lemmens, et al., 2017 ; Cain et al., 2016).

To unravel the link between media multitasking, sustained attention, impulsivity, and sensation seeking, the aims of the present study were (a) to conduct the same study across three samples to test for sample heterogeneity and evaluate the extent to which small-study effects may be of concern, (b) to examine the associations between media multitasking and attention, as assessed from both self-reports and a performance-based task of sustained attention, in the same samples, (c) to examine the associations between media multitasking and impulsivity and sensation seeking, as assessed with self-reports, in the same samples, and (d) to develop and validate a novel media multitasking measure that is shorter to administer and less repetitive for participants and to test whether this new measure correlates to the same extent with attention, impulsivity, and sensation seeking than the previously used and established Media Multitasking Index (MMI).

Media Multitasking and Sustained Attention Assessed With Performance-Based Tasks

The original study by Ophir et al. (2009) found that heavy media multitaskers (HMM) performed less well than LMM on the AX-continuous performance task. In this paradigm, participants are presented with a sequence of five letters (made of a cue, three distractors, and a probe) and asked to press a button marked “YES” to the probe “X” if preceded by the cue “A,” but otherwise press a button marked “NO.” HMM were overall slower than LMM when correctly responding to target trials (“X” preceded by “A”) or incorrectly to lure trials (“X” preceded by anything but “A”), when the paradigm included distractors. This was not attributable to different speed–accuracy trade-offs as HMM and LMM exhibited comparable performance as measured by sensitivity or d′. In addition, in the version of the paradigm without distractors, the two groups were comparable.

This finding led the field into trying to replicate this link between multitasking and distractibility. In a replication study, Wiradhany and Nieuwenstein (2017) presented two experiments using the same AX-continuous performance task task with distractors. The first study showed similar slowing of reaction time as Ophir et al. (2009) but only for incorrect responses to lure (“X” preceded by anything but “A”), whereas the second study found similar slowing of reaction time in HMM but only for correct hits (“X” preceded by “A”), thus only partially replicating previous findings.

Since there is not one commonly agreed task of sustained attention, studies vary highly in the use of measures. For example, Ralph et al. (2015) used the metronome response task (MRT), the sustained attention-to-response task (SART), as well as a vigilance task. Their findings raise fundamental concerns about the link between media multitasking and sustained attention, as the findings across three studies show that results may vary depending on (a) the task used (MRT, SART, vigilance task) and (b) the population of interest (undergraduate students, MTurkers). Nevertheless, all results follow the same trend indicating a negative relationship, albeit weak, between levels of media multitasking and sustained attention.

In Wiradhany and Nieuwenstein (2017), authors performed a meta-analysis revealing a small overall relationship between greater media multitasking and greater distractibility during continuous task performance with an effect size of d = .17 (corresponding to r = 0.085). Note that the pooled analyses were cross-sectional, contrasting high media multitaskers and low media multitaskers, and most of the studies suffer from the lack of a convincing number of participants. Extracting from this meta-analysis the studies using continuous performance tasks, the (transformed) effect sizes ranged from r = 0.16 (Wiradhany & Nieuwenstein, 2017) to r = 0.51 (Ophir et al., 2009).

Importantly, studies employing performance tasks typically employ small samples. Therefore, Wiradhany and Nieuwenstein (2017) concluded that when correcting for small-study effects, evidence for robust effects seems to disappear. It is thus yet unclear whether the effects found in previous studies were mainly due to small-study effects or whether the associations between media multitasking and sustained attention are indeed small and require large samples to be detected. Moreover, existing studies are characterized by a strong sample heterogeneity. Existing samples did not only differ in cultural backgrounds (e.g., samples coming from the United States, Europe, or Asia) but also in educational background (student samples vs. general population), and whether samples were recruited in person (e.g., on campus) or online (e.g., MTurk).

More recently, using a long-term memory task where memory retrieval was elicited through a cue, Madore et al. (2020) found more attention lapses in high as compared to low media multitaskers during the processing of the retrieval cue (see Extended Data Figure 7 in Madore et al., 2020). Using the gradual-onset continuous performance task, known as the GradCPT (Rosenberg et al., 2013), they established that propensity to memory lapses was related to performance on the GradCPT, arguing that “trait-level differences in sustained attention are related to individual differences in [forgetting]” (Madore et al., 2020, p. 4). Indeed, the GradCPT has been shown to provide a robust way to identify the neural networks that mediate sustained attention (Rosenberg et al., 2016; Scheinost et al., 2020). Combined with the ease of remote administration of the GradCPT via online data collection (unlike the AX-continuous performance task for which task instructions are more challenging), the present work used the GradCPT task. Furthermore, in the present study, a sufficiently powered sample was used to examine whether there are robust and consistent associations between media multitasking and performance-based sustained attention. We hypothesize:Hypothesis 1: There is a negative relationship between media multitasking and performance on a sustained attention task.

Media Multitasking and Sustained Attention, Impulsivity, and Sensation Seeking Assessed With Self-Reports

In general, studies using self-reports for measuring attention in everyday life found more consistent and stronger negative correlations with media multitasking compared to performance-based measures of sustained attention (see Parry & le Roux, 2021, for a recent meta-analysis). For example, Ralph et al. (2014) found relationships between the frequency of media multitasking and attentional failures and mind-wandering in daily life. Similarly, in two large adolescent samples, Baumgartner, van der Schuur, et al. (2017) found that adolescents, who engaged in media multitasking more frequently, reported more attention problems in their daily lives.

As media multitasking comprises switching between different media activities, it does not only have an attentional component (e.g., switching attention between media tasks) but also a behavioral component (e.g., typing a message, scrolling through social media). It is thus not surprising that media multitasking has also been related to more behavioral components such as impulsiveness. Several studies have shown that media multitasking is negatively related to motor impulse control and behavioral inhibition. For example, Cain et al. (2016) and Baumgartner, Lemmens, et al. (2017) found associations between media multitasking and motor impulsivity among adolescents, Sanbonmatsu et al. (2013) and Shin et al. (2019) found these associations among adults. Interestingly, Sanbonmatsu et al. (2013) found similar associations between media multitasking and the motor as well as attentional component of impulsiveness providing support for the idea that media multitasking is related to both attention as well as motor control.

Overall, these findings are supported in a recent meta-analysis (Wiradhany & Koerts, 2019), which concluded that heavier media multitasking was consistently, albeit weakly, related to self-reported attention problems (pooled effect size, r = 0.161, k = 21) and increased motor impulsiveness in everyday life and sensation seeking (pooled effect size, r = 0.216, k = 15).

Sensation seeking is a personality trait that is closely related to cognitive control and impulsivity (Holmes et al., 2016). It has been recently argued that the tendencies to seek stimulation and to control impulses are both correlated with the anatomical structure of cognitive control circuitry (Holmes et al., 2016). This indicates that impulsivity and sensation seeking might be linked due to a common biological origin. Interestingly, both personality traits have been previously linked to media multitasking (Baumgartner, Lemmens, et al., 2017; Sanbonmatsu et al., 2013). Individuals who seek stimulation are more likely to engage in media multitasking. This might be due to the oftentimes stimulating and hedonic nature of media content (e.g., Panek, 2014). Yet, the link between sensation seeking and media multitasking remains largely understudied, which will be addressed in the present study.

Overall, recent studies and meta-analyses on media multitasking point to a negative link between media multitasking and self-reports of attention, impulsivity, and sensation seeking, whereas the correlations between media multitasking and cognitive tasks of sustained attention are less consistent (e.g., Baumgartner et al., 2014; Luo, Li, et al., 2021; Luo, Yeung, et al., 2021; Parry & le Roux, 2021; Seddon et al., 2018; Wiradhany & Koerts, 2019; Wiradhany & Nieuwenstein, 2017). This discrepancy between findings based on self-reports and those based on task performance might be due to self-reports and performance-based tasks tapping into different psychological constructs that both contribute independently to the prediction of daily-life behavior (e.g., Luo, Li, et al., 2021; Parry et al., 2021; Sharma et al., 2014; Toplak et al., 2013). Another potential explanation is that particularly in the case of media multitasking, studies employing self-reports were frequently based on larger samples and might thus have had more power to detect even small effects. It is, therefore, important to study the relationship between media multitasking and its correlates based on self-reports and performance-based tasks within the same, sufficiently powered, sample. As previous studies have shown consistent relationships between media multitasking and self-reported measures of attention, impulsivity, and sensation seeking, we also expect that:Hypothesis 2: There is a positive relationship between media multitasking and self-reported measures of attention problems as well as of impulsivity and sensation seeking.

Addressing Measurement Weaknesses in Media Multitasking Measures: Toward a Shorter, Frequency-Based Measure of Media Multitasking

Most existing studies in the field have used the MMI (Ophir et al., 2009) or a slightly adapted version as the key indicator for media multitasking (Madore et al., 2020). However, several authors have noted that the use of the MMI might be problematic for at least three reasons (Baumgartner, van der Schuur, et al., 2017; Uncapher & Wagner, 2018; Wiradhany & Nieuwenstein, 2017).

The first major problem that the field faces is the fast-changing media landscape. For example, students filling in the media multitasking questionnaire 10 years ago might have used completely different technologies for multitasking than students nowadays. Thus, ideally, media multitasking measures should be less sensitive to technological developments and should in particular be platform-independent, so that the same measure can be used over time. Although it is difficult to foresee future technological developments, it is possible to focus on the key functionalities of media products. For example, in the original Media Use Questionnaire (MUQ; Ophir et al., 2009), a differentiation was made between watching TV and online videos. In today’s media landscape, this differentiation is frequently obsolete. More importantly, however, whether someone engages in a secondary media activity while watching TV on an actual TV device or online videos on a laptop should not make a qualitative difference for media multitasking as both types of media rely on the same psychological modalities (e.g., see Baumgartner & Wiradhany, 2021).

A second critique of the MMI concerns the exact quantity measured. The MMI reports the number of media used in a typical media hour. This has been criticized (Uncapher & Wagner, 2018; Wiradhany & Nieuwenstein, 2017) because it does not assess the frequency of media multitasking. Thus, someone who scores high on the MMI could in principle engage in media multitasking only very rarely, but when they do so use a large variety of media at the same time. Typically, media effects research relies heavily on measures based on the frequency of media use. It has thus been debated whether the frequency of media multitasking would not be a better indicator than the number of media used in a media hour (e.g., Uncapher & Wagner, 2018; Wiradhany & Nieuwenstein, 2017).

Finally, the length of the MUQ on which the MMI is based is quite extensive which makes it difficult to use in studies with larger assessment batteries. Its length is mainly due to the questionnaires probing the combination of each media activity with all others. Recent studies have used a slightly adapted scale from the original Ophir et al. (2009) scale, such as in Madore et al. (2020). This version keeps asking about all possible combination of media but assumes symmetry between two activities. In short, if someone answers that while reading, they often listen to music; this adapted scale assumes that while listening to music, they often read. In this version, media activity is probed in a way that is less dependent on exact technology or app used, but rather aligned with the cognitive processes engaged. Given this slightly adapted scale tends to be more used in recent studies, this is the established scale we will be using in the present work. We will term it “adapted scale” thereafter to make clear it is not exactly the original Ophir et al. (2009) scale.

Additionally, a few shorter scales have been successfully introduced in the past (Baumgartner, Lemmens, et al., 2017; Luo et al., 2018). These have been based on a selection of only a few media types, however, making the comparison of these scales with the MMI more difficult. We therefore developed a shorter measure not by reducing the number of primary media activities but by reducing the amount of secondary media activities. That is, instead of asking for each primary media activity (e.g., watching videos), how frequently someone uses each other type of medium as secondary activity (e.g., social media, gaming, audio), we ask for the general frequency of using any type of other media while engaging in a primary media activity (e.g., while watching videos how often do you use other types of media at the same time?). This approach does allow us to keep a similar sensitivity concerning the primary activity but reduces the number of items substantially. In addition, by assessing the frequency of media multitasking instead of the original “number of media used per media hour,” we hope to increase the interpretability of the measurement. We thus aimed at developing a new short scale that covers the same platform-independent media activities as more recent versions of the MMI but that assesses the frequency of media multitasking rather than the number of media used per media hour.

In sum, this new measure is adapted in three ways: (a) it is shorter than the MMI, (b) it is frequency-based, and (c) it is platform-independent and therefore not as sensitive to technological developments. To successfully validate this newly developed measure, we compare how well it correlates with the MMI, and whether it correlates equally well than the MMI with all outcome measures. We expect:Hypothesis 3: The new frequency-based media multitasking questionnaire will correlate (a) highly with the MMI and (b) equally well with task-based and questionnaire-based measures of interest as the original MMI.

The Present Study

Previous findings on the relationship between media multitasking and attention, impulsivity, and sensation seeking indicate small-to-medium effect sizes at best (see Uncapher & Wagner, 2018; Wiradhany & Nieuwenstein, 2017). However, many studies suffer from small samples and inconsistent measures across samples. This makes it difficult to conclude whether the effects that have been found were due to small-study effects or to the specific measures used.

In the present study, we assess the association between media multitasking and sustained attention, impulsivity, and sensation seeking in a large sample of N = 924 participants. We examine the associations between media multitasking and attention by using self-reports as well as a performance-based task for sustained attention (GradCPT). We expect media multitasking to be negatively related to sustained attention across self-reports and the cognitive task (Hypothesis 1 and Hypothesis 2). We are cognizant that previous work showed heterogeneous effects across studies, raising the issue of whether differences in findings may be explained by insufficiently powered samples, by differences in the measures that were used for either media multitasking or for sustained attention, or by possible sample differences. In the present study, we recruited participants from three different samples, via MTurk, Prolific, and a university student pool. This puts us in the position to assess whether online samples and the standard student-based samples used in most psychology experiments are comparable. Within that debate, MTurk has been pointed out to possibly provide lesser quality data (e.g., Peer et al., 2022). We therefore included two different online samples, recruited from MTurk as well as Prolific.

Moreover, following the call by Uncapher and Wagner (2018), we present a technology-independent, frequency-based, and shorter measure of media multitasking and test whether this shorter, frequency-based questionnaire may be used equally well as the adapted media multitasking questionnaire that targets number of media used when multitasking and not how frequently one multitasks. We expect this novel media multitasking questionnaire to show similar correlations as the original media multitasking questionnaire (Hypothesis 3). The extent to which the frequency-based questionnaire can be used to assess media multitasking has both practical implications (i.e., shorter measures) as well as theoretical ones, as it will characterize the relative importance of frequency of media multitasking versus number of media used during media multitasking in driving an association between media multitasking and sustained attention.

Method

Participants

This study was deployed online through three data collection phases, one on the online platform Amazon Mechanical Turk (MTurk sample, https://www.mturk.com), another on the online platform Prolific (Prolific sample, https://www.prolific.co), and the third through remote testing within a university student pool (Student sample) of the University of Amsterdam.

The studies were approved by the respective universities’ ethics board. Participants were compensated for the completion of the whole study which was expected to last between 25 and 30 min at the rate of $3.25 for MTurk, £3.80 for Prolific, and with course credits for the Student participants.

Data collection for the MTurk sample was performed between November 2020 and April 2021. In total, 409 participants were initially enrolled in the study using the following qualification requirements: (a) human intelligence task approval rate greater than or equal to 99, (b) number of human intelligence tasks approval rate greater than 1,000, (c) age between 18 and 35 years (to keep the sample comparable concerning age range to the student sample).

Data collection for the Prolific sample was performed between November 2020 and April 2021 with a total of 302 participants taking part in the study. The following audience settings were used: (a) approval rate greater than or equal to 95; (b) age between 18 and 35 years old (to keep the sample comparable concerning age range to the student sample).

Data collection for the Student sample took place in November 2020. A total of 299 undergraduate psychology or communication science students participated in the study. The usable sample was between the ages of 18 and 30 years old. The studies were conducted online; it was recommended that participants use a desktop or laptop.

Measures

Adapted MUQ

The original Ophir et al. (2009) MUQ was adapted and shortened following the version used by Madore et al. (2020): (a) the number of media was reduced from twelve to nine; (b) the wording about the different media was changed; (c) the media multitasking items were reduced to half as, if the item “when doing i, how frequently do you use j at the same time” was probed, then the item “when doing j (…) use i” was not probed, in essence assuming symmetry in multimedia usage. The nine media considered were as follows: (1) reading; (2) watching videos, movies, TV; (3) listening to music, radio, audiobooks, podcasts; (4) playing video games; (5) browsing the internet; (6) texting, using social media or instant messaging; (7) talking on the phone or video chatting; (8) other computer activities; (9) performing work or studying. The scale was specially designed to abstract from specific apps or services to be more robust to the fast-changing pace of the technological developments.

As in the Ophir et al. (2009) work, the first part of the questionnaire asks for the hours spent in an average week engaging in each of the nine media; the second part asks for each primary medium i listed above, the frequency of engaging simultaneously in each of the other secondary media j, assuming symmetry in usage as explained above. Answers were given on a 4-point Likert-type scale and were assigned numeric values: never (=0), a little of the time (=0.33), some of the time (=0.66), most of the time (=1).

Scoring proceeded separately from the first and second part of the questionnaire. A media hours (MH) score was computed from summing all nine self-reported hours per week in the first part of the questionnaire. A MMI was computed similarly to Ophir et al. (2009) using the original formula. $\text{MMI} = \sum_{i = 1}^{9}\frac{m_{i} \times h_{i}}{h_{\text{total}}}$, where m_i is the sum of all frequencies when using one medium i of simultaneously using each of the other media j; h_i is the self-reported hour per week of the medium i; and h_total, thereafter termed MH (for media hours) is the sum of all self-reported hours per week. The adapted questionnaire and its scoring sheet can be downloaded on the Open Science Framework repository of the study (Rioja et al., 2022).

Novel Frequency-Based Media Multitasking Use Questionnaire (F-MUQ)

A novel MUQ was developed that aimed at examining the frequency of media multitasking, rather than number of media used simultaneously. The nine different media types, which closely mirrored the categories used in our adapted version of the MUQ, were (1) reading; (2) watching videos, movies, TV; (3) listening to music, radio, audiobooks, podcasts; (4) playing video games; (5) browsing the internet; (6) texting, using social media or instant messaging; (7) talking on the phone or video chatting; (8) content creation; (9) performing work or studying. Thus, the nine media were exactly the same except for the eighth medium “other computer activities” which was replaced by asking about “content creation.”

The F-MUQ has three parts: the same two-parts structure as the original MUQ, and two items related to nonmedia multitasking. The first part of the F-MUQ asks about the frequency of usage in general for each of the nine media through a 5-point Likert scale. The second part of the questionnaire asks for each of the nine media, the frequency of using this medium while engaging simultaneously in other media in general (whatever type of media it was). An example question is: “While reading, I use other media simultaneously ….” The third part asks about multitasking in nonmedia activity (talking face-to-face and eating), and will not be discussed further. Each response was given on a 5-point Likert-type scale with the extreme button choices labeled never and very Often (and no label given to the three intermediate button choices). For data coding, the numeric value of 0 was assigned to never followed by 0.25, 0.5, 0.75, and finally 1 for very often.

Scoring of the first part allowed us to compute a media frequency (MF) index by averaging the frequencies of usage of the nine media. A frequency-based MMI (F-MMI) was computed as the average of the nine multitasking media items from the second part of the questionnaire. The frequency-based questionnaire can be found on the Open Science Framework repository of the study (Rioja et al., 2022).

Attention Problems

Attention problems were measured with a five-item, abridged version of the nine-item inattentiveness scale from the Diagnostic and Statistical Manual of Mental Disorders, fifth edition. The five questions were the same as in Baumgartner, van der Schuur, et al. (2017) and included questions such as (e.g., “I am easily distracted.”). Responses were given on a 5-point Likert scale ranging from strongly disagree to strongly agree. The attention problems score is computed as the sum of the five items.

Motor Impulsiveness

The Abbreviated Barratt Impulsiveness Scale to assess motor impulsiveness (Coutlee et al., 2014). It has four items (e.g., “I do things without thinking”) with response on a 5-point Likert scale ranging from not at all to very often. The motor impulsiveness score is computed as the sum of the four items.

Sensation Seeking

This two-item scale (Slater, 2003) aims to measure one’s propensity to risk taking in a self-reported way. The two items are “How often do you do dangerous things for fun?” and “How often do you do exciting things, even if they are dangerous?” using a 5-point Likert scale ranging from not at all to very often. The sensation seeking score is obtained by summing answers of the two items.

Cognitive Task—GradCPT

The GradCPT was selected as continuous performance tasks that are related to impulsivity and other forms of top-down control dysfunctions in both humans and animals (Dalley et al., 2011). Also, its relationship with the MMI has emerged as a topic of interest to understand the impact of MMI on top-down control (Madore et al., 2020).

This 10-min task-based assay of sustained attention requires participants to view pictures of cities (90% of trials) and of mountains (10% of trials—497 pictures total) presented sequentially at a rate of one picture every 1.2 s (with a 0.6 s shared between picture n fading out and picture n + 1 fading in). Participants were asked to press the space bar to cities while withholding responses to mountains as in Esterman et al. (2013) and Rosenberg et al. (2016).

The original MATLAB code from Rosenberg et al. (2016) was reprogrammed in JavaScript to be administered through an internet browser with data saved in a MySQL database. Participants were prompted to adjust screen parameters such as brightness, color, and size-on-screen through a short setting phase at the beginning of the task as in Yung et al. (2015). The pictures were grayscale images of 10 cities and of 10 mountains, formatted to be circular subtending 7° of visual angle. A short video of the task is available on the Open Science Framework repository of the study (Rioja et al., 2022).

As in most go/no-go task, the presented analyses will focus on the accuracy variable d-prime (d′).

Demographic Information

Participants were asked to answer demographic questions about sex, gender, age, and education level.

Procedure

After reading a short description of the overall study, participants were redirected to Qualtrics where they were asked to provide consent. Demographic questions were displayed first, then participants were randomly assigned to complete one version of our two media multitasking questionnaires (MUQ or F-MUQ). They then were administered the GradCPT task, asked to fill the three self-report questionnaires and to complete the other version of our two media multitasking questionnaires (F-MUQ or MUQ). At the start of the study, participants were instructed “to limit any source of distraction and leave their phone far from them so they can focus on the study.”

Data were processed, analyzed, and visualized in R (v3.6.1; R Core Team, 2019), using the following packages: dplyr (v1.0.7; Wickham et al., 2021), effectsize (v0.4.5; Ben-Shachar et al., 2020), ggeffects (v.1.1.4; Lüdecke, 2018), ggplot2 (v3.3.5; Wickham, 2016), ggpubr (v0.4.0; Kassambara, 2020), pacman (v0.5.1; Rinker & Kurkiewicz, 2017), papaja (v0.1.0.9997; Aust & Barth, 2020), psycho (v0.6.1; Makowski, 2018), pwr (v.1.3-0; Champely, 2020), rockchalk (v.1.8.157; Johnson, 2022), and rstatix (v0.7.0; Kassambara, 2021). Power analysis was performed with the function pwr.r.test() from the pwr package. Spearman correlations were calculated with the function cor() from the stats package with the arguments method = spearman. Extreme group analyses were performed by comparing lower and higher quartile groups from MMI and F-MMI distributions. These analyses are t tests performed with t.test() from the stats package; d and 95% confidence intervals were computed with cohens_d() from the effectsize package. Interaction analyses were performed through ggpredict() from the ggeffects package, and the ΔR²s are computed with getDeltaRsquare() from the rockchalk package. Sequential analyses of variance (ANOVAs) and model comparison were computed through aov() and BIC() functions from the stats package, and η² were computed through eta_squared() from the rstatix package. The data and analyses for this study are available on the Open Science Framework repository of the study (Rioja et al., 2022).

Participant Exclusion

All three samples underwent the same procedure: (a) removal of participants having the same ID (26 for MTurk, none for Prolific and Student), (b) removal of participants below 18 or above 35 years old (three for MTurk, none for Prolific and Student), (c) removal of participants not reaching the 497 trials on the GradCPT or having a GradCPT duration of shorter than the 10 min expected for the 497 trials, the latter signaling a technical problem (six for MTurk, three for Prolific, two for Student), (d) inconsistent reports between the two MUQ (three for MTurk, none for prolific, one for Student), (e) removal of participants with bot-like answers on our open-ended question for MTurk and Prolific (see below, 29 for MTurk, none for Prolific).

For the two online samples, an open-ended question was asked at the end of the GradCPT allowing us to filter out bots (Chmielewski & Kucker, 2020), “What were the instructions of the task?” Following common recommendations (Aguinis et al., 2021; Bai, 2018; Dennis et al., 2019), participants were removed if their answer to the open-ended question was either replicated word-by-word across three participants or more (e.g., three different participants reporting the exact string “SEEING THE CITY TO CLICK SPACEBAR”), irrelevant to the task, or nonsensical (e.g., “BULDING SUVERY,” “no,” “data protection”). We are thus keeping participants who gave structured answers and relevant to the task, whether reported correctly (e.g., “I had to press space when the city appeared and not press when the image was of mountains”) or incorrectly (e.g., “press the spacebar for every mountain and city shown”).

Sample

A total of 924 participants (51.84% females) were recruited: the MTurk sample consisted of 322 participants (36.96% females) between 18 and 35 years of age (M_age = 24.73, SD_age = 2.66), the Prolific sample consisted of 304 participants (39.47% females) between 18 and 35 years of age (M_age = 24.42, SD_age = 4.73), and the Student sample consisted of 298 participants (80.54% females) between 18 and 27 years of age (M_age = 20.08, SD_age = 1.68).

Power Analysis

We overall aimed at detecting effects greater than r = 0.1, which corresponds to the smallest robust effect size in psychology (Ferguson & Heene, 2021). Power analysis revealed that our sample size of N = 924 had the potential to detect a correlation of r = 0.09 for a 5% α and an 80% power. The power analyses plot can be found in Figure A in the online Supplemental Material.

Data Availability

All data, analyses, and study materials can be found on Open Science Framework at https://osf.io/nt7de/.

Results

The analyses below first report on the distributions of the MMI and F-MMI and then examine the associations between media multitasking and our four measures of behavior. We then turn to extreme group analyses as they are often reported in the literature. Finally, through systematic sequential ANOVAs and Bayesian model testing, MMI versus F-MMI are compared in terms of best predictor, while also estimating possible sample effects.

All our results are interpreted in terms of effect sizes. For zero-ordered correlations, we follow Funder and Ozer (2019) whereby r between 0.1 and 0.2 is considered as small, between 0.2 and 0.3 as medium, between 0.3 and 0.4 as large, and greater than 0.4 as very large. Results from t tests are interpreted in terms of Cohen’s d following Cohen (1988) whereby a d between 0.2 and 0.5 is considered as small, between 0.5 and 0.8 as medium, and greater than 0.8 as large. For ANOVAs, based on Field (2013), an η² between 0.01 and 0.06 is considered as small, between 0.06 and 0.14 as medium, and greater than 0.14 as large. Finally, the comparison of nonnested models performed with the Bayesian information criterion (BIC) was interpreted following best practices in the field, with a BIC difference between 0 and 2 to be interpreted as “not worth mentioning,” between 2 and 6 as “positive,” between 6 and 10 as “strong,” and greater than 10 as “very strong” evidence toward the model with the lower BIC (Kass & Raftery, 1995, p. 777).

MMI and F-MMI Distributions

The distributions of the MMI and F-MMI for each of the three samples are shown in Figure 1A and B. Both measures are approximately normally distributed, with the MMI being somewhat skewed to the right. The distributions of the three samples differ by their tail whereby the MTurk sample has numerically more individuals with higher MMI and F-MMI scores than the other two samples. Figure 1C depicts the Spearman correlations between the two indices across samples which are very highly related (r values around .6). Note that the correlation between MMI and F-MMI is numerically higher for the MTurk sample, r(320) = .73, than for the other two samples, Prolific: r(302) = .57; Student: r(296) = .60.

Correlations of Media Multitasking With the Four Measures of Interest

To test the hypotheses, Spearman pairwise correlations were performed in order to investigate the zero-order correlational links between media multitasking and our four measures of interest: self-reports of attention problems, motor impulsiveness and sensation seeking, plus the d′ for the GradCPT task. Figure 2 illustrates those correlations with the original MMI and Figure 3 with the novel F-MMI over the whole sample (N = 924). Within each panel of Figures 2 and 3, the data are also shown separately for each of the three samples of participants using different colors.

Self-reported attention problems, motor impulsiveness, and sensation seeking tended to increase as individuals reported greater media multitasking using the MMI; along the same line, lower d′, which are indicative of worse performance on the GradCPT, were associated with greater multitasking. All effects are of a medium effect size, r(922) = 0.20 for self-reported attention problems, r(922) = 0.24 for motor impulsiveness and sensation seeking, and r(922) = –0.21 for d′. When using the F-MMI, the effects tended to be the same or slightly smaller for the self-reported measures, r(922) = 0.18 for self-reported attention problems, r(922) = 0.23 for motor impulsiveness, r(922) = 0.21 for sensation seeking. However, for the GradCPT performance, the effects dropped to a small effect, r(922) = –0.11 for d′.

Looking at the correlations for each sample separately, these effects varied between large and very large size in the MTurk sample (r > .3) to small size (.1 < r < .2) in the other two samples. When using the F-MMI, the effects tended to be slightly smaller, an effect especially pronounced for the Prolific sample leading to very small to almost null effects (r < .1). Finally, when considering the GradCPT performance (d′), correlations were all numerically weaker with the F-MMI as compared to the MMI.

Overall, the findings support Hypothesis 1 and Hypothesis 2, but only partly support Hypothesis 3. The MMI displays the expected links with greater MMI being associated with greater self-reported behavioral problems and worse sustained attention performance in the GradCPT. Although the F-MMI shows the same trends, the numerically smaller correlations point to a lesser sensitivity than the MMI. Finally, when exploring samples separately, all correlations point in the same direction, but the effect sizes differ across samples, suggesting that sample heterogeneity in terms of origin (and not necessarily in terms of small samples) may lead to different results.

Interactions Analyses

Multiple regressions were performed in order to investigate the interaction effects of sample with MMI and F-MMI on the measures of interest. Figure 4 depicted those interactions for self-reported attention problems and the d′ and Figure 5 for motor impulsiveness and sensation seeking. Note that covariates are z scored before estimating interactions.

Globally all interactions were significant, and as expected from previous exploratory analyses, indeed, the slopes for the relationship between MMI/F-MMI with attention problems, d′ for the GradCPT task, motor impulsiveness and sensation seeking are more pronounced for MTurk than for the Prolific and Student samples. However, ΔR² (the change in the R² observed when a single term is removed) for the interactions involving attention problems and d′ are negligible. Specifically, ΔR²s are 1.3% for the interaction between Sample and MMI on attention problems, 1.7% for the interaction between Sample and F-MMI on attention problems, 0.8% for the interaction between Sample and MMI on d′, and 1.4% for the interaction between Sample and F-MMI on d′. Yet, ΔR²s for the interactions involving motor impulsiveness and sensation seeking were small. In particular, ΔR²s were 2.1% for the interaction between Sample and MMI on motor impulsiveness, 2.2% for the interaction between Sample and F-MMI on motor impulsiveness, 2.7% for the interaction between Sample and MMI on sensation seeking, and 2.9% for the interaction between Sample and F-MMI on sensation seeking.

These results show that overall, the effects are negligible-to-small when it comes to quantifying the differences of slopes between MMI and F-MMI and our measures of interest.

Extreme Group Analysis by Samples

As many previous studies have examined the effects of media multitasking for extreme groups (HMM vs. LMM; e.g., Ophir et al., 2009), we followed this procedure as well. Extreme groups in MMI or F-MMI were performed separately for each sample, given the sample of origin effect documented above. HMM/frequency-based high media multitaskers (F-HMM) and LMM/frequency-based light media multitaskers (F-LMM) were set at the higher and lower quartile of, respectively, the MMI and F-MMI distributions of each sample. As a result, the extreme group analyses (whether based on MMI or F-MMI scores) compared 80 HMM/F-HMM and 81 LMM/F-LMM for the MTurk sample, 76 HMM/F-HMM and 76 LMM/F-LMM for Prolific, and finally, 74 HMM/F-HMM and 75 LMM/F-LMM for the Student sample (see details in Supplemental Materials, Tables A and B). Figures 6 and 7 depict the Cohen’s d together with their 95% confidence intervals for the extreme groups defined by MMI and F-MMI.

For extreme groups defined by MMI, the results do confirm differences in attention problems with a large effect for MTurk (d = 0.83) and small effects for the Prolific and Student samples (d = 0.26 and d = 0.38), in motor impulsiveness with a large effect for MTurk (d = 1.21) and small effects for the Prolific and Student samples (d = 0.28 and d = 0.41), in sensation seeking with a large effect for MTurk (d = 1.29) and small-to-medium effects for the Prolific and Student samples (d = 0.44 and d = 0.61), and for d′ in the GradCPT with a large effect of MTurk (d = –0.95) and small effects for the Prolific and Student samples (d = −0.40 and d = −0.28).

Analyses from extreme groups based on the F-MMI show the overall same results for the MTurk and Student samples, but with a notable decrease in effects for the Prolific sample. The MTurk sample displays again large group effects for all self-report variables (attention problems: d = 0.86, motor impulsiveness: d = 1.12, sensation seeking: d = 1.01) and a medium group effect for GradCPT d′ (d = −0.53). The Student sample displays again small-to-medium effects across all four variables (attention problems: d = 0.34, motor impulsiveness: d = 0.43, sensation seeking: d = 0.52, GradCPT d′: d = –0.23). However, in the Prolific sample, only very small effects are observed (attention problems: d = 0.18, motor impulsiveness: d = 0.17, sensation seeking: d = 0.13, and GradCPT d′: d = –0.10).

Sequential ANOVAs for Assessing Variance Explained by Sample and by MMI

To further test Hypothesis 3, and to evaluate the usefulness of the MMI versus the F-MMI in estimating the relationship between media multitasking and behavioral problems, as well as to quantify any sample effects, sequential ANOVAs were performed including all three samples. Sample, MMI, and F-MMI were defined as predictors for attention problems, motor impulsiveness, sensation seeking, and GradCPT performance. More precisely, for each of our behavioral measures, a first model was built including sequentially Sample then MMI and then F-MMI. Then, a second model with the same independent variables but introducing F-MMI before MMI was built. Results are reported in Table 1. Those two sequential ANOVAs allowed us to quantify first the remaining variance explained by MMI once the variability of sample is considered (Table 1, column 1) and then the remaining variance explained by F-MMI once the variability of Sample and the variability of MMI are taken into account (Table 1, column 3). And vice versa, the second model allowed us to quantify the remaining variability explained by F-MMI once the variability of Sample was considered (Table 1, column 4) and the variability of MMI once the variability of Sample and the variability of F-MMI are taken into account (see Table 1, column 5).

Type-I Sequential ANOVAs η² (N = 924) Testing the Effect of Sample of Origin, Then MMI, and F-MMI in Both Order

The results show that first, the variance explained by Sample is lower than the remaining amount of variance explained by either MMI or F-MMI. In addition, the variance explained by Sample is null for the task-based measure. Overall, this suggests that differences in samples explain only little variability on the self-reported questionnaires and none for task performance.

The remaining variance explained by either MMI or F-MMI on each of our four dependent variables ranges from small to medium, with a maximum η² of 0.1% or 10% (see Table 1, columns 2 and 4). Furthermore, attention problems, motor impulsiveness, and sensation seeking are similarly captured by either MMI or F-MMI, with values varying between small-to-medium effect sizes. In contrast, d′ is better captured by MMI than F-MMI.

Importantly, these analyses reveal that once the variability accounting for the different samples and MMI is removed, F-MMI does not explain additional variability on the dependent variables (see Table 1, column 3). In contrast, once the variability accounting for the samples and F-MMI is taken into account, MMI still explains some remaining variability, with most of these effects being small reaching 4% for sensation seeking and 6% for d′ (see Table 1, column 5).

A set of nonnested model comparisons was then performed in order to confirm that a model with Sample and MMI performed better than a model with Sample and F-MMI. For that purpose, we estimated those two models for our four dependent variables and then calculated the BIC goodness-of-fit index for each of them. Then, for each dependent variable, we compared both BIC in order to quantify if the difference between the two models was meaningful. Results are displayed in Table 2. When comparing both models in terms of BIC for each dependent variable, it appears that Model 1 with Sample and MMI performs better than Model 2 with Sample and F-MMI. The magnitude of this difference was between strong and very strong for all of our dependent variables (Kass & Raftery, 1995, p. 777).

Control Analyses for the Sample Effect

The correlational and extreme group analyses point to different effect sizes in the MTurk sample, and the sequential analyses reveal that, at least when it comes to the self-report scales, some variance is explained by the sample (Table 1). As can be seen in Figure 2, participants with a high MMI score tended to be overrepresented in the MTurk sample and may have disproportionally affected the sample effects reported in the previously presented sequential ANOVAs. To examine whether this sample effect may hinge on the small number of participants with extremely large media multitasking values as seen in the MTurk sample, sequential ANOVAs were performed on the collated sample as in Table 1 after removing individuals with MMI values higher or equal to 6 in all three samples. The results are presented in the Supplemental Materials, Table C.

The variance captured by the Sample predictor remains very similar as that observed in previous analyses presented in Table 1 (only adding up to 2% more variability for self-reported questionnaires) indicating that Sample effect on self-reported questionnaires shown in the previous analyses is not easily attributable to that small cluster of high media multitaskers in the MTurk sample. Although this subsample replicates the observation that the MMI, unlike the F-MMI, accounts for most variance, it also highlights a contribution of high media multitaskers in the magnitude of the effects observed. Indeed, as compared to Table 1, the variance accounted by MMI and F-MMI is about halved (see details in Supplemental Materials, Table C).

Control Analyses for Media Use Measures: Effects of Total Media Use (Hours or Frequency) Versus Media Multitasking

Data about media use whether in hours spent in an average week or in frequency of use were also collected through the MUQ and the F-MUQ. Although, we have been mostly focused on media multitasking, it could be that media use, rather than media multitasking, drives the effects reported so far. To control for the role of hours/frequency of multitasking, sequential ANOVAs analyses are presented that remove sequentially the part of the variance explained by Sample, then by MH for the MUQ (respectively, media frequency—MF for the F-MUQ), to confirm that the proportion of captured variance by MMI (respectively, F-MMI) remains after controlling for the variability accounted by media use.

Table 3 shows that MH does not capture any variability on any dependent variables. Even if MF has numerically higher η² than MH, captured variances remain lower than 2%. Hence, the proportion of time spent on media, whether it is given as number of hours or as frequency of use, does not explain the variability in our behavioral measures. These analyses confirm that the variability accounted for by MMI and F-MMI is unique and not related to overall measures of media use, whether in terms of hours or frequency of media use.

To further establish that the key variable is likelihood of media multitasking when using a media, independently of the number of hours or frequency of media multitasking, a final analysis was run. Instead of using the MMI as originally defined by $\sum_{i = 1}^{9}\frac{m_{i} \times h_{i}}{h_{\text{total}}}$, this analysis focused on what we call $M_{i} = \sum_{i = 1}^{9}m_{i}$. As a reminder m_i is the sum of all frequencies of simultaneously using each of the other media j when using medium i. Importantly, this novel M_i index does not include either h_i, or the self-reported hour per week of the medium i, or h_total (or MH) the sum of all self-reported MH per week and is thus distinct from hours of primary media use.

Even when removing hours of use from the MMI computation, the novel M_i index shows comparable results to the MMI. Indeed, this M_i index explains most of the variability leaving to MMI very poor or null variance on the dependent variables (see Table 4, column 3). Furthermore, M_i does not capture more variance after taking into account the variance of MMI, with effects being negligible reaching 3% for sensation seeking and d′ (see Table 4, column 5).

Type-I Sequential ANOVAs η² (N = 924) Contrasting the Effect of the Number of Media Simultaneously Used (M_i) Versus the MMI

These two last results lead us to conclude that media use, as measures by self-reported hours/frequency, as measured in the MUQ/F-MUQ, does not drive the overall effects when assessing attention problems, motor impulsiveness, sensation seeking, and performance in task-based measure.

Discussion

Inconsistent findings across previous studies have left the research field in doubt whether media multitasking is indeed related to cognitive functioning and daily-life measures of attention, impulsivity, and sensation seeking. Although some studies found that HMM performed worse on sustained attention tasks (Madore et al., 2020; Ralph et al., 2015), and indicated more behavioral problems in daily life (Baumgartner et al., 2014; Baumgartner, van der Schuur, et al., 2017; Ralph et al., 2014; Wiradhany & Koerts, 2019), other studies could either not replicate these findings or found only very weak effects (Wiradhany et al., 2020; Wiradhany & Nieuwenstein, 2017).

Our collected samples included a total of 924 participants, a sample size that is sufficiently powered to detect even small effect sizes. In line with Hypothesis 1 and Hypothesis 2, we found consistent medium effects between media multitasking as measured by the MMI and our outcome measures. These findings support the idea that HMM show distinct cognitive and behavioral profiles. There was a negative association between MMI and sustained attention as measured with the GradCPT, indicating that individuals who media multitask more show lower sustained attention. This is partly in line with findings by Madore et al. (2020) and Ralph et al. (2015) and corroborates the idea that HMM have more difficulties to control their attention.

Yet, this latter effect is larger than expected as recent literature reports small-to-inexistent effects for the relationship between task-based sustained attention and multitasking with an overall small effect of 0.133 in the meta-analytic work of Parry and le Roux (2021). A more consistent and stronger effect has been reported between self-reported measure of attention and multitasking with an overall medium effect of 0.20 in Parry and le Roux (2021). Our study points to very similar effect sizes for the relationship between media multitasking and self-reported versus task-based outcomes. This lack of effect size difference between both effects may result from the larger sample size used in our study to investigate the link between MMI and task-based sustained attention, compared to the small sample sizes used in previous studies investigating that link (Parry & le Roux, 2021). Yet, we also recognize there was a large heterogeneity between the MTurk sample and the two other samples, also calling attention to differences across samples.

We found clear indications for differences across media multitasking measures. In response to recent calls to develop a media multitasking measure that is frequency-based (Uncapher & Wagner, 2018; Wiradhany & Nieuwenstein, 2017), we developed a shorter, frequency-based measure. The new F-MMI is normally distributed and highly correlated with the MMI. As expected, this new scale also correlated well with self-reports of attention, impulsivity, and sensation seeking. However, the MMI showed a stronger sensitivity to detect differences particularly in the sustained attention task. Thus, for researchers interested in differences in cognitive ability as measured with computer-based tasks, we recommend using the MMI. In general, we would also advise future studies to switch to an asymmetric, shorter, version of the MUQ, like the one presented in Madore et al. (2020) and reused in the present study. The latter statement is rooted on unpublished data of 1,566 participants from the Bavelier Lab. Their data show a very high correlation of r = 0.961 between the original, symmetric, MMI from Ophir et al. (2009) and an asymmetric MMI via the same mapping than what was used in Madore et al. (2020; see online Supplemental Materials for more information). Furthermore, the F-MMI might still be a possible alternative for researchers interested in studying the relationships between media multitasking and daily-life functioning in larger survey studies assessing a variety of different constructs since it is much shorter to administer; yet, one should remain cognizant of its weak correlation with our task-based measure of sustained attention.

Interestingly, our findings show that the differences in sensitivity between the MMI and the F-MMI depend on the inclusion of secondary activities in the MMI and not on the way the MMI is calculated. When summing all MUQ multitasking items without taking into account the hours someone engages in each media, this version still performed as well as the MMI. This indicates that asking participants how often they overall multitask with medium i is less sensitive than asking for how often they engage in specific secondary activities for each medium. There are at least two potential reasons for why it is advantageous to specifically probe secondary activities. First, due to the increased amount of items that are needed for this approach, the MMI has a larger range and potentially more variance. Second, including secondary activities might help respondents to remember specific situations in which they multitasked and thus help them to provide more accurate answers.

This finding is of practical relevance for researchers. First, it suggests that it is indeed desirable to include secondary activities for media multitasking. However, employing the adapted, asymmetric version of the MUQ seems to be sufficient which is in line with previous studies (e.g., Madore et al., 2020; Pea et al., 2012). Second and most importantly, the findings indicate that it is not necessary to ask for the self-reported hours that someone believes they spend with each media activity. A recent meta-analysis (Parry et al., 2021) has shown that correlations between self-reported and logged measures of digital media use are rather low indicating that self-reports of media use are potentially unreliable. Moreover, in line with previous studies, the frequency of engaging in media per se was not related to any outcome measure (Baumgartner et al., 2014). It thus seems that the way we use media, and in particular how many media we use at once, is more predictive of cognitive control abilities than the frequency or hours of media multitasking we engage in (Wilmer et al., 2019). This could be a sign that the activity of media multitasking itself, that is, the switching between various media activities, interferes with the development of cognitive control abilities more than single media use as initially proposed by Ophir et al. (2009). At the same time, it could be a sign that individuals with lower cognitive control capacities are more prone to use several types of media simultaneously, possibly owing to difficulties focusing on a single media activity.

Our findings also support the notion that previous inconsistencies in the research field might be at least partly due to sample heterogeneity across studies. Although in the present study, we employed consistent outcome measures across the three samples, we nevertheless found differences in the associations between media multitasking and behavioral outcomes across samples. For example, we found stronger effects for the MTurk sample which has been used in several previous studies on media multitasking (e.g., Ralph et al., 2015). Why these correlations were stronger in that sample, remains unclear. One potential explanation is that the MTurk sample is more diverse and that there are more participants with extreme media multitasking scores. However, even after removing these extreme media multitaskers from the data, the differences in samples remained.

Two other explanations for these differences across samples are worth considering. First, there might be unique differences in media multitasking across samples. Although the distribution of media multitasking was rather comparable across samples, it could still be that what constitutes media multitasking varies across samples. For example, in a social network analysis of media multitasking variations, Wiradhany and Baumgartner (2019) found that although media multitasking behaviors were similar across samples, some media multitasking combinations occurred more in some samples than in others. It could thus be that the associations between media multitasking and outcome measures are affected by specific media multitasking combinations more than by others. Second, the three samples might differ in other characteristics, such as demographics, educational background, and specific traits, which might function as moderators for the effects. For example, a recent study by Matthews et al. (2022) found that age across the life span constitutes an important moderator in this relationship. Thus, a major concern for future studies is to identify the characteristics of samples that could function as potential moderators in the relationship between media multitasking and attention.

Meta-analytic work indicates that the relationship between media multitasking and self-reported outcome measures tends to be of greater strength than that between media multitasking and task-based measures (Parry & le Roux, 2021; Wiradhany & Koerts, 2019). Possible explanations come from noting that self-reports assess everyday functioning, whereas cognitive tasks assess maximum performance during a brief task (e.g., Parry & le Roux, 2021; Toplak et al., 2013). Thus, it could be that even those individuals, who show substantial attention problems in their everyday life, are able to focus for a limited amount of time when asked to do so. Moreover, the stronger correlations in self-reported outcomes could be due to common-method variance between the media multitasking measure and the self-reports of attention. Notably, in the present study, we found such an asymmetry in the strength of the relationship only for the F-MMI, but not the MMI. For the MMI, medium size correlations were noted for both types of outcomes. This state of affair highlights the importance of the measurement tool used to evaluate media multitasking; from this point of view, it may be preferable for researchers to continue using the asymmetric MMI as we and others have begun to do in the literature.

Limitations

This study provides evidence for a relatively robust negative link between media multitasking and attention. Nevertheless, there are a few limitations of the present study that deserve attention. First, we found striking differences in the strengths of relationships across the three samples, with the MTurk sample diverging the most from the other samples. To better understand why these differences emerged, future studies should assess further characteristics of these samples (e.g., socioeconomic status, educational background, cultural differences, motivations). Controlling for these differences across samples might have explained the differences between the MTurk and the two other samples (Prolific and Student). However, in the present study, we were not focusing on differentiating populations, rather, our aim was to compare across platforms of recruitment (MTurk, Prolific, university) and, in particular, to address the possibility that online platforms like MTurk provide lesser quality data. Had we only collected a MTurk and a university sample, one may have concluded that recruitment via an online platform leads to differences as compared to recruitment through the student body of a higher education institution. Yet, our findings clearly show that the Prolific and university samples are quite comparable, suggesting special caution when using MTurk as a recruitment platform in future studies. Furthermore, this work highlights the importance for future research to identify clear characteristics of samples that may explain differences in the strength of the link between multitasking and attention.

We acknowledge the fact that we are only focusing on one single cognitive task, the GradCPT. Many other performance-based measures of sustained attention exist (Ralph et al., 2015). As different tasks may extract different sources of variance, it would be interesting in future studies to relate the latent construct of sustained attention as measured by a battery of such tasks, rather than focusing on one task as we do here.

Finally, the most important shortcoming is that after more than a decade of research on media multitaskers and their unique cognitive and behavioral profiles, the question of the causal nature of this relationship is still unanswered. In today’s media landscape in which we have ubiquitous access to a wide variety of media, it is crucial to understand the effects of being constantly involved with media technologies. However, only very few studies to date have tried to examine the effects of media multitasking in experimental studies. Although there are theoretical reasons to believe that media multitasking has an impact on cognitive functioning (Baumgartner, van der Schuur, et al., 2017; Ophir et al., 2009), it could also be that media multitasking is a behavioral manifestation of a preexisting attentional control failure. Engaging in several media activities simultaneously could indicate a tendency to being distracted, to prefer higher levels of stimulation, and a disability to control impulses (e.g., checking push notifications immediately even when engaging in other activities). Thus, media multitasking in itself might be an indication of problems in attention and cognitive control, rather than affecting these control patterns. Of importance, the current work highlights again that media usage whether total hours or frequency is not the driver of that negative link—rather how many media one is typically engaged with captures all of the unique variance. Future research on the causality of this relationship is direly needed.

Supplementary Materials

Copyright © 2023 The Authors

Received October 6, 2022
Revision received February 3, 2023
Accepted February 13, 2023