Abstract

As video game play has increased in popularity, so too have reports that a subset of individuals play games in a way that causes negative consequences to their lives, which has resulted in the proposed inclusion of internet gaming disorder (IGD) as a formal mental health diagnosis. Given the mass popularity of video games, it is critical that screening materials for this proposed disorder are sufficiently sensitive to ensure that individuals who suffer harm are identified, while those who do not are not mislabeled as such. Here, we examined the extent to which participants’ responses to a typical IGD questionnaire predicted academic behaviors that could be associated with harm. We recruited 42 college students and tracked their gaming and studying habits weekly over the course of one semester, taking particular note of weeks in which participants did or did not have exams. We predicted that college students overall would spend less time engaging with video games and more time with academics when they had exams, but such modulations would be smaller among individuals with more initial IGD symptoms (i.e., the “more IGD symptoms” group, as compared to the “fewer IGD symptoms” group, would show loss of control). We did not find that college students overall spent less time engaging with video games and more time with academics when they had exams, but post hoc effect size analyses indicated that our study was underpowered. Implications of potential results using this methodology are discussed and estimates of powerful sample sizes are provided.

Keywords: sensor data, internet gaming disorder, academic, addiction, diagnostic

Acknowledgments: The authors thank Yolanda L. Jones, NIH Library, for article editing assistance; the members of the NIH/NIDA Technology and Translational Research Unit for article review and commentary; Marina Kheyfets, George Washington University Milken Institute School of Public Health, for reproducing our analyses and aiding with data processing; Lauren Anthony, University of Wisconsin-Madison Department of Psychology, for analysis and graphic assistance; and Hanna Klecka, University of Wisconsin- Madison Department of Psychology (currently employed at Illumina, Inc.), for study recruitment and procedure assistance.

Funding: This study was funded in part by the Intramural Research Program of the National Institutes of Health (NIH), NIDA. Data collection was performed at the University of Wisconsin—Madison and analysis and writing was conducted at the National Institute on Drug Abuse (NIDA).

Disclosures: There are no perceived or potential conflicts of interest to report.

Open Science Disclosures: The data are available at https://osf.io/ht5p7/ The experimental materials are available at https://osf.io/ht5p7/. The preregistered design (transparent changes and data exist notation) is available at https://osf.io/ht5p7/.

Correspondence concerning this article should be addressed to Zachary Demko, Department of Psychological and Brain Sciences, University of Iowa, G60 Psychological and Brain Sciences Building, Iowa City, IA 52242, United States. Email: [email protected]

Since the invention of the first consumer home video gaming consoles in the early 1970s, playing video games has become one of the world’s most popular pastimes (Ivory, 2015). In 2021, there were an estimated 3 billion video game players in the world (Newzoo, 2021), up from 2.3 billion in just 2018 (Newzoo, 2018). As video game engagement has increased, so too have concerns that they may be “addictive” for a subset of individuals who engage with them (Brunborg et al., 2014; Dowling & Brown, 2010; Richtel, 2017; Young, 1998). To catalyze investigation of this possibility, the Diagnostic and Statistical Manual of Mental Disorders—5th edition and its text revision (DSM-5-TR) identified internet gaming disorder (IGD) as a condition warranting further research (American Psychiatric Association, 2022).

The DSM-5-TR proposed nine possible criteria for IGD, with those criteria chosen to parallel the sole behavioral addiction currently recognized in the DSM-5-TR: gambling disorder (e.g., the IGD criterion “withdrawal symptoms like sadness, anxiety, and irritability when gaming is taken away or not possible” parallels the gambling disorder criterion “restless or irritable when trying to cut down or stop gambling”). This was followed by recommendations to collect data on the utility of these proposed criteria (Petry et al., 2014). Indeed, while it is fairly well established that some small proportion of people who engage with video games experience symptoms similar to those exhibited by individuals who meet diagnostic criteria for gambling disorder (Andreassen et al., 2016; Brunborg et al., 2014; Darvesh et al., 2020; Dowling & Brown, 2010; Petry et al., 2015), more research is needed to establish that the proposed criteria are specific (i.e., they are associated with characteristics of a disorder and not with innocuous characteristics) and valid.

The DSM-5-TR proposed the following criteria for IGD: (a) preoccupation with gaming, (b) withdrawal, (c) tolerance, (d) persistence or an inability to stop playing, (e) continuing to game despite problems, (f) deception about time spent gaming, (g) use of gaming to relieve negative moods (“escape”), (h) risking relationships or academic or job opportunities due to gaming, and (i) displacement or loss of interest in other activities (American Psychiatric Association, 2022). As these criteria are stated in nonspecific, general terms, Petry et al. (2014) described an international consensus for how to word potential assessment questions. Following these guidelines, researchers have subsequently developed several scales (e.g., Pontes & Griffiths, 2015), many of which include multiple questions to assess the same criterion (e.g., Király et al., 2017; Pontes et al., 2014; Rehbein et al., 2015). The present study uses a scale derived from the widely cited Gentile (2009) scale and adapted upon the release of the DSM-5 following Lemmens et al. (2015). It assesses the nine DSM-5-TR criteria with 13 questions. Two questions each were meant to assess “preoccupation,” “continuing despite problems,” and “displacement,” and one item assesses each of the remaining criteria. One question assesses financial problems related to video game engagement.

The general population of those with high video game engagement may be a suitable community in which to assess the proposed criteria using these scales. Research in clinical psychology broadly provides compelling evidence that symptoms associated with most mental disorders occur along a continuum throughout the broader population (Krueger et al., 2018). In support of a broad population dispersal of symptoms in the context of IGD, surveys have tended to find around one-third of the overall gaming population indicating at least one criterion (Przybylski et al., 2017), with higher proportions among those who engage more regularly with video games (Luo et al., 2022; Mihara & Higuchi, 2017; Pontes et al., 2014). Furthermore, the relative frequency of criteria tends to distribute similarly in both clinical and nonclinical samples; for example, both groups tend to report the “using games to escape from negative moods” criterion with high frequency relative to the other criteria and the “deception” criterion with low frequency relative to others (Carlisle, 2021; Kim et al., 2016; Pontes et al., 2014). This suggests that those who meet the clinical threshold for IGD are at the extreme end of a continuum that includes subclinical symptoms distributed through the general population. Endorsement of IGD criteria may thus be (a) associated with symptoms along a continuum (i.e., indicating more criteria is associated with more severe symptoms) and (b) common enough to merit study among a subclinical sample of those who engage regularly with video games. If higher (vs. lower) scores on scales assessing the IGD criteria are associated with substantive problems among a population of those with regular video game engagement, this would serve to validate them by demonstrating their specificity to problematic, and not innocuous, video game engagement.

The kinds of “substantive problems” that would serve to validate the criteria can be inferred from the DSM-5-TR definition of a mental disorder and the proposed classification of IGD as, more specifically, an addiction. The DSM-5-TR defines a mental disorder as “clinically significant disturbance in an individual’s cognition, emotion regulation, or behavior … usually associated with significant distress or disability in social, occupational, or other important activities” (American Psychiatric Association, 2022). Some research has found that the criteria for IGD are indeed associated with “disturbance,” “distress,” and “disability” in such manifestations as aggression, problems in social and occupational performance, and comorbid attention-deficit/hyperactivity disorder, depression, and anxiety symptoms (Andreassen et al., 2016; Brunborg et al., 2014; Darvesh et al., 2020; Müller et al., 2015; Ostinelli et al., 2021). Billieux et al. (2015) proposed though that to demonstrate the validity of IGD, its criteria must identify both functional impairment and stability of dysfunctional behavior over time. Indeed, other research has found that the criteria neither distinguish between high recreational video game engagement and functionally impairing play, nor are stable over time (Deleuze et al., 2017; Przybylski et al., 2017; Scharkow et al., 2014).

The DSM-5-TR proposed classification of IGD as an addiction implies further means of validating its criteria. The National Institute on Drug Abuse defines addiction to include continuing “use despite consequences” (Volkow, 2020), and Sussman and Sussman’s (2011) synthesis of literature on the concept of addiction identifies “loss of control” and “suffering negative consequences” as core elements. Indeed, loss of control is one of the nine proposed criteria for IGD (i.e., “inability to reduce playing”) and is operationalized in nearly every IGD scale (e.g., “have you ever felt you could not stop playing video games?”). What remains to be elucidated though is the extent to which self-reporting losing control is associated with true functionally impairing behavior. As an analogy, one of the symptoms of major depressive disorder is change in weight or appetite (American Psychiatric Association, 2022). If researchers found that self-reported change in appetite was unrelated to any observable changes in true eating behavior, it would suggest the need to rethink the criteria (or at minimum to shift to “perceived change in appetite” instead of “change in appetite”). Accordingly, we reasoned that if, among a population with high engagement with video games, higher versus lower scores on scales assessing IGD are associated with an observable loss of control over gaming engagement in such a way that could harm life functioning, this would provide a strong validation of the proposed criteria and the current conceptualization of IGD. This validation would be strengthened if, in line with proposals from Billieux et al. (2015), loss of control was observed to remain stable over time.

College students are a suitable population to test the proposed IGD criteria, particularly with respect to the loss of control. College students have a newfound autonomy and independence and spend significant portions of their time using electronic screens for recreation (Ohayon & Roberts, 2021). Symptoms of “addiction” to gaming are prevalent in this population (C. H. Ko et al., 2014; Stockdale & Coyne, 2018), especially in the United States (Tang et al., 2018). In addition, generally much of the lives of college students are directed toward earning good course grades. In keeping with the “functional impairment” standard of Billieux et al. (2015) and the definition of a mental disorder, studies have found an association between IGD criteria and problems in academic performance among college students (Fung et al., 2021; Mihara & Higuchi, 2017; Schmitt & Livingston, 2015; Wang et al., 2014). Using the IGD criteria to tie these problems in academic performance to specific functionally impairing habits of engagement with video games would contribute compelling evidence to their validity.

As such, our primary goal was not to examine academic performance directly; rather our goal was to directly examine loss of control over gaming as a possible mechanism for impairment in academic performance. We considered that self-reporting IGD criteria should theoretically hold the strongest ties with true video gaming behavior, which in turn could result in more distal impairment in academic performance. As numerous factors are involved in determining college students’ academic performance (e.g., differences in cognition, personality, class topic, grading criteria, etc.), one might expect the relationship between IGD and overall academic performance to be somewhat weak. The association with true gaming behavior, though, is more proximal to IGD and therefore should be much stronger. Continuing the earlier analogy to the “change in appetite” component of the criteria for major depressive disorder, numerous factors are involved in determining one’s weight other than appetite, so one would expect the proximal link between depression and appetite to be stronger than the distal link between depression and weight. Comparably, we examined whether IGD was related to the proximal (and theoretically strongly linked) outcome of true gaming behavior, in ways that could theoretically impact the more distal (and weakly linked) outcome of academic performance.

To summarize, we examined the effectiveness of the proposed DSM-5-TR criteria for IGD in identifying patterns of engagement with video games that may cause impairment in academic performance (i.e., stable functional impairment resulting from addiction like engagement with video games). We did so via a longitudinal design wherein regularly gaming college students, some of whom had initially indicated a larger number of IGD symptoms and some of whom had initially indicated a smaller number of IGD symptoms, provided weekly reports about their study and gaming habits. Specifically, we sought to take advantage of the fact that academic workload for most college students is not constant across the semester, but instead fluctuates based upon exam schedules in their classes. It is thus necessary for college students to show purposeful and time-dependent control over their behaviors. We hypothesized that the students overall would spend more time working on academics, and less time engaging with video games, during weeks in which they had exams. However, we hypothesized that students who indicated more IGD criteria would experience a loss of control over their gaming, which we could observe as equivalent video game engagement during exam weeks as nonexam weeks. Further, this would displace increases in academic work during exam weeks. In other words, we surmised that greater numbers of IGD symptoms would be associated with an impairment in “normal” college studying habits. The two core IGD-specific hypotheses we tested were therefore:Hypothesis 1a: In comparison to nonexam weeks, college students overall will spend more time engaging with academics and less time engaging with video games during exam weeks. Hypothesis 1b: For college students who score higher on IGD (“more IGD symptoms” group), the amount of time spent engaging with academics and video games will not change as much between exam and nonexam weeks as for those who score lower on IGD (“fewer IGD symptoms” group).

Furthermore, while not specifically related to IGD per se, there is considerable interest across essentially all domains studying video game use as to the accuracy of self-report measures (Parry et al., 2021). We therefore sought to validate the self-report measurements of video game play time by comparing them to logged console measurements. Our second hypothesis thus speaks both to the utility of the self-report measures utilized to test the hypotheses above, as well as to the broader issue:Hypothesis 2: Self-reported measurements of the amount of time spent engaging with video games will be moderately correlated with logged measurements, and on average will not be significantly different.

To summarize, research is needed to examine the validity of the proposed DSM-5-TR criteria for IGD. One way to test their validity is to examine whether they are related to functionally impairing habits of video game engagement. We sought to do so among college students. If our hypotheses are supported, this would demonstrate validity of the proposed DSM-5-TR criteria and help more precisely define the behavioral structure of internet gaming disorder. Alternatively, if they are not supported, this may indicate a need to search elsewhere for observable (rather than purely self-reported) “loss of control” and “impairment” in IGD (e.g., displacing social engagement rather than academic), as if such features are not convincingly tied to video game engagement, this may call into question the IGD criteria.

Method

The study was preregistered prior to beginning analysis. The preregistration, all study materials, and analysis code are available online under an umbrella Open Science Framework project at: https://osf.io/ht5p7/

To aid in finding specific components, the preregistration and study materials are available at https://osf.io/mxy82, data are available at https://osf.io/9pe5k, and analysis code is available at https://osf.io/tdb6s/

Participants

This study used a quasiexperimental design to track gaming engagement and academic/study habits over the course of an academic semester. Participants were recruited among students enrolled in introductory psychology courses at a large Midwestern public university, and data collection was done in two waves, Fall 2020 and Spring 2021 (all sample sizes combined below). An initial screening survey was sent to 1,610 students who enrolled in the classes. All 564 people who passed our screening survey (see “Screening Survey” below) were invited to participate in a semester-long study (given that it was a priori unclear how large an effect on time displacement could be expected, our disposition was simply to recruit as large a sample as was possible—that is, to attempt to enroll all eligible participants. We discovered small effect sizes, making our study underpowered but informing potential future work). Of the 564 that were contacted, 107 consented and initially enrolled (see Figure 1, Fall 2020: N = 59, Spring 2021: N = 47). The demographics of the final sample (i.e., those who completed all study procedures) are listed in Analysis. All study procedures took place online, complied with American Psychological Association ethical standards for the treatment of human participants, and were approved by the University of Wisconsin—Madison Education and Social/Behavioral Sciences Institutional Review Board. All surveys were administered to participants and data were generated using Qualtrics (https://www.qualtrics.com/).

Procedure

Participants were first screened at the beginning of each semester (Fall 2020 and Spring 2021) using a Video Game Habits question and the 13-item Internet Gaming Disorder Scale (adapted based on the work of Gentile, 2009 and Lemmens et al., 2015). Participants who reported playing video games at least “a couple times a month” on the Video Game Habits question were invited to enter our 10-week assessment study.

In the 10-week assessment study, participants were sent a first-week survey, eight short weekly surveys, and a final week survey. These surveys assessed when participants had exams and how much time they spent engaging with video games and academics in each past week. The schedules of the weekly surveys were decided on a participant-by-participant basis according to their exam schedules. In order to capture as much time leading up to exams and as little time following exams as possible (because participants could increase game engagement following exams which would distort the results), surveys were sent out during the evenings of the days on which participants had exams the most frequently. For the first and final weekly surveys, we requested logged console measurements of gaming engagement. Participants were asked to complete their surveys on the same day that they received them, and those included in our Hypothesis 1 analyses did so for 48% of their surveys and completed an additional 32% of surveys the following day.

Surveys

Screening Survey

Video Game Habits Question

The Video Game Habits question assesses frequency of engagement with video games (“How often do you play video games?”) with a 1–8 Likert scale (“almost every day,” “about 4 or 5 times a week,” “about 2 or 3 times a week,” “about once a week,” “a couple of times a month,” “about once a month,” “less than once a month,” and “I never play video games”).

Internet Gaming Disorder Scale

We adapted this 13-item scale based on Gentile (2009) and Lemmens et al. (2015) and used it to assess the severity of proposed IGD criteria in terms of preoccupation, withdrawal, tolerance, persistence, continuing despite problems, deception about gaming, escape from negative moods, conflict to other areas of life, displacement of other activities, and the additional construct of financial problems. Participants could respond yes, no, sometimes, or do not know. Following the recommended coding scheme by Gentile (2009) and to capture severity with more granularity, we coded yes as 1, no as 0, sometimes as 0.5, and do not know as 0, and summed each participant’s score (see similar practices in Pontes et al., 2014 and Pontes & Griffiths, 2015). Multiple questions that assessed the same criterion were each added to a participant’s total IGD score (as opposed to condensing them into a single present/absent score for that criterion). This approach has the advantage of increasing the range and variation in IGD scores. Standard approach would consider participants to surpass the threshold for clinically significant impairment at half or more (seven) items, which only one participant in the “more IGD symptoms” group did (Gentile, 2009). See Table 1 for the full scale and the frequency of endorsement of each item in both the “fewer IGD symptoms” and “more IGD symptoms” groups.

First Week Survey

Demographics

Participants were asked to indicate their age, gender, and ethnicity.

Logged Console Measurements of Time Spent Engaging With Video Games

To obtain logged measurements of time spent engaging with video games, participants were first asked to indicate which gaming devices they used and whether multiple people used their gaming accounts/devices, and then received instructions to navigate to their gaming device’s measurement of their gaming hours, take a screenshot, and upload it. Gaming devices consisted of Xbox, PlayStation, computer games, Wii U, and Nintendo Switch. Engagement with phone games was assessed separately using the Phone Gaming Survey (see below).

Xbox

Xbox tracks total play time for each video game. As such, we asked participants to submit screenshots depicting console measurements of playtime of each of the top five games they expected to play the most over the semester during the first week of the study and then to submit the information for those games again during the final week. We additionally asked participants to let us know if they began playing any new Xbox games aside from their top five and to submit corresponding playtime screenshots.

Playstation

Playstation sent participants regular email newsletters that sometimes included a total playtime measurement. This was the least reliable measurement due to the unpredictability of whether these newsletters included playtime.

Computer Games

Computer gamers were all asked to route their games through Steam so that it would record playtime and submitted total Steam playtime measurements during the first and final weeks of the study.

Wii U

Wii U tracks total playtime for each video game; as such we followed the same method as for Xbox.

Nintendo Switch

Nintendo Switch included a monthly playtime summary; during the final week we asked participants to submit screenshots of these summaries for each of the past 3 months (the duration of the study).

The popularity of each gaming medium (including consoles and phones) is depicted in Table 2, the top 10 most popular video games are depicted in Table 3, and the number of unique video games participants played is depicted in Table 4.

Syllabi

Participants were asked to upload copies of all of their syllabi or course schedules, such that we could identify the days on which they had exams.

Weekly Surveys

Weekly Time Use

Participants were asked to indicate the average length of time per day over the past week that they spent working at a job, participating in organized extracurricular activities, socializing, on academics, sleeping, and engaging with video games.

Phone Gaming Survey

Unlike other gaming devices, iPhone, and Android phones keep track of gaming hours on a 10-day and weekly basis, respectively. To gather logged measurements of time spent engaging with mobile games, participants who reported playing mobile phone games were asked to navigate to their phone settings and submit a screenshot of the logged time spent engaging with games over the past 10 days (for iPhones) or the past week (for Android phones). Participants received these surveys regularly according to these respective intervals.

Final Week Survey

The final weekly survey included the same materials as the weekly surveys, with the addition of the participants’ expected final grades for each course (note that participants submitted these surveys prior to receipt of their final grades), end-of-study logged console measurements of time spent engaging with video games, and a question assessing whether other people used the participants’ gaming consoles during the study (e.g., roommates or friends).

Analysis

All analyses were conducted using R (v4.2.2, R Core Team, 2022) and graphics were created using the R package “ggplot2” (Wickham, 2016).

Deviations From Preregistration

Our preregistration stated that we would conduct a multivariate analysis of variance (MANOVA) and two exploratory analyses of covariance (ANCOVAs). However, as part of the review process, it was indicated that linear mixed-effects models would be preferable for the main body text for advantages in interpretability. Results from our preregistered MANOVA and ANCOVAs and exploratory MANOVAs are now reported in Supplemental Material 2. Results across approaches aligned. Additionally, we deviated from the preregistration to exclude isolated outliers (i.e., one in the Hypothesis 1 analyses and two in the Hypothesis 2 analyses) which substantially altered the structure of the data. Finally, our preregistered Hypothesis 1 hypothesis was separated into two components (Hypothesis 1a and Hypothesis 1b) for comprehension and clarity.

Hypothesis 1

Only 43 participants completed all 10 weekly surveys (noting that these data were collected during the largely online COVID-19 pandemic year, where, anecdotally at least, attrition rates, particularly for multisession/longitudinal designs were seen to be higher than was previously expected; e.g., Parong et al., 2022). We further excluded one outlier (which was 4.67 standard deviations from the mean time spent engaging with video games), which resulted in a final sample of 42 participants included in the Hypothesis 1a and Hypothesis 1b analysis. Demographic information for these 42 participants is reported in Table 5. See Supplemental Material 1 for Supplemental information about data cleaning.

About one-third of participants (36%; 19 in the “more IGD symptoms” group and 19 in the “fewer IGD symptoms” group) reported engaging with video games on their mobile phones. We created an exploratory Sankey diagram using the R package networkD3 (Allaire et al., 2017) to visualize the stability of past-week IGD symptoms within each participant over 10 weeks (Figure 2; one participant who did not complete the relevant surveys was excluded for a sample size of 41 participants).

We divided participants into “more IGD symptoms” and “fewer IGD symptoms” groups using the screening survey. We employed a median-split approach, to contrast individuals who endorsed “more than the median number of symptoms” with individuals who endorsed “fewer than the median number of symptoms.” We employed a median-split approach (with relatively fewer group differences than may exist with a more standard clinical cut-off approach) due to sample size limitations in this exploratory study and to derive conservative effect size estimates. There were accordingly slight differences in median cutoffs between waves (i.e., Fall 2020 threshold: “more IGD symptoms” group was ≥ 3.5 symptoms and “fewer IGD symptoms” group was ≤ 3 symptoms; Spring 2021 threshold: “more IGD symptoms” group was ≥ 3 symptoms and “fewer IGD symptoms” group was ≤ 2.5 symptoms). This resulted in 21 “more IGD symptoms” and 21 “fewer IGD symptoms” individuals. Critically, these participants were almost entirely below the threshold for clinically significant impairment (≥7 using our scale). However, the “more IGD symptoms” (M = 4.38) and “fewer IGD symptoms” (M = 1.57) groups were significantly different in their baseline IGD symptoms (df = 40, t = 8.35, d = 2.64, p < .001), and 64% of the between-person variance in IGD symptoms was attributable to the group to which participants were assigned (Figure 3), indicating that most of the variation in IGD is attributable to group assignment rather than individual differences. Table 2 depicts the frequency of participants in both groups endorsing each IGD criterion. Participants in both groups engaged with video games with similar frequency, around 3−4 times per week.

To test Hypothesis 1, we conducted two linear mixed-effects models using the R package “lmerTest” (Kuznetsova et al., 2017). To adjust for covariance between time spent engaging with video games and academics (e.g., one might expect that as gaming increases, studying decreases), we included each of these dependent variables as a predictor while modeling the other as an outcome. As such, in each model, we estimated the unique effects of our predictor variables (i.e., IGD group and exam presence), while holding the other outcome variable equal (i.e., holding equal either time spent engaging with video games in the model with academics as the outcome, or academics in the model with gaming as the outcome). We modeled individual participants as random effects, to account for between-person differences in overall time spent gaming or studying.

Our first model, designed to address our research question regarding loss of control over engagement with video games, was specified as follows:

With IGD group, exam presence, their interaction, and time spent engaging with academics all modeled as fixed effects, baseline individual variation modeled as a random effect, and time spent engaging with video games as our outcome. Our second model assessed whether loss of control over video gaming resulted in functional impairment, operationalized as reduced time spent engaging with academics:

With IGD group, exam presence, their interaction, and time spent engaging with video games all modeled as fixed effects, baseline individual variation modeled as a random effect, and time spent engaging with academics as our outcome. Because we found significant effects of time spent engaging with academics in predicting time spent engaging with video games and vice versa, we examined whether IGD group moderated these effects in additional exploratory analyses. Specifically, we conducted two post hoc linear mixed-effects models specifying only the interactions between IGD group and one dependent variable in predicting the other dependent variable. These are detailed in Supplemental Material 3.

In post hoc power analyses, we found that our exploratory sample sizes did not afford sufficient power to detect statistical significance in effect sizes of the observed magnitude (and, correspondingly, preclude a strong argument for null results). As such, our study crucially lacked statistical power. We conducted additional power analyses to determine the sample sizes needed for future work to detect significance in effect sizes of the magnitude observed in our data. We conducted these power analyses using the R package “simr” (Green & MacLeod, 2016).

Hypothesis 2

The analysis testing Hypothesis 2 differed from that testing Hypothesis 1. As such, the participant data included in these analyses differed from those of Hypothesis 1. This data set included every participant who completed at least five weekly surveys and submitted screenshots of their consoles’ logged measurements of gaming hours (46 participants, though two were excluded, see below). All logged gaming hours screenshots were manually coded by author Zachary Demko and other research assistants. For consoles that collected playtime data at the level of individual games, we calculated the sum playtime of all measured games. We subtracted first-week measures from last-week measures to calculate the logged time spent engaging with games over the course of the study. We imputed any missing weekly self-reported gaming hours measurements (8% of total remaining data) based on the average from each participant’s existing self-report data. We removed self-reported measurements from weeks that did not overlap with console measurements (e.g., the first week of surveys captured time spent engaging with video games that occurred before we began tracking console measurements). We report results from analyses excluding data from two outliers for a final sample of 44.

To examine the accuracy of self-reported gaming hours, we conducted a Pearson correlation between study-long self-reported and logged measurements of gaming hours. We also conducted a two-tailed t test comparing self-reported and logged measurements. Our sample of 44 participants allowed us 80% power to detect an effect size of r = 0.3 or greater and a Cohen’s d effect size of the differences between logged and self-reported measurements of d = 0.6 (which, based on estimates of standard deviation from our data, amounts to an effect size of 47 hr; conducted using G*Power Version 3.1.9.7; Faul et al., 2009). In order to examine whether our results were biased due to multiple people using the same gaming console, we conducted a follow-up exploratory analysis using only data from participants who reported that no one else used their console (for an exploratory sample of 31, note that this was a separate analysis to support our main Hypothesis 2 analysis with 44 participants).

Deidentified data, survey materials, study procedures, and analysis methods and code are available on OSF at https://osf.io/ht5p7/

Results

Hypothesis 1

We did not find support for either Hypothesis 1a (that participants would spend more time academics and less on gaming depending on whether an exam was upcoming) or Hypothesis 1b (that increases in academic and decreases in gaming time during exam weeks would be more pronounced for individuals who endorsed fewer IGD symptoms than those who endorsed more IGD symptoms). That is, holding all other predictors constant, our first model found no significant effect of the presence of an exam (β = 0.07, SE = 0.13, 95% CI [−0.19, 0.33], p = .59), being in the “more IGD symptoms” group (β = 0.41, SE = 0.32, 95% CI [−0.23, 1.05], p = .21), or their interaction (β = −0.25, SE = 0.18, 95% CI [−0.61, 0.11], p = .18) on time spent engaging with video games per day in a given week. It did, however, find a significant effect of time spent engaging with academics on game playing in general (β = −0.07, SE = 0.03, 95% CI [−0.13, −0.01], p < .05), such that for each additional hour spent on academics per day, the amount of time spent gaming decreased by 4.2 min per day (i.e., β = −0.07 hr per day × 60 min = −4.2 min per day). Our second model found a significant effect of the presence of exams (β = 0.57, SE = 0.21, 95% CI [0.15, 0.99], p < .01), but no effect of being in the “more IGD symptoms” group (β = −0.07, SE = 0.65, 95% CI [−1.37, 1.23], p = .91), and no effect of the interaction between having an exam and being in the “more IGD symptoms” group (β = −0.34, SE = 0.29, 95% CI [−0.92, 0.24], p = .25) on time spent engaging with academics per day in a given week. Similarly, it did find a significant effect of time spent engaging with video games on academic hours (β = −0.18, SE = 0.08, 95% CI [−0.34,−0.02], p < .05), such that for each additional hour spent on video games per day, time spent on academics decreased by 10.8 min per day. Figure 4 displays changes in time spent engaging with video games and academics related to exams and IGD group.

Since the results were largely underpowered, we calculated the sample sizes necessary to detect statistical significance in effect sizes of the observed magnitude. To detect statistical significance in the observed increase of 4.2 min per day spent engaging with video games when students had exams, one would need 200 participants to achieve 81% power. To detect statistical significance in the observed increase of 24.6 min per day spent engaging with video games related to belonging to the “more IGD symptoms” group, one would need 150 participants to achieve 79% power. To detect statistical significance in the observed increase of 34.2 min per day spent engaging with academics when students had exams, one would need only 50 participants to achieve 84% power. Finally, to detect statistical significance in the observed decrease of 4.2 min per day spent engaging with academics related to belonging to the “more IGD symptoms” group, one would need 300 participants to achieve 83% power.

All told then, we found only that holding the other variables constant, the presence of an exam was associated with an increase of 34 min per day studying in a given week, and that as gaming increased, studying slightly decreased (and vice versa). Therefore, we did not find support for Hypothesis 1a: While students overall spent more time engaging with academics during exam weeks, they did not reduce their time engaging with video games (Figure 4); and we did not find support for Hypothesis 1b: There were not significant differences between the “more IGD symptoms” and “fewer IGD symptoms” groups in the amount of time they spent engaging with academics or video games during exam or nonexam weeks (i.e., there was not greater consistency in the “more IGD symptoms” group). In additional exploratory analyses, we also found that grades did not significantly differ between the two groups (t = −0.28, df = 15.84, p = .78).

Finally, as our results with respect to the relations between changes in gaming hours and “more/fewer IGD symptoms” group assignment based upon initial IGD score were largely null, we further explored the extent to which the low-level IGD symptoms our sample reported were stable through the semester with a Sankey diagram (Figure 2). We found that while two of the participants with initial IGD scores of 3 or greater continued to have IGD scores of 3 or greater throughout the semester, most participants moved from reporting 0–2.5 symptoms toward reporting no symptoms as the semester progressed. It appears that these low-level IGD symptoms were higher at the beginning of the semester but declined, perhaps as participants devoted more energy toward academics or developed better time-management skills. Since most participants fell below the clinical threshold, this Sankey diagram does not speak to the stability of IGD as a disorder; rather it speaks more to the stability of individual symptoms on the lower end of the spectrum of severity.

Hypothesis 2

In analyses excluding two outliers, we found a significant moderate correlation between self-reported and logged gaming hours (r = 0.56, 95% CI [0.31, 0.73], df = 42, p < .001; Figure 5). Under and overestimation seemed to be roughly equally common, as self-reported gaming hours ($\overline{X}$ = 80, SD = 64) were only off by about 5 hr compared to logged measurements ($\overline{X}$ = 85, SD = 89), and this was not a statistically significant difference (t = 0.34, df = 78.07, p = .74).

One possible source of inaccuracy in logged measurements is that multiple people may have used a participant’s gaming console (e.g., a participant’s roommate). If this were the case, the console would log more hours than an individual participant actually played. However, we found almost identical results to our initial analysis when only including those who reported that no one else used their console (n = 31). We found a moderate correlation (r = 0.57, df = 29, p < .001) and that on average self-report ($\overline{X}$ = 97) aligned almost perfectly with logged measures ($\overline{X}$ = 97) (t = −0.01, df = 54.89, p = .99).

Discussion

In this study, we examined whether college students who indicated more of the proposed DSM-5-TR criteria for IGD also had addiction like engagement with video games, specifically engagement that may cause problems in academic performance. We found that the self-reported gaming measurements upon which we relied were moderately accurate, and that over and underestimation seemed equally common. We did not find evidence for the premise of our study, that students overall reduced the amount of time they spent engaging with video games when they had exams. We found only that students spent more time engaging with academics when they had exams, and overall that as engagement in one of these activities increased, the other decreased. This precluded us from effectively examining whether students in the “more IGD symptoms” group reduced their gaming less when they had exams. In other words, we could not examine whether those in the “more IGD symptoms” group lost control over their gaming, because we could not detect “controlled” gaming. This may be partially explicable by our lack of power to detect effects of exams (power = 22%) or IGD group (power = 28%) on time spent gaming and the related generally low levels of IGD in our sample (i.e., an average score of 4.38 out of 13 in the “more IGD symptoms” group).

However, the development of the present framework for assessing loss of control due to IGD remains a contribution. It builds upon past work, for example, that has found a relation between reporting IGD criteria and experiencing problems with academic performance (e.g., C.-H. Ko et al., 2020; Ohayon & Roberts, 2021; Samaha & Hawi, 2020; Toker & Baturay, 2016). However, to paraphrase Billieux et al. (2015), it is not surprising that items developed using an addiction framework (i.e., the proposed IGD criteria) are correlated with established risk factors for substance use disorders (e.g., problems with academics; Gau et al., 2007). A stronger test of the proposed criteria is to examine whether they connect distal problems in academic performance to proximal stable, functionally impairing habits of video game engagement. Our methodology builds upon this past work by examining whether academic problems could be partially due to specific functionally impairing habits of video game engagement, habits defined with greater granularity and more clearly tied to “loss of control” than blunt correlations between IGD and increased time gaming (e.g., Grüsser et al., 2007; Hussain et al., 2012; Mihara & Higuchi, 2017). To reiterate, we sought to identify whether IGD was tied not just to increased time spent engaging with video games or a broader functional outcome such as grades, but to specific behaviors that could reflect loss of control over gaming (e.g., reduced ability to moderate gaming during exam weeks). We lacked statistical power to do so though in this preliminary study, but did draw estimates of effect sizes to aid future researchers.

One may imagine possible outcomes of this line of research. If researchers do not observe associations between self-reporting IGD symptoms and true “loss of control” such as reduced ability to moderate gaming during exam weeks, and no other forms of “loss of control” over gaming are identified, then the proposed criteria for IGD would not be specific to true behavior. If, on the other hand, future work does observe an association between self-reporting IGD symptoms and true loss of control over gaming, then returning to the analogy to depression, this would be equivalent to researchers observing an association between self-reported change in appetite and true eating behaviors. In other words, it would directly confirm the validity of the self-reported symptoms. The present methodology for observing “loss of control” also contributes toward a roadmap for the validation of any new behavioral addiction, which should, by definition, relate to observable loss of control in ways that lead to functional impairment.

Our digital tracing methods did have moderate statistical power (i.e., we had 80% power to detect a correlation of r = 0.3 or greater), and we found that over the entirety of 10 weeks, the amounts of time participants self-reported to spend engaging with video games were moderately correlated (r = 0.56) with measurements logged by their consoles. On average, self-reported measurements were very close to those logged by consoles, only off by 5 hr, indicating and under and overestimation seemed equally common. These results were similar when excluding consoles used by multiple people. In their meta-analysis comparing self-reported and logged digital media use, Parry et al. (2021) found a similar, albeit smaller correlation of r = 0.38. In a more specific examination of League of Legends players (i.e., an online multiplayer video game), Jin et al. (2022) found a correlation closer to ours of r = 0.49. One reason for the greater accuracy of self-report in the present study compared to past studies may lie in the sample population. Here, we recruited college students, who only spent around six and a half hours per week engaged with video games (i.e., the median), while Jin et al. recruited participants from online gaming forums, who may have engaged more frequently and have had more opportunity for misestimates. Jin et al. also asked participants to self-report their gaming in the past 2 weeks, while we asked about the past week, which may have been easier for participants to accurately estimate.

We have briefly touched on some limitations of this study, but they bear repeating to emphasize the nature of this study as exploratory methodology development without firm conclusions. Our study was significantly underpowered. We would need to have recruited 7 times as many participants to detect significance in our smallest effect sizes (i.e., 300 participants are needed to detect significance in our observed effect of IGD group on time spent engaging with academics). Of course, “true” effect sizes may not exist or may be insignificantly small (i.e., completely clinically irrelevant). Finding no relationship between one’s number of self-reported IGD symptoms and true video game play during exam weeks would (if the premise of overall gaming decreasing during exams weeks held true) speak directly to a lack of ecological validity of self-reporting IGD symptoms. The critical limitation with the present study is that it is unable to make strong population inferences about effects (or lack thereof) given our small sample size.

An additional reason for the lack of statistical power in this study is that we recruited a convenience sample of college students, and participants were almost entirely below the threshold for clinically significant impairment. Well-powered results (and we reiterate that ours were not) from such a sample though are arguably still informative; the extent to which they would speak to the full continuum of people with IGD symptoms would depend strongly on whether a disjunction occurs in how symptoms manifest as behavior when they reach clinical significance. If the disorder is best considered graded in nature (even if an arbitrary threshold is placed for what is and is not clinically significant), then results from a nonclinical sample can be (at least partially) extrapolated from the lower to the higher end of the continuum (with increasingly large uncertainty the farther one extrapolates). If, meanwhile, there is a stepwise or noncontinuous shift that occurs when symptoms reach a clinically significant level, while such results would be of utility when considering, for example, the typical college population, they might be less valuable for people past the clinical threshold. There is substantial empirical evidence that most common mental disorders (including gambling disorder) are continuous in nature, but disjunction in IGD cannot be ruled out with the present data (Haslam et al., 2012; Krueger et al., 2018; Latzman et al., 2020; Weinstock et al., 2017). Further, by definition, the greater the differences in IGD between the groups, the stronger the test of hypothesized group differences in IGD-related outcomes.

If, hypothetically, one were to replicate this study with a large sample that included people with high levels of IGD, an additional challenge would be posed by the possibility that functionally impairing engagement with video games may only manifest in ways other than displacing time spent studying for exams. For example, “uncontrolled” gaming may displace social obligations or sleep. Findings of specificity in functional impairment though would still provide important information about IGD. It would be highly relevant to researchers, clinicians, and students with IGD to know that the disorder specifically impacts sleep, and not exam performance (e.g., to target sleep for amelioration). As Galatzer-Levy and Bryant (2013) demonstrated, DSM classifications allow for enormous heterogeneity in disorder manifestation (there are 636,120 combinations of criteria that meet the clinical threshold in the case of post-traumatic stress disorder). It may be that there is also individual heterogeneity in the manifestation of functional impairment in IGD; the advantage then of recruiting large samples is that this approach pools various forms of functional impairment and allows for comparison with nondisordered gameplay. Past research into IGD has typically applied this amalgamated approach and has found that IGD relates distally to a wide array of functional outcomes, such as social, psychological, and occupational difficulties (Hawi et al., 2018; C.-H. Ko et al., 2020; Laconi et al., 2017; Paulus et al., 2018). As such, there may be many possible manifestations of “losing control,” and discovering some but not others would provide valuable information. We reiterate that the present data cannot speak to any possible manifestations. We did though collect data on the amount of time participants spent sleeping, working a job, engaging in extracurricular activities, and socializing, and future researchers may access these data in our registry to explore other possible forms of displacement (noting of course the limited power).

In addition, risk of IGD symptoms may vary considerably with respect to the video game genres (Mihara & Higuchi, 2017; Müller et al., 2015; Paulus et al., 2018), motivations for game play (Bäcklund et al., 2022), and social contexts of video game engagement (Hu et al., 2019; Kowert et al., 2014). Meanwhile, we summed all forms of video game engagement in our analyses, obscuring any such potential variance. We collected data on these subconstructs of interest though, and future researchers may examine them by accessing our data registry.

Further, the research was conducted during the height of the COVID-19 pandemic, which may have increased the flexibility of students’ academic environment and reduced exam-contingent changes in studying and gaming. For example, professors may have granted more individual accommodations such as extending deadlines. The exam schedules we charted at the beginning of each semester using course syllabi could have also been altered (e.g., a week-long delay in teaching could alter a course’s entire exam schedule). This would result in an incongruence between the exam dates included in our analysis and the dates when participants actually completed exams. Indeed, participants indicated whether they had an exam or other major project during their weekly surveys, which allows us to partially quantify this incongruence. Participants indicated having zero exams or major projects during 16% of weeks that we coded as having an exam, and indicated having two or more exams or other major projects during 23% of the weeks that we coded as having no exams. It is possible that participants altered the amount of time they spent gaming or studying after completing an exam and before completing a weekly survey, obscuring effects of exam weeks. However, we asked participants to report the amount of time they spent gaming or studying on average each day in the preceding week, and this averaging would partially absorb any such changes that occur on the scale of only a few (e.g., 1–3) days. Additionally, we imputed missing self-reported gaming hours in our comparison of logged and self-reported measurements and these imputations may not have accurately reflected what participants would have actually reported. Finally, our sample included only 18- to 21-year-old college students in the United States and had few women and very few Black or Native American participants. Future research should examine whether any findings of loss of control (or lack thereof) related to IGD generalize beyond the largely White, Asian, and male population of the present sample.

In conclusion, we did not find evidence that college students overall decreased the amount of time spent engaging with video games when they had exams, which precluded us from examining differences in gaming behavior related to IGD. We found only that college students spent more time studying when they had exams, and that overall, time spent engaging with video games and academics slightly displaced one another. We encourage future researchers to replicate this methodology using different (and larger) samples of participants (especially those that surpass the clinical threshold), methods of operationalizing loss of control over gaming, and IGD scales. For instance, although our scale assessing IGD shares common wording and content with most other IGD scales (e.g., Gentile, 2009; Király et al., 2017; Lemmens et al., 2015; Petry et al., 2014), replication using such alternatives would ensure results are robust. Further, it may be that loss of control is simply not well captured by gaming during exam weeks; new data using different operationalizations of “loss of control” would be necessary to demonstrate this. Our preliminary effect sizes allow future researchers to estimate sample sizes necessary to achieve at least 80% power, ranging from 200 to 300 participants to detect the smallest effects. Alternatively, larger effect sizes may be detected (and therefore smaller samples required) by recruiting more targeted participants to the “more IGD symptoms” sample, such as by only recruiting those who surpass the clinical threshold for IGD. A major contribution of the present work is our methodology operationalizing loss of control over gaming in specific functionally impairing ways, which we believe is crucial to demonstrate in ways beyond pure self-report prior to the classification of internet gaming disorder as a formal diagnosis. Adjacent to research, clinical practitioners should consider the links between stated symptoms and true behaviors; while both may cause distress, each would benefit most from a distinct intervention (e.g., changing self-perceptions in the former and changing behavior in the latter). Finally, if functional impairment due to loss of control over gaming is not observed, then possible future amendments to the criteria may benefit from qualitative work such as Petrovskaya (2022) and Carras et al. (2018) building criteria from the ground-up, beginning with people’s subjective experiences, and from parsing the extent to which perceived impairment corresponds to true behavior.

Supplemental Materials

https://doi.org/10.1037/tmb0000119.supp

Copyright © 2023 The Author(s)

Received August 29, 2022
Revision received July 19, 2023
Accepted July 21, 2023