Abstract

Almost all research on adolescents’ problematic social media use (PSMU) utilizes self-reports of symptoms. Recently, parent-report scales have been validated. Yet, in order to use parent-reports, it is imperative to understand the level of agreement between self- and parent-reported adolescents’ PSMU. This study examined agreement and discrepancies between adolescent-, mother-, and father-reported adolescents’ PSMU (assessed with the Social Media Disorder scale) on classification, the overall number of symptoms, and individual symptom level. Data from 234 Dutch adolescents aged 10–19 years and their parents (160 mothers and 91 fathers) from the Digital Family project were used. Configural invariance was established across all dyads and scalar invariance across mothers and fathers. Across adolescents and mothers/fathers, partial scalar invariance was found (one of the nine-item thresholds appeared noninvariant). Overall, we found poor agreement, but the level of agreement seems biased by the low prevalence of PSMU. Positive agreement was lower for symptoms that are harder to observe. Parental over- and underreporting compared to adolescent self-reports on the number of PSMU symptoms occurred to the same extent. Moreover, parental over- and/or underreporting were related to the number of adolescent- and parent-reported adolescents’ PSMU symptoms, adolescents’ gender, and maternal worrying about children’s social media/gaming addiction. Researchers should be aware that self- and parent-reports are not (always) interchangeable.

Keywords: adolescents’ problematic social media use, self-reports, parent-reports, agreement, discrepancies

Funding: The present study is part of the Digital Family project which is supported by funding from the Faculty of Social and Behavioral Sciences of Utrecht University, the Netherlands awarded to Ina M. Koning.

Disclosures: The authors have no conflicts of interest to disclose.

Data Availability: The data that support the findings of this study are publicly available at https://osf.io/ra2wt/. There have been no prior uses of these data.

Open Science Disclosures: The data are available at https://osf.io/ra2wt/. The experimental materials are available at https://osf.io/ra2wt/. The preregistered design and analysis plan (transparent changes and data exist notation) are accessible at https://osf.io/ra2wt/.

Correspondence concerning this article should be addressed to Suzanne M. Geurts, Department of Interdisciplinary Social Science, Utrecht University, Padualaan 14, 3584 CH Utrecht, The Netherlands [email protected]

With social network sites and instant messengers such as Instagram and TikTok being indispensable in adolescents’ daily life, addictive use of social media among adolescents is an increasing concern in society. Addictive use of social media refers to being unable to control its use resulting in continuation despite negative consequences for daily functioning (Griffiths et al., 2014; Van den Eijnden et al., 2016). As a result of the growing concern, this phenomenon is attracting increasing attention in research (La Barbera et al., 2009; Sun & Zhang, 2021). However, research on social media addiction is still in its infancy and this type of behavioral addiction has—in contrast to internet gaming addiction (IGD)—not (yet) been included in diagnostic manuals such as the Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition (DSM-5; American Psychiatric Association, 2013). Therefore, we refer to social media addiction as problematic social media use (PSMU). The prevalence of adolescents’ PSMU varies to a great extent among studies and countries (1.6%–82%; Boer et al., 2022; Cheng et al., 2021). To understand the phenomenon of adolescents’ PSMU and to help this research field to move further, a first and imperative step is to accurately identify adolescents’ who display PSMU symptoms.

Self- and Parent-Reports of Adolescents’ PSMU

So far, almost all research has used self-reports to measure adolescents’ PSMU. A frequently used self-report measure is the Social Media Disorder (SMD) scale by Van den Eijnden et al. (2016). The SMD scale is based on the addiction criteria for IGD as defined in the DSM-5 (American Psychiatric Association, 2013) such as tolerance, withdrawal, preoccupation, persistence, and conflict. The SMD scale proved to be a psychometrically sound and valid instrument in adolescent samples in 44 countries (Boer, Stevens, Finkenauer, et al., 2021). However, several concerns regarding adolescent self-reports in general have been highlighted, such as socially desirable answering tendencies (Hughes & Gullone, 2010) and the fact that adolescents’ self-reflection ability has not yet fully developed (Weil et al., 2013). In case of addiction specifically, there are additional reasons for concern regarding the reliability of self-reports. One of them is that the addiction itself might affect adolescents’ memory retrieval, as demonstrated for internet addiction (Zhou et al., 2016) and therefore might interfere with reporting accuracy. Another reason is that problem denial is a prominent characteristic of addiction (Pickard, 2016). Because of these concerns regarding self-reports, it is argued that the use of multi-informant reports should be the norm in assessing psychological disorders among children and adolescents (Comer & Kendall, 2004; De Los Reyes et al., 2015; Salbach-Andrae et al., 2009). Austermann et al. (2021) suggested that parent-reports could be promising in assessing adolescents’ PSMU and validated a parental version of the SMD scale in a German sample.

Discrepancies Between Self- and Parent-Reports of Adolescents’ PSMU

The study of Austermann et al. (2021) showed moderate agreement on the overall PSMU symptom level between adolescent- and parent-reports. This is in line with the literature on agreement between parent and adolescent ratings of adolescent psychopathology in community samples (Achenbach et al., 1987; De Los Reyes & Kazdin, 2005; Hemmingsson et al., 2017; Youngstrom et al., 2000). Besides, Austermann et al. (2021) showed that parents reported overall more PSMU symptoms than adolescents themselves. This corresponds with a study by Kewitz et al. (2021) on adolescent IGD in a clinical sample and with the study by Youn et al. (2018) on adolescent smartphone addiction; parents reported more addictive symptoms than their offspring. These discrepancies in reports can be caused by different underlying mechanisms. First, discrepancies could be a function of measurement error, such as measurement noninvariance meaning that different informants attach different meanings to the question items (Van De Schoot et al., 2015). Another mechanism may be that adolescent-report or parent-report, or both reports may be biased. One source of reporting bias could be, as shortly mentioned earlier, social desirability. For example, on the one hand, adolescents or parents may deny the presence of PSMU symptoms because they do not want to disclose socially undesirable risk behaviors (of their child). On the other hand, adolescents could exaggerate the number of symptoms as they might see it as cool to report as they are involved in risk behavior (Jeong et al., 2018). Previous studies have shown that other personal factors may also influence the accuracy of reports, such as parents’ mental well-being (Ehrlich et al., 2011; Hughes & Gullone, 2010) and adolescents’ age or gender (Brener et al., 1995; De Los Reyes & Kazdin, 2005). However, discrepancies do not necessarily reflect inaccurate or biased reporting. They could also reflect the unique perspectives that parents and adolescents provide because parents may be aware of adolescent behaviors that adolescents themselves are not aware of and vice versa (De Los Reyes, 2011). Thus, discrepancies between self- and parent-reports of adolescents’ PSMU are to be expected.

Knowledge Gaps

To the best of our knowledge, so far, the available scientific knowledge on agreement and discrepancies between self- and parent-reports of adolescents’ PSMU is limited to the previously discussed study by Austermann et al. (2021). Although the study of Austermann forms a good starting point, some relevant issues remain to be addressed in order to evaluate the usefulness of parent-reports of adolescents’ PSMU. First, measurement invariance of the SMD scale across adolescents and parents needs to be established. With investigating the measurement invariance, we can determine whether discrepancies reflect true differences between reporters, or rather are a result of the SMD scale not measuring the same underlying construct across adolescents and parents. Second, it is imperative to investigate agreement on individual level in addition to overall symptom level, since agreement may vary across symptoms. In line with the literature on internalizing versus externalizing problems (Berg-Nielsen et al., 2003; Seiffge-Krenke & Kollmar, 1998), it is likely that higher agreement between self- and parent-reports can be found for symptoms that are easily observable (e.g., conflict: serious conflicts with parents or siblings because of social media use [SMU]) than for symptoms that are difficult to observe (e.g., preoccupation, i.e., not able to think of anything else but the moment of being able to use social media again; Comer & Kendall, 2004; De Los Reyes et al., 2015; Salbach-Andrae et al., 2009). Third, it is important to make a distinction between mother- and father-reports on adolescents’ PSMU because differences in agreements are likely to be observed. Even though nowadays fathers are more involved in the upbringing of their children, mothers remain the primary caregiver (Bastiaansen et al., 2021). Therefore, we expect higher agreement between self- and mother-reports than between self- and father-reports. Fourth, it is important to explore factors related to the type of discrepancy. For example, is SMU of parents or the number of adolescents’ PSMU symptoms related to whether parents report more or fewer adolescents’ PSMU symptoms than their children. Addressing these knowledge gaps will be helpful with the interpretation and use of (a combination of) self- and parent-ratings on adolescents’ PSMU, both in research as well as for clinical purposes.

The Present Study

It is pivotal to broaden our understanding of agreement and discrepancies between self- and parent-reported adolescents’ PSMU because, as a result of discrepancies, reliance on different reporters may lead to different adolescents being identified in a given sample as displaying PSMU (symptoms). In turn, this may result in different research findings and subsequent implications for prevention and treatment. Therefore, we will expand upon the study by Austermann et al. (2021) by:1Evaluating measurement invariance between self-, mother-, and father-reports on adolescents’ PSMU; 2 Examining the level of agreement between self-, mother-, and father- reports on classification (absence/presence of PSMU), the overall number of symptoms, and individual symptom level; 3 Examining mean differences between self- versus mother- and father-reports on the overall number of symptoms, and; 4 Exploring possible associated factors with type of discrepancy (i.e., adolescents reporting more symptoms, mother/father reporting more symptoms and agreement), whereby we will include the following demographic and media-specific factors:–Adolescents’ gender, age, frequency of SMU frequency, and PSMU; – Mothers’/fathers’ age, worrying about adolescent social media/gaming addiction, SMU frequency, PSMU. Based on the previously discussed literature, we expect moderate agreement between self- versus mother- and father-reported adolescents’ PSMU. In addition, we expect that, on average, parents report more symptoms of adolescents PSMU than adolescents themselves. Besides, we expect higher agreement between self- and mother-reports than between self- and father-reports. Furthermore, we expect higher agreement between self- versus mother- and father-reports for symptoms that are easily observable and family-related (e.g., conflict: serious conflicts with parents or siblings because of SMU) than for symptoms that are difficult to observe and nonfamily related. Last, we have no prior hypotheses regarding the aforementioned possible related factors to the three discrepancy groups. Thus, that research question will be examined in an exploratory way.

Method

Participants and Procedure

For this study, we used adolescent and parent data from the third measurement wave of the “Digital Family project” (Geurts, Vossen, et al., 2022). Data from the third wave were used since parent-reported adolescents’ PSMU was only included from this wave onwards. The Digital Family project was approved by the Ethics Committee of the Faculty of Social and Behavioral Science at Utrecht University (FETC20-92). Data from the third wave were collected from May till July 2021 among Dutch families that were recruited using various means, including social media channels, newsletters of schools and sport clubs, article flyers, and word-of-mouth. Families could participate in this research project with at least one parent or main caregiver (hereinafter referred to as parent throughout the article) and one child, with a maximum of two parents and two children. Adolescents and their parents were asked to fill in an online questionnaire at home individually. At the beginning of the questionnaire, participants provided active informed consent. Active parental informed consent was obtained through the register form. Completing the questionnaire took 30 ± 45 min. Each participant received a gift card of €5, and families could win a gift card of €100.

After data collection, data were detected for careless responding by looking at response time (below 10 min), instructed response item, response inconsistency (contradicting responses on conceptually similar items), and response invariability (nonvarying answers on scales that consist of dissimilar items; see Geurts, Koning, et al., 2022, for more details). We removed participants from the data set who were flagged as careless responder by at least one of these indicators, as these indicators may detect different types of careless responding (Curran, 2016). In total, 24 adolescents and 29 parents were removed, resulting in a sample of 234 adolescents (including siblings), 160 mothers (of which one other maternal caregiver), and 91 fathers (of which six stepfathers) who completed the variables of interest. The ages of the adolescents ranged from 10 to 19 years (M = 14.12, SD = 2.22), 45.7% was boy and almost all adolescents were born in the Netherlands (98.3%). Regarding educational level of the adolescents, 21.8% was in primary school, 11.1% in prevocational education (i.e., all so-called “VMBO” levels and “VMBO/HAVO” in the Dutch educational system), 22.2% in general higher education (i.e., “HAVO” or “HAVO/VWO”), 36.3% in preuniversity education (i.e., “VWO”), 3% in secondary vocational education (i.e., “MBO”), and 3.4% in higher professional education (i.e., “HBO”) or university (i.e., “WO”). The ages of mothers ranged from 34 to 57 (M = 46.44, SD = 4.57) and the ages of fathers from 39 to 70 (M = 48.81, SD = 5.72). The majority of mothers and fathers was born in the Netherlands (94.6% and 96.7%, respectively) and finished college or university (71.8% and 74.7%, respectively).

Measures

Adolescents’ PSMU was reported by adolescents as well as their mothers and fathers using the nine-item “Social Media Disorder (SMD) scale” (Van den Eijnden et al., 2016). The nine items are displayed in Table 1. Response scales were dichotomous, indicating whether the symptom was present or not in the past year (1 = yes and 0 = no). For the analyses based on the overall number of symptoms, a sum score was calculated per informant with a higher score representing more symptoms of PSMU. For the analyses on classification level, the sum score was recoded into two groups: 0 = absence of PSMU (score < 6), 1 = presence of PSMU (score ≥ 6; cf. Boer, Stevens, Finkenauer, et al., 2021). The cutoff score of ≥6 is based on a validation study of the SMD scale by Boer, Stevens, Finkenauer, et al. (2021) using a large, nationally representative sample of Dutch adolescents and has been used repeatedly in studies using the SMD scale (e.g., Boer et al., 2022; Boniel-Nissim et al., 2022; Paakkari et al., 2021). Ordinal α for adolescent-reported PSMU was .90, for mother-reported PSMU .95, and for father-reported PSMU .93.

Adolescents’, mothers’ and fathers’ reported their own SMU frequency by answering five items about active and passive SMU activities (e.g., “How many times a day do you check social network sites?”; Van den Eijnden et al., 2018). Response options ranged from 1 (less than once a day/week) to 7 (more than 40 times a day/week). A sum score was calculated with a higher score representing more frequent SMU.

Mothers’/fathers’ worrying about social media/gaming addiction was measured asking them the following question: “Do you ever worry that your child(ren) get addicted to social media and/or games?” Response options ranged from 1 (never) to 5 (very often). Higher scores indicate more parental worrying about social media/gaming addiction.

Adolescents’ and parents’ age was calculated using their date of birth and date of participation. Besides, participants reported their gender (0 = boy/male, 1 = girl/female).

Statistical Approach and Results

Descriptive Results

Descriptive results of the study variables are shown in Table 2. The average number of adolescents’ PSMU symptoms reported by adolescents, mothers, and fathers was almost two. Adolescents used social media more frequently than their parents, and mothers used social media more frequently than fathers. On average, mothers worry as much as fathers about their children getting addicted to social media/games. Regarding type of discrepancy based on the number of reported adolescents’ PSMU symptoms, overreporting by parents occurred (about) as much as underreporting.

Measurement Invariance

To evaluate whether the SMD scale measures the same underlying construct across the different types of reporters (adolescent, mother, and father; Aim 1), we first tested configural and scalar measurement invariance in Mplus 8. Since the SMD scale consists of dichotomous items, metric invariance cannot be identified (Muthén & Muthén 2017a). Configural measurement invariance (whether the overall factor structure of the measure fits well for all reporters) was tested by running a multigroup confirmatory factor analysis in which the factor loadings and thresholds were allowed to freely vary across reporters and evaluating model fit. We used the comparative fit index (CFI), Tucker–Lewis index (TLI), the root-mean-square error of approximation (RMSEA), and the standardized root-mean-square residual (SRMR). Values of CFI and TLI between 0.90 and 0.95, values of RMSEA ≤ 0.08, and values of SRMR ≤ 0.10 indicate acceptable fit, and values of CFI and TLI ≥ 0.95, RMSEA ≤ 0.06, and SRMR ≤ .08 indicate good fit. Scalar invariance (whether the item thresholds are equivalent across reporters) was tested by constraining both the factor loadings and the thresholds to be equal using the default model settings. We evaluated scalar invariance by comparing the model fit of the scalar model with the model fit of the configural model. A reduction in CFI of not more than 0.010 and an increase in RMSEA by not more than 0.015 implies scalar invariance (Chen, 2007; Cheung & Rensvold, 2002). When scalar invariance is demonstrated, we can compare means of different reporters and be confident that any differences are not due to adolescents and parents interpreting and responding to the SMD items in a different way. When scalar invariance is not demonstrated, partial scalar invariance was tested to identify the source(s) of noninvariance by releasing constraints on one or more item thresholds. When an item is identified as noninvariant regarding thresholds, this means that there are differences in the average item responses between reporters that are not due to differences in the mean score on the latent variable. Weighted least squares mean and variance adjusted estimator was used, as the indicators of PSMU are dichotomous (Rhemtulla et al., 2012). Results are displayed in Table 3.

Adolescent–Mother–Father

Analyses were first performed for adolescents, mothers, and fathers simultaneously. Except for SRMR, all fit indices indicated good fit of the configural model providing evidence that the same factor structure for the SMD scale holds across adolescents, mothers, and fathers. ΔCFI and ΔRMSEA showed that constraining factor loadings and thresholds to be equal across all reporters significantly worsened model fit, meaning that scalar invariance was not established. In order to examine which dyads are responsible for the scalar noninvariance, subsequently, analyses were performed for each dyad separately.

Adolescent–Parent

For both the adolescent–mother and adolescent–father dyad, all fit indices except SRMR demonstrated a good fit of the configural model, indicating similar factor structures across adolescents and mothers/fathers. However, scalar invariance was not established, as a reduction in CFI > 0.010 and an increase in RMSEA > 0.015 indicated significant worse model fit. Modification indices recommended freeing the threshold of Item 7 (escapism symptom). After doing this, model fit was good and showed a minor reduction when compared to the configural model. Thus, partial scalar invariance was established; Item 7 was identified as the noninvariant item in terms of thresholds. More specifically, at the same level of the latent variable PSMU, parents tend to endorse a higher threshold for this specific item than adolescents. Since observed (mean) scores were used for subsequent analyses in SPSS, removing this item might seem the logical choice. However, we decided to keep Item 7, as with only one noninvariant item threshold, the SMD scale is not measuring fundamentally different constructs among reporters while removing Item 7 would mean a substantial change to the original measurement instrument and existing cutoff scores for classification purposes becoming unapplicable.

Mother–Father

According to all fit indices, except for SRMR, the configural model fits the data well, indicating similar factor structures across mothers and fathers. Constraining factor loadings and thresholds to be equal did not significantly worsen model fit, indicating evidence for scalar invariance.

Level of Agreement

The level of agreement between each reporter dyad was assessed on the classification level (absence/presence of PSMU), on the overall number of symptoms of adolescents’ PSMU (sum score), and on each separate symptom (Aim 2). We looked at both relative and absolute indices of agreement. First, relative agreement indices were obtained. For the classification and individual symptom level, we calculated the Cohen’s kappa (κ) which is a widely used statistic to measure agreement between reporters for categorical items that accounts for agreement expected by chance (Cohen, 1960). In addition, we calculated the prevalence index (PI), bias index (BI), and prevalence-adjusted bias-adjusted kappa (PABAK) to assist the interpretation of κ, as this has been recommended when having binary and skewed data (Byrt et al., 1993; Delgado & Tibau, 2019; Sim & Wright, 2005). PI is the difference in the probability of “yes” and “no” responses (calculated as the difference between the number of “yes” agreements and “no” agreements divided by N). BI measures the difference in probabilities of “yes” responses between two reporters. PABAK adjusts for these prevalence and bias effects and is calculated as 2P₀ − 1, where P₀ is the overall agreement percentage (Nurjannah & Siwi, 2017; Sim & Wright, 2005). To interpret κ and PABAK, the following criteria were used: ≤.20 indicates no agreement, 0.21–0.39 minimal agreement, 0.40–0.59 weak agreement, 0.60–0.79 moderate agreement, 0.80–0.90 strong agreement, and >0.90 almost perfect agreement (McHugh, 2012). For the overall number of symptoms, we calculated the intraclass correlation coefficient (ICC). ICC < 0.40 indicate poor agreement, 0.40–0.59 fair agreement, 0.60–0.74 good agreement, and ≥0.75 excellent agreement (Landis & Koch, 1977). Second, absolute agreement indices were obtained. Overall percent agreement (OPA), positive percent agreement (PPA), and negative percent agreement (NPA) were calculated using adolescent-reports as the reference measure when evaluating adolescent–mother and adolescent–father agreement. When evaluating mother–father agreement, mother-reports functioned as the reference. For example, PPA on individual symptom level between adolescents and mothers is the proportion of adolescents whose mothers reported “yes” on a certain item when they themselves reported “yes” on that item.

Classification Level

Agreement indices for the classification level (absence/presence PSMU) are presented in Table 4. Based on the κ values, adolescent–mother, and mother–father dyads showed minimal agreement, whereas adolescent–father dyads showed no agreement on the classification level. However, PI values were high, indicating an increased chance agreement resulting in lower κ values as κ is the agreement beyond chance (Sim & Wright, 2005). Contrary, bias effects were small as the BI values were low. When prevalence and bias effects were taken into account (PABAK), agreement improved to moderate between mothers and fathers, and strong between adolescents and mothers/fathers. OPA ranged from 87.2% to 91.1%. Agreement percentage on being a problematic user (PPA) between adolescents and mothers was 37.5%, between adolescents and fathers 20%, and between mothers and fathers 30.8%. There was over 93% agreement on being a nonproblematic user (NPA) between all dyads.

Overall Number of Symptom Level

With ICC values ranging from 0.28 to 0.36, we found poor agreement on the overall number of PSMU symptoms across all dyads. Percent agreement ranged from 35.4% to 39.2% (Table 4).

Individual Symptom Level

Table 5 lists the agreement indices for the individual symptom level. Across all dyads, the level of agreement on individual symptom level ranged from no (κ = .008) to weak (κ = .431) agreement. However, for many items, PI was rather high. PABAK values ranged from 0.28 (minimal agreement) to 0.92 (excellent agreement). OPA ranged from 63.9% to 95.8%, PPA from 15.6% to 66.7%, and NPA from 70.9% to 97.3%. Based on PABAK, OPA, and PPA, agreement between adolescent–mother and adolescent–father was highest for conflict, and agreement between adolescent–father was lowest for escape. Between adolescents and mothers and between mothers and fathers, PABAK, OPA, and NPA were lowest for displacement. Between mothers and fathers, OPA was highest for conflict, NPA and PABAK was highest for problems, and PPA was highest for preoccupation. Between adolescents and parents, PPA was lowest for symptoms that are difficult to observe (withdrawal, tolerance, deception, escapism, and problems).

Mean Differences

We conducted Wald tests using the “model test” option in Mplus to examine significant mean differences between adolescent- and mother-reports, adolescent- and father-reports, and between mother- and father-reports on the overall number of symptoms (Aim 3). Maximum likelihood estimation with robust standard errors (MLR) in combination with TYPE = COMPLEX were used to deal with nonnormality of PSMU and clustering of the data on family-level (Muthén & Muthén, 2017b). We found no significant mean differences (Table 4).

Factors Related to Type of Discrepancy

In order to examine possible associated factors with type of discrepancy between adolescents and parents, we first differentiated the following three groups: (a) mother/father reporting more adolescents’ PSMU symptoms compared to adolescent (mother/father-overreporting group), (b) mother/father reporting fewer adolescents’ PSMU symptoms compared to adolescent (mother/father-underreporting group), and (c) mother/father reporting the same number of adolescents’ PSMU symptoms as adolescent (agreement group). Next, we compared these discrepancy groups in terms of the following factors using Wald tests:–Adolescents’ age, gender, SMU frequency, self-reported PSMU, and parent-reported PSMU; – Parents’ age, SMU frequency, and parental worrying about social media/gaming addiction (Aim 4). We tested this for mothers and fathers separately. In the analyses, nonindependence of observations was taken into account and MLR was used (Muthén & Muthén 2017b). In total, we performed 24 tests for mothers and 24 tests for fathers. After a Bonferroni correction, results are considered significant at a p value of <.002 (α = 0.05/24).

Adolescent–Mother

The discrepancy groups significantly differed from each other in terms of adolescent- and mother-reported number of PSMU symptoms, adolescents’ gender, and maternal worrying about children’s social media/gaming addiction (Table 6). The mother-underreporting group (31.1%) scored significantly higher on adolescent-reported adolescents’ PSMU than the mother-overreporting (31.1%) and agreement (37.9%) group. Besides, the mother-underreporting group consisted of more girls than the mother-overreporting and agreement group.

Mean Differences Between Discrepancy Groups

The mother-overreporting group scored significantly higher on mother-reported adolescents’ PSMU than the mother-underreporting and agreement group. The mother-overreporting group also scored significantly higher on maternal worrying about their child getting addicted to social media/gaming than the agreement group.

The discrepancy groups did not significantly differ from each other in terms of adolescents’ age and SMU frequency, and mothers’ age and SMU frequency.

Adolescent–Father

The discrepancy groups significantly differed in terms of adolescent- and father-reported number of PSMU symptoms (Table 6). The father-underreporting (30.7%) scored significantly higher on adolescent-reported adolescents’ PSMU than the father-overreporting (33.9%) and agreement group (35.4%).

The father over- and underreporting group scored significantly higher on father-reported adolescents’ PSMU than the agreement group.

The discrepancy groups did not significantly differ from each other in terms of adolescents’ gender, age, and SMU frequency, and fathers’ age, SMU frequency, and worrying about children’s social media/gaming addiction.

Preregistration and Data Availability Statement

This study was preregistered after data collection, but before data analysis via the Open Science Framework (https://osf.io/ra2wt/). We deviated from this preregistration in two ways. First, we reported PI, BI, and PABAK in addition to κ to evaluate the level of agreement. The reason for this is that—as discussed in earlier in this section—several researchers have pointed out that only reporting κ is problematic when prevalence of the behavior being rated is low. Second, to assess the level of agreement on classification level, we decided to recode the sum score on PSMU into two groups (absence vs. presence of PSMU) instead of the preregistered three groups (normative, at-risk, and PSMU), since PABAK can only be calculated for dichotomous variables (Bernstam et al., 2005). The data that support the findings of this study are publicly available at https://osf.io/ra2wt/.

Discussion

This study is a first attempt to give detailed insight into agreement and discrepancies between self-, mother-, and father-reported adolescents’ PSMU measured by the SMD scale. We (a) evaluated measurement invariance of the SMD scale between self-, mother-, and father-reports; (b) examined the level of agreement between these three dyads on classification, the overall number of symptoms, and individual symptom level; (c) examined mean differences between self- versus mother- and father-reports on the overall number of symptoms; and (d) explored possible associated demographic and media-specific factors with type of discrepancy.

As configural invariance was established, the overall factor structure of the SMD scale proved to be the same across adolescents and parents. Furthermore, scalar invariance across mothers and fathers, and partial scalar invariance (one noninvariant item) across adolescents and parents, demonstrated that the different reporters interpret and respond to the items in a (largely) similar way. These findings imply that, regarding discrepancies between mother- and father-reported adolescents’ PSMU, we can be sure that these differences are not a function of the measurement instrument. Discrepancies between adolescents and parents on classification and the overall number of symptoms level may be a function of the measurement instrument to a small extent, as they may be attributable to one of the nine SMD scale items functioning differently for adolescents than for parents. Parents had a higher threshold for the item “… often used social media so you/your child didn’t have to think about unpleasant things?” meaning that for parents, it takes a higher score on the latent variable PSMU to endorse the escapism symptom than for adolescents.

The different relative and absolute agreement indices give an ambiguous picture of the level of agreement across adolescents, mothers, and fathers. ICC values showed poor agreement among all three dyads on the overall number of symptoms level. On this level, overall agreement percentages were low as well (<40%). κ values also indicated no to minimal agreement among the dyads on the classification level and for most of the individual items. However, in most cases, the PI value indicated the presence of a prevalence effect (i.e., increased chance agreement resulting in biased [too small] κ values). When adjusting for these effects, PABAK showed moderate (mother–father) to strong (adolescent–parent) agreement. In addition, overall agreement percentages were high, both on classification (>87%) as individual symptom level (>63%). Yet, this was mainly due to high percentages of negative agreement; positive agreement was low. For adolescent–parent dyads, this means that whereas parents agreed with their children when adolescents reported no (symptom of) PSMU, they tended to disagree when their children reported the presence of PSMU or a specific PSMU symptom. For example, 62.5% of the adolescents reporting the presence of PSMU was not identified as such according to mother-reports (PPA = 37.5%). This percentage is even higher for fathers. These findings suggest PSMU symptoms remaining unnoticed or unrecognized by parents, which has also shown to be the case for alcohol abuse (Fisher et al., 2006) and depression symptoms (Orchard et al., 2019). This would indicate that solely relying on parent-reports may be problematic as this would result in a high chance of missing adolescents that may be in need of clinical attention. Our obtained κ values (0.10–.22) for the classification level deviate from the κ values reported in a study on agreement between self- and parent-reports of adolescents’ internet gaming disorder by Wartberg et al. (2019; around 0.60 indicating moderate agreement). A possible explanation for this difference may be that Wartberg et al. oversampled adolescents with subjectively perceived problematic digital media use. As PABAK indicated, the obtained agreement values in our study could be influenced by the small number of adolescents displaying PSMU or specific PSMU symptoms. Therefore, it is warranted to replicate this study in samples with a higher prevalence of PSMU. Still, we can conclude that self- and parent-reports do not seem interchangeable.

One of the reasons why PSMU (symptoms) may remain unnoticed or unrecognized by parents could be that multiple PSMU symptoms are hard to observe for parents, because they occur internally and concern subjective feelings (e.g., “… often used social media so you didn’t have to think about unpleasant things?”) or happen out of sight of parents (e.g., in school; e.g., “… regularly had arguments with others because of your social media use?”). This is underlined by the relative higher positive agreement percentages we found—as expected—for symptoms that are better observable for parents (e.g., “… had serious conflict with your parents, brother(s) or sister(s) because of your social media use?”).

The difference in PPA on classification level between adolescent–mother and adolescent–father dyads (36.5% vs. 20%, respectively) could be interpreted as some evidence for our expectation that fathers are more likely to be unaware of their children’s PSMU than mothers, as fathers often spend less time with their children (Åman-Back & Björkqvist, 2004). However, the PPA for both mothers and fathers was low and we did not find (notable) differences in the level of agreement between adolescent–mother and adolescent–father dyads based on the other agreement indices. Thus, our findings do not confirm our hypothesis that there would be higher agreement between self- and mother-reports than between self- and father-reports.

As measurement invariance analyses showed that the SMD scale measured the same underlying construct among mothers as fathers, discrepancies between parents may reflect different but valid perspectives on the child’s behavior that might be caused by the child behaving differently in front of one parent versus the other (Davé et al., 2008). Another explanation for discrepancies between mothers and fathers could be that personal factors such as mental health status influence the accuracy of reporting (Ehrlich et al., 2011; Hughes & Gullone, 2010). Future studies are needed to further unravel mechanisms underlying discrepancies.

Contrary to our hypothesis, on group level, we found no significant mean differences between the number of adolescents’ PSMU symptoms reported by adolescents versus their mothers and fathers, reasonably because parental over- and underreporting canceled each other out as both occurred to the same extent.

Examining factors related to type of discrepancy gave more insight into when parental over- or underreporting compared to self-reports on the number of PSMU symptoms is more likely to occur. Our findings suggest that parental underreporting is more likely to occur, compared to overreporting and agreement, when adolescents display more PSMU symptoms according to self-reports. Besides, mothers were more likely to underreport on their daughters’ PSMU symptoms than on their sons’ PSMU symptoms. This is in line with the previous finding, as girls are likely to report more PSMU symptoms than boys (Boer, Stevens, Finkenauer et al., 2021). These findings point to the earlier mentioned trend that when adolescents develop PSMU symptoms, it remains unnoticed or unrecognized by parents. Yet, these findings could be a result of a statistical effect: Statistically, the chance of parents reporting less PSMU symptoms than adolescents themselves is higher when adolescents report more PSMU symptoms. However, when looking at father-reported adolescents’ PSMU, we do not see this statistical effect occurring. Results of the analyses regarding parent-reported adolescents’ PSMU again suggest that discrepancies in reports—for mothers overreporting and for fathers both over- and underreporting—are more likely to occur when adolescents score higher on PSMU.

Type of discrepancy was also associated with maternal worrying about children’s social media/gaming addiction. Maternal overreporting seems more likely to occur when mothers are more worried about their children getting addicted to social media or games. In contrast, we found no differences between discrepancy groups for fraternal worrying about their child’s social media/gaming addiction while fathers worry as much as mothers, suggesting that father-reports are not affected by their worries. This could be another explanation for the discrepancies between mothers and fathers.

Limitations

This study comes with some limitations that merit attention. First, the study sample has limitations. For example, the sample consisted to a great extent of highly educated and intact families, which may have influenced the level of agreement between reporters (Van Roy et al., 2010). Besides, the sample size was relatively small. Therefore, this study should be replicated in different and larger samples, to see if similar values of agreement and discrepancies will be obtained. Additionally, the SMD scale consisted of one item with a noninvariant threshold across adolescent and mothers/fathers. Future studies should test whether this item remains noninvariant in other samples. Besides, since there is a lack of a gold standard to measure adolescents’ PSMU, it is unknown whether, for example, parents reporting less symptoms than adolescents themselves truly means parental underreporting or adolescent overreporting. To obtain more knowledge about which reporter can be considered as providing more adequate information and whether using single- or multi-informant reports of PSMU has to be recommended, research comparing self- and parent-reports with clinical assessments by professionals is needed. However, this kind of research is not feasible (yet), because to date a clinically validated measure of PSMU is lacking. As an alternative, future studies could examine whether self- or parent-reports are a stronger predictor of, preferably objective, outcome variables that are theoretically related to PSMU (e.g., mental well-being, attention problems, school grades; Boer et al., 2020; Boer, Stevens, Finkenauer et al., 2021; Boer, Stevens, de Looze et al., 2021; Van den Eijnden et al., 2018) to see which type of reports shows better construct validity.

Conclusions

Our study is the first that sheds light on agreement and discrepancies between adolescent-, mother-, and father-reported adolescents’ PSMU on classification, the overall number of symptoms, and individual symptom level. The current findings show overall poor agreement between adolescent-, mother-, and father-reported adolescents’ PSMU. However, additional measures indicated that most of the found values are biased by the low prevalence of adolescents’ PSMU. Therefore and as this is the first study addressing this topic, replications of this study are needed. Besides, this study highlights the importance of further research on the assessment of adolescents’ PSMU, including the need of a clinically validated measure of PSMU, as at present much remains unknown about the utility of parent- versus self-reports of adolescents’ PSMU. Although it is far too early to draw firm conclusions about the utility of parent-reports on adolescents’ PSMU instead or in addition to self-reports, our findings do suggest that solely relying on parent-reports is not recommended as it seems that PSMU symptoms may remain unnoticed or unrecognized by parents. Researchers should be aware that relying on mother- and/or father-reports may result in different research findings than when relying on adolescent-reports, especially when coding PSMU as categorical outcome variable, since different adolescents may be identified as being involved in PSMU. We believe this study serves as a valuable starting point for future research.

Supplemental Materials

https://doi.org/10.1037/tmb0000110.supp

Copyright © 2023 The Author(s)

Received August 23, 2022
Revision received January 2, 2023
Accepted March 23, 2023