Abstract

Though smartphones have been shown to undermine well-being and social connection, evidence also suggests that these effects depend on when and how people use their phones. To examine whether the effects of phones on well-being (affect valence) and social connection depend on the situation, we compiled data across eight published and unpublished experiments where phone use was manipulated (N = 1,778). These experiments included situations ranging from parents visiting a science museum with their children, eating a meal with a group of strangers, to looking for an unfamiliar building. We found that phones have a significant negative impact on people’s feelings of social connectedness across situations. The impact of phones on well-being, however, depended on the situation: Phones negatively impacted well-being when used during ongoing social interactions, but not when used to find information relevant to current goals. Our large data set also allowed us to examine whether the effects of phones depend on individual differences. We found that gender moderated the effects of phones on social connectedness, whereby phones negatively impacted men more than women. Overall, even after including unpublished studies with nonsignificant findings, we find that the negative effects of phones on well-being and social connection persist. Going beyond past research, however, we also show that these negative effects on social connection are driven by men more so than women.

Keywords: smartphone, subjective well-being, digital media, technoference, social connection

Funding: There was no source of external funding to declare.

Disclosures: The authors have no conflicts of interest to declare.

Author Contributions: Matthew R. Leitao, Jason D. E. Proulx, and Kostadin Kushlev contributed to conception and design. Kostadin Kushlev and Jason D. E. Proulx contributed to acquisition of data. Matthew R. Leitao and Kostadin Kushlev contributed to analysis and interpretation of data. Matthew R. Leitao, Jason D. E. Proulx, and Kostadin Kushlev drafted and revised the article. Matthew R. Leitao, Jason D. E. Proulx, and Kostadin Kushlev approved the submitted version for publication.

Data Availability: The data, materials, R code, and extra analyses can be found on the Open Science Framework at https://doi.org/10.17605/OSF.IO/ X73M9. These data were previously presented as a poster at the Society of Personality and Social Psychology Conference in 2022.

Open Science Disclosures: The data are available at https://doi.org/10.17605/OSF.IO/X73M9. The experimental materials are available at https://doi.org/10.17605/ OSF.IO/X73M9.

Correspondence concerning this article should be addressed to Matthew R. Leitao, Department of Psychology, Georgetown University, 37th and O Streets, NW, Washington, DC 20057, United States. Email: [email protected]

Video Abstract

Correlational research has led to different conclusions about the net effects of digital media on well-being. Even while using the same data sets, some researchers have concluded that digital media is contributing to rising rates of poor mental health among adolescents (Twenge & Campbell, 2019), while others have suggested that digital media is as harmless as eating potatoes (Orben & Przybylski, 2019). This controversy underscores the limitations of correlational research where the findings depend on the analytic strategy, proper inclusion of control variables, and so forth (Rohrer, 2018). Of course, correlational research also cannot establish causality. It is possible, for example, that being less happy leads to greater digital media use (Orben et al., 2019). Finally, a great deal of the existing research tries to quantify the net effects on the well-being of various types of digital media and devices—from video to social media and smartphones—taken together.

Even though researchers disagree about the net effects of digital media on well-being, there is a consensus that the effects depend on how much, who, and when people are using their digital devices. Because smartphones are on us throughout the day, their effects on well-being should be particularly dependent on the context in which they are used. According to the interference hypothesis (Kushlev & Leitao, 2020; Sbarra et al., 2019), smartphones impact well-being by distracting us from concurrent activities. Thus, smartphones should decrease well-being when their use distracts us from pleasant activities (e.g., social interactions). According to the complementarity hypothesis (Kushlev & Leitao, 2020; Waytz & Gray, 2018), smartphones should also impact well-being to the extent that they serve as a source of information, entertainment, and social interactions that would otherwise be difficult to access. For example, a person can find getting directions much easier and faster on their smartphone than by using a map or asking strangers for directions. Though such efficiency may often promote well-being, the effects of smartphones should still depend on the context. By obviating the need to rely on others for information, for example, the complementary use of smartphones could undermine an opportunity for social connection.

Experimental research provides initial evidence to support the interference hypothesis (Kushlev et al., 2019). For example, people randomly assigned to share a meal with their phones on the table enjoyed their experience less than those whose phones were locked away (Dwyer et al., 2018). Similarly, parents assigned to use their phones a lot while spending time with their children experienced less meaning in life than parents assigned to minimize their phone use (Kushlev & Dunn, 2019). Experimental studies have also shown evidence for the complementarity hypothesis (e.g., Holtzman et al., 2017; Kushlev et al., 2017). After a stressful experience, participants in one study who received emotional support via text message reported greater positive affect than participants who did not (Holtzman et al., 2017).

Even though experimental studies have shown that smartphones do influence well-being, existing evidence is limited in several ways. First, experimental studies typically focus on a single situation, preventing us from directly examining whether the effects of smartphones on well-being depend on the situation. Second, experimental studies are typically not well powered to examine whether the main effects depend on individual differences, such as personality, age, or gender. Finally, due to publication bias, small experimental studies that observe large effects are more likely to get published than studies that observe smaller, nonsignificant effects. Thus, published studies may overestimate the size of the true effect (Nelson et al., 2018). In the present research, we aim to overcome these limitations by using a mini mega-analytic approach, whereby we analyze both published and unpublished experimental studies conducted by us and our collaborators.1

The Present Research

A mega-analysis is the analysis of the combined raw data from multiple studies that measure the same constructs (Eisenhauer, 2021; Olkin, 1995). Unlike a meta-analysis, a mega-analysis utilizes the original raw data instead of summary statistics without assuming the uniformity of within-study effects. Utilizing a mega-analysis allows us to find comparable outcomes to a meta-analysis while preserving the ability to examine additional moderators and mediators (Boedhoe et al., 2019; Sung et al., 2014; Tierney et al., 2015). Thus, given our access to the original raw data for all studies, a mini mega-analysis offered clear advantages over a traditional meta-analytic approach. We use the word “mini” to describe our mega-analysis in order to denote that the data were not collected from a systematic review of the literature but originated from a series of studies conducted by a single lab (Goh et al., 2016). This mini mega-analytic approach necessitated using previously analyzed data. As a result, our registered hypotheses and analyses regarding the effect of phones on social connection and affect valence could have been influenced by our prior knowledge of the data.

In the present mini mega-analysis, we compiled data from eight experiments that manipulated phone use (N = 1,778). We used data from all studies—both published and unpublished—that we have conducted where we experimentally investigated the effects of smartphones on well-being. Of those eight studies, four were previously published (n = 809) and four were unpublished (n = 969; see Footnote 1). All studies except one were preregistered (see Table 1). We sorted the eight studies into six different experimental paradigms. The number of paradigms is smaller because some studies were replication attempts and used the same procedure. In studies where a second variable was manipulated, we split the study into two different paradigms. Thus, we organized the studies into the following paradigms: (a) getting directions to an unfamiliar building (getting directions), (b) parents with their children during an outing (parents with children), (c) a meal with friends and family (strong ties meal), (d) a meal with strangers (weak ties meal), (e) waiting in a room with a stranger (waiting room together), and (f) waiting in a room alone (waiting room alone). Information about the experiments and paradigms is included in Table 1.

Table 1
Paradigm	Publication	Study description	Study population	N	Manipulation	% Female/M_age	Preregistration
Experimental Paradigms and Study Information
Getting directions	Kushlev et al. (2017)	College students were asked to find an unfamiliar building on their campus either with or without their phones.	College students	Study 1: 98 Study 2: 189	Phone/no phone	74.4%/20.1	https://doi.org/10.17605/OSF.IO/GZV74 (Study 1) https://doi.org/10.17605/OSF.IO/DRK3Z (Study 2)
Parents with children	Kushlev and Dunn (2019)	Parents/guardians visiting a science museum with their children were asked to use their phones either a lot or limit their phone use during their visit.	Parents/guardians visiting a museum	Study 3: 217	More phone/less phone	57.3%/37.6	https://doi.org/10.17605/OSF.IO/IT5RQ (Studies 3 and 4)
	Unpublished	Parents/guardians visiting a children’s festival with their children were asked to use their phones either a lot or limit their phone use during their visit.	Parents/guardians visiting a children’s festival	Study 4: 191	More phone/less phone	75.9%/37.6
Strong ties meal	Dwyer et al. (2018)	Groups of friends and families sat for a meal at a local café. Half were asked to keep their phone on the table with them; the other half were told to turn off their phone and to put it away.	College students and community members	Study 5: 305	Phone/no phone	66.4%/29.9	https://osf.io/Q4M6C (Study 5)
Weak ties meal	Unpublished	Groups of strangers were told to get to know each other over lunch. Half of the groups were asked to keep their phone on the table; the other half was told to turn off their phone and put it away.	College students	Study 6: 271	Phone/no phone	77.5%/20.3	https://doi.org/10.17605/OSF.IO/R8AVU (Study 6)
Waiting room alone	Unpublished	Participants were told to wait in a room for 10 min alone. Half of the participants kept their phones, and the other half sat without their phones.	College students	Study 7: 130	Phone/no phone	69.0%/20.7	https://doi.org/10.17605/OSF.IO/3SXT7 (Study 7)
Waiting room together	Unpublished	Participants were told to wait in a room for 10 min with a stranger: half of the pairs with their phones and the other half without their phones.	College students	Study 7: 143 Study 8: 234	Phone/no phone	70.7%/20.2	https://doi.org/10.17605/OSF.IO/ZVD5F (Study 8)
Note. The sample size of each study indicates the number of participants who were assigned to a condition, regardless of whether the manipulation was successful. Thus, Study 1 included six additional participants and Study 2 included seven additional participants who were excluded from the analyses reported in the original publication. In addition, our samples in Study 7 included 37 participants who were assigned to wait together but waited alone, and 11 participants who were assigned to wait with no phones, but waited with phones. In the parents with children paradigm, we included both parents and participants who were taking care of somebody else’s children (e.g., relatives, guardians). The waiting room paradigms were part of the same study (Study 7) where both phone and partner presence were manipulated; subsequently, an additional study was conducted where phone presence was again manipulated but all participants waited together (Study 8), resulting in more participants in the waiting room together paradigm.

In three of the paradigms (parents with children, strong ties meal, and weak ties meal), phone use was manipulated during ongoing social interactions. Similarly, the waiting room together paradigm and the getting directions paradigm included social opportunity costs of phone use, whereby phone use may interfere with initiating a social interaction with others. In contrast, the waiting room alone paradigm was designed to present no opportunity costs of phone use by having participants wait in a room by themselves with nothing else to do. According to the interference hypothesis (Kushlev & Leitao, 2020), phones should decrease well-being and social connection in social situations by interfering with the benefits of social interactions but should have little impact in nonsocial situations. Thus, we registered the following hypotheses about the effects of phones on well-being and social connection:

People randomly assigned to a “phone use” condition during social situations will report (H₁) lower emotional well-being and (H₂) lower social connection than people assigned to have no access to their phones.

In other words, we expected that phones would have a negative effect on social connection and well-being across all situations except in the waiting room alone paradigm. Though not registered, we also wanted to examine the possibility that the interference effects of phones on well-being and social connection in the getting directions paradigm may be offset by the positive effects of having access to timely information relevant to current goals (i.e., complementarity effects).

In addition to testing our hypothesis, we registered an analysis to examine whether gender moderates the effects of phones on well-being and social connection. Previous research has shown that social media use, for example, is associated with worse mental health in adolescent girls than boys (Kelly et al., 2019; Orben et al., 2019; Twenge et al., 2022), though meta-analytic evidence has failed to establish consistent gender differences (Meier & Reinecke, 2021). Similarly, correlational research finds that in romantic couples, higher partner technology use predicts worse relationship satisfaction (McDaniel et al., 2021) and partner phone use predicts worse marital quality (Khodabakhsh & Le Ong, 2021) in both men and women; some evidence suggests that this negative effect may be stronger for women than men (Khodabakhsh & Le Ong, 2021). Above and beyond the partner’s technology use, one’s own technology use in romantic relationships predicts less satisfaction with shared leisure time for both men and women (McDaniel et al., 2021). Few, if any, studies, however, have examined gender differences in how one’s own phone use impacts well-being and social connection beyond the context of romantic relationships. Thus, we registered no specific hypotheses about gender.

Finally, we registered age as a covariate in our analyses. Though included in our models, we cannot draw meaningful conclusions about the main and moderating effects of age because age was heavily conflated with the paradigm. Specifically, participants in the parenting and restaurant paradigms were significantly older than those in the rest of the paradigms (who were primarily undergraduate students).

Method

Registration

In an initial registration (https://doi.org/10.17605/OSF.IO/JE4SM), we specified our hypotheses before we ran any analyses. After conducting the registered analyses (see Supplemental Tables S1 and S2 for details), we determined a need to update the analytic plan for two reasons, which we also registered (https://doi.org/10.17605/OSF.IO/H48RA). First, we removed covariates that were not uniformly measured across paradigms to preserve power and provide uniformity to analyses with and without controls. Out of a total of 1,778 participants across paradigms, ethnicity was measured in 1,065 and phone dependence was measured in 713; thus, we removed these covariates from the registered analysis to increase power and uniformity across models. Second, we updated our statistical approach from ordinary least squares (OLS) regression to multilevel modeling, which allowed us to model between-study variance and thus more precisely estimate the overall effects of phone use on well-being and social connection. In the present research, we follow this updated analysis plan. We made no changes to our hypotheses in this updated registration. The data, results, and code can be found on the Open Science Framework at https://doi.org/10.17605/OSF.IO/X73M9 (Leitao et al., 2023).

Participants

The experiments were conducted in British Colombia with 1,778 participants across all studies (70.4% female; M_age = 25.8 years, SD_age = 9.33, range: 13–74 years). See Table 1 for gender and age composition by paradigm. Except for participants in the parents with children and strong ties meal paradigms, participants in our data were undergraduates from the University of British Columbia. We adopted a conservative, intent-to-treat approach, whereby we included all participants who were randomly assigned to the condition regardless of whether the manipulation was successful (see Table 1). We conducted a sensitivity analysis on G*Power using a “fixed model with a single regression coefficient” (Faul et al., 2009, p. 1153) and found we could detect effects as small as f² = 0.007 with 95% power in models with up to four predictors across the full sample (α = .05, two-tailed). Additional sensitivity analyses are provided in online Supplemental Materials.

Design

In all but the parents with children paradigm, participants were randomly assigned to either have their phone (phone condition) or not have their phone (no-phone condition) during the study (50.1% in phone condition).2 In the parents with children paradigm (Kushlev & Dunn, 2019), parents/guardians were asked to either use their phones a lot (phone condition) or limit their phone use (no-phone condition). In situations where multiple participants were seated together, everyone within a group was assigned to one condition (e.g., everyone at the table either had their phone or did not have their phone). Across paradigms, we coded the phone condition as 1 and the no-phone condition as 0.

Measures

Social Connectedness

We created a measure of social connectedness by combining two items that were consistently measured across all studies: “I feel/felt close to people” and “I feel/felt distant from people.” The items were adapted from the Social Connectedness Scale (Lee et al., 2001) and were measured on a 7-point scale from not at all to very much (correlation coefficient ranged from −.59 to −.41). To create a composite of social connectedness, we subtracted feelings of distance from feelings of closeness. Before combining the data, we standardized the scale at the level of the paradigm, as registered. Overall, 1.18% of participants had missing data on this question even though only participants who answered both questions were included in the analyses.

Well-Being: Affect Valence

In this article, we operationalized subjective well-being as affect valence, a dimension of affect that captures a continuum from pleasure (i.e., feeling pleasant, good) to displeasure (i.e., feeling negative, bad). This measure of well-being is similar to the construct of affect balance—a common measure of subjective well-being that captures the preponderance of positive affect over negative affect (Diener et al., 2010, 2018). Unlike affect balance, however, affect valence has been shown to be fairly independent of other dimensions of affect, such as arousal and fatigue (Schimmack & Grob, 2000).

All but the parents with children paradigm measured affect valence using a 7-point validated affect valence scale (Schimmack & Grob, 2000). Specifically, participants were asked to report how often during the study they felt: good, positive, pleasant, bad, negative, and unpleasant (α = .85–.90, see Supplemental Table S3 for more details). We subtracted the sum of bad, negative, and unpleasant from the sum of good, positive, and pleasant to create our measure of affect valence (Schimmack & Grob, 2000). The parents with children paradigm measured affect valence with a single 7-point affect valence measure asking participants how they felt during the experiment from very bad to very good. To accommodate for differences in the scales, we standardized the scores at the paradigm level before combining the data.3 Across studies, 2.02% had missing data on affect valence.

Age and Gender

Age, across studies, had 3.37% missing data. Gender was dummy-coded, with men coded as 1 and women coded as 0. Across studies, gender had 1.24% missing data and .002% reported a gender other than male or female. Statistical models with gender included only people who identified as either a man or woman due to insufficient power to conduct analysis with people who identified as nonbinary or genderqueer.

Results

Analysis Plan

Before our primary multilevel analyses, we used OLS regressions to examine whether the effect of the phone condition on social connectedness and affect valence differed by paradigm.4 As registered, we regressed well-being and social connection on phone condition, paradigm, and their interaction term, while controlling for age and gender. We used deviation contrast coding for each paradigm, thus comparing each paradigm with all the other paradigms (Sundström, 2010). We chose to use the weak ties meal paradigm as the reference category for these analyses as it contained many of the functional qualities of the other paradigms. Specifically, participants in the weak ties meal paradigm conversed with strangers, as in the getting directions and waiting room together paradigms, but for a substantive amount of time, as in the parents with children, waiting room together, and strong ties meal paradigms. This initial step was taken to understand whether the effects of phones differed by paradigm so that we can separately examine any positive and negative effects of phones within different situations. This approach was informed by the displacement–interference–complementarity framework (Kushlev & Leitao, 2020), which predicts that the effects of phones should depend on the situation. Based on these initial analyses, we then grouped paradigms that did not significantly moderate the effect of phones for our primary multilevel modeling analyses.

We utilized a multilevel model approach for our primary analyses. This allowed us to model the clustering within the paradigm when examining the effect of experimental conditions. Specifically, we ran a series of multilevel models with person clustered within the paradigm. As registered, we introduced variables in steps using a hierarchical approach, starting with the phone condition, then adding gender and age, and finally adding the interaction between the phone condition and gender. For each multilevel model, we used restricted maximum likelihood estimation and omitted cases listwise.

We used R (Version 1.4.1717; R Core Team, 2019) utilizing a combination of the “Stats” package (Version 3.6.1; R Core Team, 2019), “nlme” package (Version 3.152; Pinheiro et al., 2021), “emmeans” package (Version 1.6.2-1; Lenth, 2021), “sjPlot” package (Version 2.8.10; Lüdecke, 2021), and “EMAtools” package (Version 0.1.3; Kleiman, 2017).

Social Connectedness

Differences Between Paradigms

In the initial OLS model, we found no interactions between the phone condition and paradigm, suggesting that the effects of the condition on social connectedness did not differ by paradigm (see Table 2).5 Therefore, we ran all the paradigms together in the subsequent models predicting social connectedness. See Figure 1 for a forest plot of Cohen’s d effect sizes of phone condition on social connectedness within each paradigm.

Table 2
Fixed effect	β	b [95% CI]	t
Initial Model—Ordinary Least Squares Regression Predicting Social Connectedness
(Intercept)	.00	.01 [−.20, .22]	.09
Age	.04	.00 [−.00, .01]	.93
Condition (phone = 1, no-phone = 0)	−.07*	−.14 [−.25, −.02]	−2.33
Gender (man = 1, woman = 0)	.01	.02 [.12, .16]	.27
Getting directions	.07	.12 [−.03, .28]	1.54
Strong ties meal	−.05	−.09 [−.25, .07]	−1.15
Waiting alone	−.03	−.06 [−.26, .15]	−.54
Waiting together	−.02	−.03 [−.17, .11]	−.40
Parents with children	.02	.04 [−.13, .20]	.44
Condition × Gender	−.11**	−.32 [−.52, −.11]	−3.03
Condition × Getting Directions	−.08	−.20 [−.41, .01]	−1.88
Condition × Strong Ties Meal	.06	.16 [−.06, .38]	1.45
Condition × Waiting Alone	.04	.11 [−.19, .41]	.73
Condition × Waiting Together	.05	.11 [−.08, .30]	1.16
Condition × Parents With Children	−.08	−.17 [−.36, .02]	−1.79
Note. Number of paradigms = 6, N = 1,694. Deviation coding was used to determine differences in the paradigm. The weak ties meal paradigm was used as the reference category for all paradigms. CI = confidence interval. p < .05. * p < .01.

**Figure 1**
The Forest Plot Represents the Size and 95% Confidence Interval of the Cohen’s d Effect of the Phone Condition on Social Connectedness by Paradigm
*Note*. Cohen’s $d=\frac{m_{2}\, - \, m_{1}}{\sqrt{\left( \frac{SD_{1}^{2} + SD_{2}^{2}}{2} \right)}}.$

Multilevel Models

In Model 1 (Table 3), we included only the phone condition to determine the general effects of phones on social connectedness. We approximated Cohen’s d for this analysis using a transformation of the t statistic: $d = \frac{2t}{\left. \sqrt{}df \right.}$ . We found that people in the phone condition (M = −.13, SE = .04) felt less socially connected than people in the no-phone condition (M = .13, SE = .04), b = −.259, SE = .060, t(1749) = −4.33, p < .001, d = −.207. In Model 2, we included the demographic covariates of gender and age along with the phone condition (Table 3). The effect of the condition on social connectedness remained significant, b = −.241, SE = .065, t(1685) = −3.713, p < .001, d = −.181, with those who had their phones (M = −.15, SE = .05) still reporting lower social connectedness than those who did not have their phone (M = .09, SE = .04). Gender was also a significant predictor of social connectedness, b = −.134, SE = .052, t(1685) = −2.584, p = .010, d = −.126, with men (M = −.09, SE = .04) reporting lower social connectedness than women (M = .04, SE = .03). Finally, age did not significantly predict social connectedness, b = .002, SE = .003, t(1685) = .900, p = .368.

In Model 3, we added an interaction between condition and gender (Table 3). We found a significant Gender × Phone Condition interaction on social connectedness, b = −.313, SE = .103, t(1684) = −3.03, p = .003, d = −.148. In simple effects analyses, we found that the effect of phones on social connectedness was larger for men, b = −.461, SE = .098, t(1684) = −4.706, p < .001, d = −.473, than for women, b = −.148, SE = .072, t(1684) = −2.046, p = .041, d = −.152 (see Figure 2, Supplemental Table S7). This difference between men with their phones (M = −.33, SE = .07) and women with their phones (M = −.036, SE = .048) appears to be driven by the gender difference in the negative effects of phones and not gender differences more generally as we did not find significant differences between men without their phones (M = .13, SE = .06) and women without their phones (M = .11, SE = .05), b = −.018, SE = .072, t(1684) = −.255, p = .799, d = −.019 (Supplemental Table S8).

Table 3
Effect	Model 1 [95% CI]	Model 2 [95% CI]	Model 3 [95% CI]
Results of Multilevel Models Predicting Social Connectedness
Fixed effects
${\hat{\gamma}}_{00}$ —(intercept)	.13** [.06, .20]	.10 [−.05, .25]	0.07 [−.09, .22]
${\hat{\gamma}}_{10}$ —condition (phone = 1, no-phone = 0)	−.26*** [−.38, −.14]	−.24*** [−.37, −.11]	−.15* [−.29, −.01]
${\hat{\gamma}}_{20}$ —gender (man = 1, woman = 0)		−.13* [−.24, −.03]	.02 [−.12, 16]
${\hat{\gamma}}_{30}$ —age		.00 [−.00, .01]	.00 [−.00, .01]
${\hat{\gamma}}_{40}$ —Condition × Gender			−.31** [−.52, −.11]
Random effects
${\hat{\tau}}_{00}$ —paradigm	.00	.00	.00
${\hat{\tau}}_{11}$ —Paradigm × Condition	.01	.01	.01
${\hat{\rho}}_{01}$ —covariance	−.95	−.95	−.96
$\sigma^{2}$	.98	.94	.94
ICC_Paradigm	.00	.00	.00
R²_Marginal	.02	.02	.02
R²_Conditional	.02	.02	.03
Deviance (−2LL)	−2477.70	−2362.83	−2359.59
AIC	4967.41	4741.66	4737.18
Note. Model 1 (N = 1,756), Model 2 (N = 1,694), and Model 3 (N = 1,694). R²_Marginal represents the variance explained by the fixed effect. R²_Conditional represents the variance explained by the entire model. Connectedness_ij = ${\hat{\gamma}}_{00}$ + ${\hat{\gamma}}_{10}$ Condition_ij + ${\hat{\gamma}}_{20}$ Gender_ij + ${\hat{\gamma}}_{30}$ Age_ij + ${\hat{\gamma}}_{40}$ Condition_ij × Gender_ij + u₀_j + u₁_jCondition_ij + e_ij: CI = confidence interval; ICC = intraclass Correlation; LL = log likelihood; AIC = Akaike information criterion. p < .05. p < .01. **p < .001.

**Figure 2**
The Rain Plots Represent the Effect of the Phone Condition on Social Connectedness by Gender
*Note*. The dots represent individual data points, while the gold points represent the mean and the black bars bordering the gold points represent the 95% confidence intervals. The top and bottom of each box indicate the 75th and 25th percentile, respectively. The thick horizontal line in each box represents the median, and the whiskers represent 1.5 times the interquartile range from the adjacent quartile to the median. The wave in the plot represents the density of the data and the data’s relative distribution.

Affect Valence

Differences Between Paradigms

In the initial OLS model, we found a significant interaction of condition with the getting directions paradigm, b = .251, SE = .111, t(1665) = 2.265, p = .024, d = .204 (see Table 4).6 After removing the getting directions paradigm and rerunning the model, we found no additional significant interactions between condition and paradigm. As registered, we ran all subsequent models separately for the getting direction paradigm and for the rest of the paradigms. See Figure 3 for a forest plot of Cohen’s d effect sizes of the phone condition on affect valence by paradigm.

Table 4
Fixed effect	β	b [95% CI]	t
Initial Model—Ordinary Least Squares Regression Predicting Affect Valence
(Intercept)	−.00	−.09 [−.31, .12]	−.85
Age	.05	.01 [−.00, .01]	1.38
Condition (phone = 1, no-phone = 0)	−.01	−.01 [−.13, .11]	−.21
Gender (man = 1, woman = 0)	−.02	−.05 [−.20, .09]	−.71
Getting directions	−.06	−.10 [−.26, .06]	−1.22
Strong ties meal	−.00	−.00 [−.17, .16]	−.03
Waiting alone	.03	.07 [−.14, .28]	.65
Waiting together	.05	.09 [−.06, .23]	1.16
Parents with children	−.04	−.07 [−.24, .10]	−.81
Condition × Gender	−.07	−.20 [−.41, .01]	−1.88
Condition × Getting Directions	.10*	.25 [.03, .47]	2.27
Condition × Strong Ties Meal	−.03	−.07 [−.29, .15]	−.61
Condition × Waiting Alone	−.03	−.10 [−.40, .21]	−.64
Condition × Waiting Together	−.04	−.09 [−.29, .10]	−.93
Condition × Parents With Children	−.00	−.00 [−.19, .19]	−.02
Note. Number of paradigms = 6, total N = 1,680. Deviation coding was used to determine differences in the paradigm. The weak ties meal paradigm was used as the reference category for all paradigms. CI = confidence interval. *p < .05.

**Figure 3**
The Forest Plot Represents the Size and 95% Confidence Interval of the Cohen’s d Effect of the Phone Condition on Affect Valence by Paradigm
Note. Cohen’s d = $\frac{m_{2}\, - \, m_{1}}{\sqrt{\left( \frac{SD_{1}^{2} + SD_{2}^{2}}{2} \right)}}.$

Models Without Getting Directions Paradigm

In Model 1 (Table 5), we found that people in the phone condition reported significantly more negative affect than those in the no-phone condition, b = −.105, SE = .052, t(1436) = −2.009, p = .045, d = −.106. This effect remained significant after controlling for age and gender (Table 5, Model 2), b = −.118, SE = .053, t(1392) = −2.217, p = .027, d = −.119. Gender also emerged as a significant predictor of affect valence, b = −.185, SE = .058, t(1392) = −3.186, p = .002, d = −.171, with men reporting more negative affect than women. Age did not significantly predict affect valence, b = .002, SE = .003, t(1392) = .737, p = .461. In contrast to social connectedness, we did not find a significant Gender × Phone Condition interaction on affect valence, b = −.159, SE = .116, t(1391) = −1.377, p = .169, d = −.074 (see Table 5).

Table 5
Effect	Model 1 [95% CI]	Model 2 [95% CI]	Model 3 [95% CI]
Results of Multilevel Models Predicting Affect Valence Without the Getting Directions Paradigm
Fixed effects
${\hat{\gamma}}_{00}$ —(intercept)	.06 [−.02, .13]	.06 [−10, .23]	.04 [−.12, .21]
${\hat{\gamma}}_{10}$ —condition (phone = 1, no-phone = 0)	−.11* [−.21, .00]	−.12* [−.22, −.01]	−.07 [−.20, .06]
${\hat{\gamma}}_{20}$ —gender (man = 1, woman = 0)		−.19** [−.30, −.07]	−.11 [−.27, .05]
${\hat{\gamma}}_{30}$ —age		.00 [−.00, .01]	.00 [−.00, .01]
${\hat{\gamma}}_{40}$ —Condition × Gender			−.16 [−.39, .07]
Random effects
${\hat{\tau}}_{00}$ —paradigm	.00	.00	.00
${\hat{\tau}}_{11}$ —Paradigm × Condition	.00	.00	.00
${\hat{\rho}}_{01}$ —paradigm	.00	−.00	−.00
$\sigma^{2}$	.99	.99	.99
ICC_Paradigm	.00	.00	.00
R²_Marginal	.00	.01	.01
R²_Conditional	.00	.01	.01
Deviance (−2LL)	−2043.42	−1990.14	−1990.43
AIC	4098.84	3996.28	3998.86
Note. Model 1 (N = 1,442), Model 2 (N = 1,400), and Model 3 (N = 1,400). R²_Marginal represents the variance explained by the fixed effect. R²_Conditional represents the variance explained by the entire model. Affect_ij = ${\hat{\gamma}}_{00}$ + ${\hat{\gamma}}_{10}$ Condition_ij + ${\hat{\gamma}}_{20}$ Gender_ij + ${\hat{\gamma}}_{30}$ Age_ij + ${\hat{\gamma}}_{40}$ Condition_ij × Gender_ij + u₀_j + u₁_jCondition_ij + e_ij: CI = confidence interval; ICC = intraclass Correlation; LL = log likelihood; AIC = Akaike information criterion. p < .05. *p < .01.

Models for Getting Direction Paradigm

Because we did not have multiple paradigms to cluster, we ran an OLS regression with the getting directions paradigm. Across all models for the getting direction paradigm (Table 6), we do not find significant effects of the phone condition on affect valence, Model 1: b = .18, SE = .119, t(281) = 1.508, p = .133, d = .180; Model 2: b = .162, SE = .119, t(276) = 1.362, p = .174, d = .163. In Model 2, gender did not significantly predict affect valence, b = .009, SE = .137, t(276) = .067, p = .947, d = .008, but age was a significant predictor of affect valence with older participants reporting greater positive affect, b = .059, SE = .024, t(276) = 2.423, p = .016, d = .293. In Model 3, we did not find a significant Gender × Phone Condition interaction on affect valence, b = −.519, SE = .270, t(275) = −1.921, p = .056, d = −.231.

Table 6
Effect	Model 1			Model 2			Model 3
Results of Hierarchical Ordinary Least Squares Regression Predicting Affect Valence—Getting Directions Paradigm Only
	b	b [95% CI]	t	b	b [95% CI]	t	b	b [95% CI]	t
Fixed effect
(Intercept)	−.00	−.09 [−.26, .08]	−1.08	−.00*	−1.26 [−2.21, −.30]	−2.59	−.00**	−1.31 [−2.27, −.36]	−2.72
Condition (phone = 1, no-phone = 0)	.09	.18 [−.05, −.41]	1.51	.08	.16 [−.07, .40]	1.36	.15*	.30 [.03, .57]	2.15
Gender (man = 1, woman = 0)				.00	.01 [−.26, .28]	.07	.12	.28 [−.11, .67]	1.43
Age				.15*	.06 [.01, .11]	2.42	.14*	.06 [.01, .11]	2.41
Condition × Gender							−.18	−.52 [−1.05, .01]	−1.92
R²	.01			.03			.04
Note. Model 1 (N = 283), Model 2 (N = 280), and Model 3 (N = 280). CI = confidence interval. p < .05. *p < .01.

Discussion

The present study is the first to combine and analyze experimental data on the effect of phone use on well-being across a variety of social situations. Our results show that individuals with their phones across situations report being less socially connected than those without their phones. People randomly assigned to be with their phones also experienced more negatively valenced affect across most situations, except when phones were useful in obtaining information relevant to current tasks (i.e., getting directions). Thus, even after including unpublished studies with nonsignificant findings in our mega-analysis, we confirm findings from previous research that phones have significant negative effects on well-being during social situations.

Going beyond past research, we also show that the negative effects of phones on social connectedness are stronger in men than in women. Though phones significantly undermined social connectedness in both genders, the size of the effect for men (d = −.473) is about 3 times larger than it is for women (d = −.152). These differences between men and women appear to stem from differences in the impact of phones. Specifically, we found that men felt significantly less socially connected than women when participants had their phones, but we found no gender differences when participants did not have their phones (Supplemental Table S7). Given that we did not register any hypotheses about gender, however, these findings should be interpreted with caution. Future confirmatory research should examine whether our exploratory findings replicate.

Implications

Though phones significantly undermine well-being and social connection, we found that the effects of phones are small to very small (Cohen, 1988). Statistically small effects, however, are not necessarily practically insignificant (Funder & Ozer, 2019; Götz et al., 2022). When looking at the effects of phones on well-being in the aggregate, across multiple social encounters in a person’s everyday life, phones may leave individuals with a regular reduction in well-being. Indeed, a key strength of the present research is being able to estimate the effects of phone use across a range of common daily experiences—from spending time with one’s children and sharing a meal with friends to meeting new people and waiting alone. Furthermore, our effect estimates were based on intention-to-treat analyses, whereby we included all participants as randomly assigned to conditions regardless of whether participants passed the manipulation check.

Though most of the situations examined in this research were social, our analysis also included one nonsocial situation where participants had to wait alone in a room for 10 min. Contrary to our interference hypothesis that phones would impact well-being primarily during social experiences, we did not find that the effect of phones on social connection and affect valence differed significantly by whether people had the opportunity to interact with others or not. Future research should further test this hypothesis by including a greater range of nonsocial situations. Still, it is notable that despite the unlimited opportunities for connection and entertainment that phones provide, waiting alone with one’s phone did not produce more positive effects. That said, our pattern of findings is not all that surprising when we consider that most existing research has failed to show any benefits of media and phone use for well-being overall (Przybylski & Weinstein, 2017; Twenge & Campbell, 2019).

We found that the pattern of effects of phones on well-being significantly differed in only a single situation: when participants had to find an unfamiliar building. This was the only scenario in our analysis where phone use was directly complementary to people’s current goals. In this situation, we did not find a significant effect of phone use on affect valence.7 We propose that the lack of positive effect in this paradigm might be due to the confluence of both interference and complementarity processes. Though phones complemented participants’ current goals by providing useful and timely information on how to find direction, phones also interfered with potential opportunities to derive social connections by relying on others for help. Indeed, the published research using this paradigm found that phones impacted affect valence through two concurrent mechanisms: a positive effect on the well-being of finding the building more easily with phones and a negative effect on the well-being of feeling less socially connected with phones (Kushlev et al., 2017). As both of these mechanisms work in tandem, finding no main effect of phone use can be understood as balancing out competing effects. Unlike with social connectedness, the impact of phones on affect valence is more nuanced, so future research needs to consider when and for what purpose phones are being used when examining their effect on affect valence.

Gender Effects

Why did we find phones negatively impact men’s more than women’s social connectedness? One possibility is that men use their phones more than women, but past research shows that women use their phones more than men (Andone et al., 2016; Przybylski & Weinstein, 2017). Another possibility could be that the effect is due to differences in how distractable men and women are, but previous literature finds no gender differences in attention (Grissom & Reyes, 2019) or ability to multitask (Hirnstein et al., 2019; Hirsch et al., 2019; Lui et al., 2021). A third possibility is that women are more careful about using their phones, specifically during social situations. Indeed, research shows that women feel that phones are less appropriate to use when interacting with others than men (Forgays et al., 2014; Washington et al., 2014). These differences in perceived social norms could result in differences in how and how much men and women use their phones during social interactions. Thus, men may use their phones even when it is detrimental to their feelings of social connectedness, whereas women may restrict their phone use. A final possibility is that women may be using their phones differently than men. Consistent with this possibility, Andone et al. (2016) found that women generally use their phone’s communication and social apps more than men. As such, one reason why women may feel more socially connected than men when they have access to their phones is that women may use their phones in more social ways. Future research should examine this possibility directly by evaluating how and how much men and women use their phones during social interactions.

Limitations

We compiled data from studies using experimental designs that were meant to resemble a variety of real-world experiences. However, randomly assigning people to have access or no access to their phones may have introduced subtle confounds that limited ecological validity. Moreover, most of the scenarios involved positive social interactions. As such, future researchers should examine more closely whether phones have more positive effects on well-being during neutral and negative situations. Our findings suggest that examining a wider range of scenarios may be particularly important when studying the impacts of phones on affect. Our research included only one paradigm in which phones were not detrimental to affect. Future research needs to examine a greater array of situations in which the use of phones may be beneficial.

Another limitation of the present research is that the population consisted of mainly college-aged individuals from British Columbia. College students are a meaningful population to examine because of the prevalence of phone use. Still, whether these effects would persist with other populations and across different cultural beliefs or social norms remains to be determined. Moreover, given that gender acted as a moderator of the effect of phones on social connectedness, other demographics such as age should be investigated as possible moderators of this effect.

Finally, because we analyzed secondary data, not all situations measure the same potential underlying mechanisms. Though we were able to use a few covariates and moderators in our models, such as gender and age, we could not evaluate other factors, such as the role of race, ethnicity, or phone distraction. This limited our ability to explain the underlying mechanism as to why phones may have impacted participants. It is important for future work to evaluate whether there are similar or diverse mechanisms across social contexts. With the increasing trend toward open access data, we hope that researchers will be able to compile data that measure the same mechanisms to help explain why phones impact well-being.

Conclusion

In sum, through a mega-analysis of published and unpublished experimental data examining the social and emotional impacts of phone use across a range of common daily situations, the present research provides further evidence that phones have small but persistent negative effects on social connection. Going beyond past research, we further discovered that the negative effects of phones on social connection might be stronger in men than women, providing a fertile ground for future investigations.

Supplementary Materials

https://doi.org/10.1037/tmb0000125.supp

tmb0000125_supp001.docx

63 KB

open-practice-disclosure_Kushlev.pdf

302 KB