Volume 14, Issue 4 p. 1105-1128
ORIGINAL ARTICLE
Open Access

Cognitive-affective responses to online positive-psychological interventions: The effects of optimistic, grateful, and self-compassionate writing

Johannes Bodo Heekerens

Corresponding Author

Johannes Bodo Heekerens

Department of Education and Psychology, Division of Methods and Evaluation, Freie Universitat Berlin, Berlin, Germany

Correspondence

Johannes Bodo Heekerens, Department of Education and Psychology, Division of Methods and Evaluation, Freie Universitat Berlin, Berlin, Germany.

Email: [email protected]

Search for more papers by this author
Michael Eid

Michael Eid

Department of Education and Psychology, Division of Methods and Evaluation, Freie Universitat Berlin, Berlin, Germany

Search for more papers by this author
Kathrin Heinitz

Kathrin Heinitz

Department of Education and Psychology, Division of Methods and Evaluation, Freie Universitat Berlin, Berlin, Germany

Department of Education and Psychology, Division of Work and Organizational Psychology, Freie Universitat Berlin, Berlin, Germany

Search for more papers by this author
Belinda Merkle

Belinda Merkle

Department of Education and Psychology, Division of Methods and Evaluation, Freie Universitat Berlin, Berlin, Germany

School of Social Sciences, Chair of Educational Psychology, University of Mannheim, Mannheim, Germany

Search for more papers by this author
First published: 09 January 2022
Citations: 6

Funding information: Friedrich Naumann Foundation

Abstract

Growing evidence suggests that online positive-psychological interventions effectively increase well-being, and a wealth of evidence describes cognitive-affective responses to such interventions. Few studies, however, have directly compared responses across popular exercises such as the best-possible-self intervention, the gratitude letter, or self-compassionate writing. In addition, current evidence is ambiguous regarding the effects of potential moderator variables such as trait gratitude and emotional self-awareness. To address these issues, we randomized 432 German adults to perform either optimism, gratitude, self-compassion, or control writing interventions in an online setting. Participants reported trait gratitude and trait emotional self-awareness before the interventions, as well as momentary optimism, gratitude, self-compassion, positive affect, and current thoughts immediately after the interventions. Results indicate higher momentary optimism after the best-possible-self intervention and higher momentary gratitude after the gratitude letter than after the control task. There were no differences when comparing the best-possible-self intervention with the gratitude letter. Both interventions increased the number of positive self-relevant thoughts. The self-compassion condition showed no effects. Moderation analysis results indicate that neither emotional self-awareness nor trait gratitude moderated the intervention effects. Future studies should compare responses across different positive-psychological interventions using more comprehensive exercises to ensure larger effects.

INTRODUCTION

Growing evidence suggests that positive-psychological interventions, which are intentional activities designed to cultivate positive emotions, behaviors, and cognitions (Sin & Lyubomirsky, 2009), effectively increase well-being (see Bolier et al., 2013; Sin & Lyubomirsky, 2009, for two independent meta-analyses). Numerous studies have successfully delivered positive-psychological interventions online (see Stone & Parks, 2018, for a review). However, there is still limited understanding of the cognitive-affective responses to these interventions and how such responses are similar or different across interventions (Lyubomirsky & Layous, 2013).

In this study, we focus on three positive-psychological interventions that are popular among researchers and practitioners: first, the best-possible-self (BPS) intervention (King, 2001), which has been repeatedly shown to increase positive affect and optimism and to decrease pessimism (see Heekerens & Eid, 2020; Loveday et al., 2016; Malouff & Schutte, 2016, for reviews and meta-analyses); second, the gratitude letter exercise (Seligman et al., 2005), which has been repeatedly shown to increase gratitude and psychological well-being (see Davis et al., 2016, for a meta-analysis); and third, self-compassionate writing (Shapira & Mongrain, 2010), a frequently used component of comprehensive interventions that focus on the cultivation of compassion, which might increase compassion, self-compassion, and mindfulness, as well as alleviate depressive and anxious symptoms (see Kirby, 2017; Kirby et al., 2017, for a review and meta-analysis). The magnitude of the effects of brief stand-alone positive-psychological interventions is typically small to medium and decreases over time (Bolier et al., 2013) and may even be smaller when using online formats (e.g., Heekerens & Eid, 2020). In practice, this problem may be resolved by embedding positive-psychological interventions in comprehensive well-being programs (e.g., Heintzelman et al., 2020). In this study, we chose to examine the effects of stand-alone online interventions because this allows to investigate the cognitive-affective responses to these interventions.

Cognitive-affective intervention effects

Positive-psychological interventions have been proposed to target positive emotions and cognitions, which in turn increase well-being (Lyubomirsky & Layous, 2013). In line with this, meta-analytic evidence shows that different positive interventions successfully induce positive affect (e.g., Davis et al., 2016; Heekerens & Eid, 2020). One experimental study has shown that increases in positive emotions during a 8-weeks loving-kindness meditation program predicted later increases in personal resources and life satisfaction (e.g., Fredrickson et al., 2008). Similarly, one study showed that the effects of two one week positive-psychological interventions on happiness and depressive symptoms were partially mediated through changes in positive emotions during the trial (Gander et al., 2020). In addition, specific positive-psychological interventions are assumed to promote adaptive cognitions (Seligman et al., 2005). For example, the BPS intervention is theorized to build positive future expectations, the gratitude letter should allow to adopt a grateful outlook, and self-compassionate writing has been proposed to induce a mindful awareness that allows to overcome negative thoughts and feelings involved in personal suffering (see Gross, 1998; Quoidbach et al., 2015, for detailed conceptual frameworks). In line with this, meta-analytic evidence shows that individuals report higher levels of optimism after writing about their best-possible future (Heekerens & Eid, 2020; Malouff & Schutte, 2016), higher levels of gratitude after writing a gratitude letter (Davis et al., 2016), and higher levels of self-compassion after a self-compassionate writing task (Kirby et al., 2017). In addition, results from one waitlist-controlled study suggest that a 12-week comprehensive positive intervention program successfully promoted specific cognitions and emotions (e.g., hope, self-compassion, and gratitude) that were targeted during the weekly online or in-person sessions of the program and that this change partially accounted for increased subjective well-being after the program (Heintzelman et al., 2020). Moreover, one experiment that manipulated the instructions for a positive-psychological intervention such that the focus was either on cognitions (“describe”) or emotions (“re-experience”) or both suggests that focusing on cognitions had a stronger effect on happiness (Gander et al., 2017; also see Wellenzohn et al., 2016). However, there is also evidence that some positive interventions have effects that are theoretically more closely linked to other interventions. For example, results from one experimental study suggest that not only variations of the best-possible self-intervention but also different gratitude interventions increase optimism and decrease hopelessness (Huffman et al., 2014). In addition, experimental results indicate that a brief self-compassion intervention not only increases self-compassion and mindfulness but also self-efficacy and optimism (Smeets et al., 2014). Thus, the specificity of effects remains somewhat controversial.

Differential intervention effects

Another line of research has addressed the question for which groups of people positive-psychological interventions show optimal effects (see Fritz & Lyubomirsky, 2018, for a recent review). Specifically, current conceptual frameworks suggest that the effectiveness of positive-psychological interventions depends on the interplay between features of the activity and participant characteristics (Lyubomirsky & Layous, 2013; Schueller, 2011). In other words, effects are expected to vary due to interactions between persons and situations (see Cronbach & Snow, 1977, for an early discussion). In this study, we focus on emotional self-awareness and trait gratitude as moderators of cognitive-affective responses to positive-psychological interventions because preliminary evidence indicates that these variables may play a role in differentiating intervention effects between participants. We did not investigate trait optimism as a moderator of the best-possible-self intervention because previous research found that intervention effects are equal among participants with varying levels of trait optimism (Harrist et al., 2007; Meevissen et al., 2011; Peters et al., 2010).

Emotional self-awareness describes how frequently individuals generally pay attention to their own emotions (see Eid et al., 2003; Lischetzke et al., 2012; Swinkels & Giuliano, 1995, for a deeper discussion). The BPS intervention should be particularly helpful for individuals low in emotional self-awareness because such individuals prefer not to approach strong emotions and writing about goals provides self-regulatory benefits without an exploration of unpleasant emotions (King, 2001). On the contrary, the self-compassion intervention should yield better results for individuals high in emotional self-awareness because the intervention offers the opportunity to explore and alleviate negative emotions, which reconciles with the needs of highly emotionally self-aware individuals (Austenfeld & Stanton, 2004). Congruently, two studies found that students who reported more active attempts to acknowledge their emotions reported larger reductions in depressive symptoms and hostility after writing about negative emotions, compared with writing about their best possible future (Austenfeld et al., 2006; Austenfeld & Stanton, 2008). The moderation effect, however, has not yet been investigated using outcomes that are more characteristic of the BPS and self-compassion interventions and more closely relate to well-being such as optimism, self-compassion, and positive affect.

Trait gratitude describes how frequently, intensely, and deeply individuals generally experience grateful affect (McCullough et al., 2002) and has been shown to relate to reports of state gratitude in daily life (McCullough et al., 2004). The gratitude intervention should be more effective for individuals higher in trait gratitude because of characteristic interpretive biases in appraising situations that explain the relation between trait and state gratitude (see Wood et al., 2008, for a conceptual model). Specifically, individuals higher in trait gratitude have been shown to make more positive help-related benefit appraisals (Wood et al., 2008), which should apply to the situation of writing a gratitude letter and result in higher levels of state gratitude among individuals higher in trait gratitude after the intervention. In line with this, initial evidence suggests that individuals high in trait gratitude expected gratitude interventions to be easier, more socially accepted, and more effective (Kaczmarek et al., 2015). In addition, one study reported that individuals higher in trait gratitude reported larger increases in positive affect after writing about someone to whom they felt grateful (Watkins et al., 2003, study 4). Other researchers, however, found that participants lower in trait gratitude reported higher positive affect (Harbaugh & Vasey, 2014), as well as higher happiness and life satisfaction (Rash et al., 2011) following gratitude-based interventions. One study suggests that individuals higher in openness and extraversion reported higher happiness and lower depressive symptoms after writing and delivering a gratitude letter than individuals lower in openness and extraversion (Senf & Liau, 2012). A later study investigating personality traits as moderators of the effects of a 10-week multicomponent online well-being program, however, did not find comparable effects (Wang et al., 2017). Taken together, studies on stable between-person differences as moderators of the effects of positive-psychological interventions have yielded few replicable effects and current evidence regarding trait gratitude as a moderator appears particularly mixed.

Aims of the present study

The aims of the present study were to investigate cognitive-affective responses to online positive-psychological interventions and to explore differential effect patterns. Specifically, we hypothesize the following:
  1. Participants in the optimism condition report higher optimism, in the gratitude condition higher gratitude, and in the self-compassion condition higher self-compassion after the intervention compared with the control condition (effects on cognitions).
  2. Participants in all positive-psychological intervention conditions report higher positive affect after the intervention compared with participants in the control condition (effect on affect).
  3. Participants low in emotional self-awareness report stronger effects in the optimism condition (compared to the control group), participants high in trait gratitude report stronger effects in the gratitude condition (compared to the control group), and participants high in emotional self-awareness report stronger effects in the self-compassion condition (compared to the control group).

METHOD

Participants

Participants were recruited online through the German platform respondi, offering them 5€ for their participation. We included German natives who were at least 18 years old and who passed all our quality checks, including assessments of whether participants read the instructions and questions carefully (Merkle & Kaczmirek, 2016). We excluded 10 participants, 3 in the control condition, 1 in the optimism condition, and 5 in the self-compassion condition, due to insufficient text quality as indicated by meaningless or defiant input. The final sample comprised 425 adults of whom 106 were assigned to the control condition, 110 to the optimism condition, 105 to the gratitude condition, and 104 to the self-compassion condition. A sample size of 425 is sufficient to detect an effect size of f = .16 of a one-way ANOVA consisting of four groups with a power of .80 (α = 0.05). Text quality was assessed by two independent raters and deviating ratings were discussed until a consensus was reached. The mean age of participants was 43.26 years (SD = 12.67, range = 18–75), and 57.2% were female. The sample comprised 7.8% students and individuals undergoing vocational training, 73.4% employees and freelancers, 3.5% jobseekers, and 15.3% others, including retirees and housewives. Data were collected in March 2018.

Procedure

Participants accessed our study through a link. On the first page, they were informed that the purpose of the study was to examine effects of writing on emotions as well as to the voluntary nature of participation and data protection. On the second page, participants answered questions regarding emotional self-awareness and trait gratitude. Afterwards, participants were randomly assigned to either perform one of the positive interventions or the control condition. We designed the survey such that participants had to spend at least 15 min on the writing task. After implementation, participants listed 10 current thoughts and rated the self-reference and valence of each thought. Next, participants reported momentary optimism, gratitude, self-compassion, and affective state. They indicated how much they liked the intervention, how much they have benefitted from the intervention, and how difficult the intervention was for them. Finally, participants answered socio-demographic questions, indicated their level of experience with self-help techniques, and whether they have been undergoing or currently undergo psychotherapeutic treatment.

Interventions

Participants were randomly assigned to one of four online interventions. The randomization sequence was computer generated, and nobody had access to the sequence at any time of the experiment. As in previous studies, participants in the optimism condition were instructed to write about their ideal future (based on King, 2001), participants in the gratitude condition wrote a letter about experiences for which they feel grateful towards the person who did the kind act for them (based on Lyubomirsky et al., 2011; Seligman et al., 2005), and participant in the self-compassion condition reflected upon their shortcomings from the perspective of a compassionate other (based on Shapira & Mongrain, 2010). Participants in the control condition were instructed to write about their previous week (based on Layous et al., 2013; Odou & Vella-Brodrick, 2013). We chose the control condition because the format is comparable to the positive interventions; however, the content should have been emotionally neutral on average. All participants were informed that their input is anonymous and were asked not to worry about grammar and spelling. All instructions were provided in German. Full texts are provided in Appendix S1. The ethics committee of the department of education and psychology at Freie Universität Berlin approved the study (No 177/2018).

Measures

Optimism

We assessed momentary optimism using an adaption of the German version of the Life Orientation Test Revised (LOT-R; Carver et al., 2010; Glaesmer et al., 2011). We only included the three items capturing optimism. Participants were asked about their momentary experience (e.g., “At the moment, I'm optimistic about my future”; see Appendix S2 for all selected items). The scale is anchored at 1 (strongly disagree) and 5 (strongly agree). McDonald's Omega (calculated using the R package MBESS; Kelley, 2007; McDonald, 1991) for optimism was .88, 95% CI [.85, .91].

Gratitude

Momentary gratitude was assessed using items borrowed from the Gratitude Questionnaire-6 (GQ-6; McCullough et al., 2002). Specifically, we used three items that make sense when assessing gratitude in the present moment (e.g., “At the moment, I have something in life to be thankful for”; see Appendix S2 for all selected items). Trait gratitude was assessed with the full scale and using the original wording (e.g., “I am grateful to a wide variety of people”). The German item versions were derived by a translation and back translation process (Proyer, 2007). The scale is anchored at 1 (strongly disagree) and 7 (strongly agree). McDonald's Omega for momentary gratitude was .91, 95% CI [.89, .93] and for trait gratitude .79, 95% CI [.76, .83].

Self-compassion

We assessed momentary self-compassion using an adaption of the German short version of the Self Compassion Scale (SCS; Hupfeld & Ruffieux, 2011; Raes et al., 2011). We included ten items that we reworded to reflect current self-compassion, and participants were asked about their momentary experience (e.g., “At the moment, I give myself the caring and tenderness I need”; also see Breines & Chen, 2012). Prior to the analysis, we excluded the item “I can imagine that feelings of inadequacy are shared by most people” from all analyses because the item was negatively correlated with all other items in the scale, demonstrating that the German translation of the item was ambiguous (see Wieland et al., 2017, for a discussion). The scale is anchored at 1 (strongly disagree) and 4 (strongly agree). McDonald's Omega for self-compassion was .81, 95% CI [.77, .84].

Affect

Positive affect was assessed using the short version A of the Multidimensional Mood Questionnaire (MDBF; Hinz et al., 2012; Steyer et al., 1994). The scale includes each four items referring to positive-negative mood (e.g., “happy”), alert-tired mood (e.g., “rested”), and calm-agitated mood (e.g., “restless”). Participants were asked how they feel “at the moment”. The scale is anchored at 1 (not at all) and 5 (very). McDonald's Omega for positive affect was .93, 95% CI [.92, .94].

Positive self-relevant thoughts

We assessed positive self-relevant thinking using the thought listing approach (Cacioppo et al., 1979). Participants were asked to list 10 current thoughts, and afterwards, they were to indicate whether each thought was self-relevant or not and whether each thought was positive, neutral, or negative. The average number of thoughts falling into each category (e.g., positive thought) was assessed by adding them up and dividing them by the total number of reported thoughts. Previous studies have established that the data obtained using the thought listing approach meet common psychometric standards (see Cacioppo et al., 1997; Glass & Arnkoff, 1994, for reviews). For example, the number of negative thoughts has been shown to relate to lower self-evaluations, providing evidence for criterion-related validity (Cacioppo et al., 1979). Another study found that the responses of participants who rated how comfortable they would feel in a hypothetical situation were similar whether or not participants completed the measure, indicating that the technique is not reactive (Fichten et al., 1988). However, clinical intervention studies demonstrated that the number of positive and negative thoughts can be changed through targeted action (Heimberg, 1994).

Emotional self-awareness

We assessed emotional self-awareness using the Attention to Feelings Scale (Lischetzke et al., 2001). The scale includes six items referring to individual differences in attention to one's feelings (e.g., “I think about my feelings.”). It is anchored at 1 (almost never) and 4 (almost always). McDonald's Omega for emotional self-awareness was .94, 95% CI [.93, .95].

Demographics and quality check items

Participants were asked to indicate gender, age, and current job status. Additionally, we asked how often they use self-help techniques such as books or mobile application on the topic of happiness offering the answer options “never,” “sometimes (once or twice a year),” “regularly (once or twice a month),” and “often (once or twice a week).” Afterwards, participants were asked whether or not they currently receive or have been receiving psychotherapeutic treatment. In addition, participants were asked how much they liked the exercise, how much they benefited from the exercise, and how difficult the exercise was for them (based on Schueller, 2011). Items were assessed separately to allow for more nuanced interpretations of the results. The scale is anchored at 1 (strongly disagree) and 7 (strongly agree). Furthermore, we used two quality check items to make sure that participants have read the instructions carefully (as recommended by Merkle & Kaczmirek, 2016). First, we included an instructional manipulation check and asked participants to respond “blue” to the question “Which color matches your text?” as part of the intervention description. The question and the instructed response “blue” were displayed after the intervention, together with the answer options “red,” “green,” “yellow,” and “purple.” Second, we displayed a 5-point rating scale anchored at 1 (not at all) and 5 (very) together with the item “sad” and asked participants “Please choose the answer option ‘very’ to show that you have read the instructions.”

Statistical analysis

We tested our hypotheses with three multiple group structural equation models (SEM). For the main analysis, we used MPLUS version 8.1 (Muthén & Muthén, 1998-2017) applying robust maximum likelihood (MLR) estimation. Specifically, we used multiple group analyses to test our hypotheses because this approach allowed us to test the expected group differences in latent means (Hypotheses 1 and 2) and differential effects (Hypothesis 3) within one statistical framework. Data were analyzed in two steps. In the first step, we defined three models with two correlated factors in the optimism condition (positive affect and optimism), the gratitude condition (positive affect and gratitude), and the self-compassion condition (positive affect and self-compassion) to assess model fits either assuming different or equal factor loadings and intercepts across the intervention and control groups. This was done to test the assumption of measurement invariance across groups. Comparing latent means and regression coefficient across groups require that strong (scalar) measurement invariance is given (e.g., Millsap, 2011, that means that the factor loadings and intercepts do not differ between groups). The models look like the models depicted in Figure 1 if the regression coefficients are replaced with covariance (correlations) between the factors. Each model contained three observed indicator variables loading on a common latent variable for each construct under investigation. The observed indicator variables for positive affect and self-compassion reflect parcels1 that were formed by aggregating randomly allocated items (as recommended by Matsunaga, 2008), whereas the indicator variables for optimism and gratitude reflect single items. Results indicate appropriate fit indices for the restricted (with the assumption of measurement invariance) optimism intervention model, urn:x-wiley:17580846:media:aphw12326:aphw12326-math-0001(60, N = 216) = 68.24, p = .218, CFI = .99, RMSEA = 0.04, 95% CI [0.00, 0.07], SRMR = .06, the gratitude intervention model, urn:x-wiley:17580846:media:aphw12326:aphw12326-math-0002(60, N = 211) = 89.49, p = .008, CFI = .97, RMSEA = 0.07, 95% CI [0.04, 0.10], SRMR = .08, and the self-compassion intervention model, urn:x-wiley:17580846:media:aphw12326:aphw12326-math-0003(60, N = 210) = 67.15, p = .246, CFI = .99, RMSEA = 0.03, 95% CI [0.00, 0.07], SRMR = .06 (as indicated by CFI > .97, RMSEA < .05, and SRMR < .08; Hooper et al., 2008; Hu & Bentler, 1999). Importantly, scaled urn:x-wiley:17580846:media:aphw12326:aphw12326-math-0004 difference tests (Satorra & Bentler, 2010) showed that assuming measurement invariance did not significantly worsen model fit for the three models, urn:x-wiley:17580846:media:aphw12326:aphw12326-math-0005= 19.30, urn:x-wiley:17580846:media:aphw12326:aphw12326-math-0006= 12, p = .082, urn:x-wiley:17580846:media:aphw12326:aphw12326-math-0007= 15.67, urn:x-wiley:17580846:media:aphw12326:aphw12326-math-0008 = 12, p = .207, and, urn:x-wiley:17580846:media:aphw12326:aphw12326-math-0009 = 19.44, urn:x-wiley:17580846:media:aphw12326:aphw12326-math-0010= 12, p = .079, respectively (see Cheung & Rensvold, 2002 for a discussion). Because strong measurement invariance is given, hypotheses 1 and 2 can be analyzed with respect to the latent means. Moreover, to analyze Hypothesis 3, we regressed our outcome variables on the proposed moderators and freely estimated the regression coefficients (see Figure 1). To test the proposed multivariate moderation effects, we compared the resulting models with models assuming equal regression coefficients across conditions. If the model with equal regression coefficient does not fit the data worse than the model with freely estimated regression coefficient, then there will be no significant moderation effect. If, however, the model with freely regression coefficient fits the data better, there will be a significant moderation, and the intervention effect will depend on the value of the moderator variable. Variance-covariance matrices for our models are shown in Appendix S3.

Details are in the caption following the image
(a) Emotional self-awareness: optimism condition (n = 110). (b) Trait gratitude: gratitude condition (n = 105). (c) Emotional self-awareness: self-compassion condition (n = 104). Note: Multiple group structural equation models comparing participants in the optimism, gratitude, and self-compassion condition with participants in the control condition. We displayed the models that freely estimate the regression coefficients to provide additional information although none of the differences between intervention and control groups reach statistical significance. Unstandardized parameter estimates with standard errors in brackets and standardized solutions for the latent regression part in bold. PA_A, PA_B, PA_C = observed variables (parcels) for positive affect; Opt_A, Opt_B, Opt_C = observed variables (items) for optimism; Grat_A, Grat_B, Grat_C = observed variables (items) for gratitude; SC_A, SC_B, SC_C = observed variables (parcels) for self-compassion; ESA_A, ESA_B, ESA_C = observed variables (parcels) for emotional self-awareness; TG_A, TG_B, TG_C = observed variables (parcels) for trait gratitude; PA = common latent state variable for positive affect; Grat = common latent state variable for gratitude; Opt = common latent state variable for optimism; SC = common latent state variable for self-compassion; ESA = common latent state variable for emotional self-awareness; TG = common latent state variable for trait gratitude. *p < .05. **p < .01. ***p < .001

Preliminary, exploratory, and additional analyses were done with R version 3.6.2 (R Core Team, 2019). For computerized text analysis, we used the German version of the Linguistic Inquiry and Word Count (LIWC) program version 2015 (see Hirsh & Peterson, 2009; Pennebaker, 2011; Tausczik & Pennebaker, 2010; Wolf et al., 2008, for a deeper discussion).

RESULTS

Preliminary analysis

Prior to the main analysis, we conducted a MANOVA to test whether participants in the positive intervention conditions and the control condition differ regarding emotional self-awareness, trait gratitude, and age. Results indicate no difference between conditions, Pillai's Trace = 0.02, F(3,421) = 0.76, p = .656, urn:x-wiley:17580846:media:aphw12326:aphw12326-math-0011 = 0.01 (using Serlin's correction as recommended by Grissom & Kim, 2012; Serlin et al., 1982). In addition, Pearson's Chi-squared test results suggest no difference regarding gender, urn:x-wiley:17580846:media:aphw12326:aphw12326-math-0012(3, N = 425) = 2.63, p = .452, ω = 0.08, 95% CI [0.00, 0.13], experience with self-help, urn:x-wiley:17580846:media:aphw12326:aphw12326-math-0013(9, N = 425) = 8.02, p = .532, ω = 0.14, 95% CI [0.00, 0.15], and therapy status, urn:x-wiley:17580846:media:aphw12326:aphw12326-math-0014(6, N = 425) = 3.87, p = .694, ω = 0.10, 95% CI [0.00, 0.12] (using the R package MBESS to calculate confidence intervals; Kelley, 2007). Taken together, results indicate no group differences before the interventions.

Tests of hypotheses

Effects on cognition

According to our first hypothesis, we expected that participants report higher optimism after the optimism intervention, higher gratitude after the gratitude intervention, and higher self-compassion after the self-compassion intervention. Before testing this hypothesis on the level of latent means, we will present the results with respect to the observed means because this is usually done in this area of research. Table 1 presents the results with respect to the total scale scores. It reveals that average observed optimism was significantly higher in the optimism compared with the control condition, 3.92 versus 3.55 on a scale ranging from 1 to 5, d = 0.38, 95% CI [0.11, 0.65] (effect size and confidence interval calculated using the R package MBESS; Cohen, 1988; Kelley, 2007), and average gratitude was significantly higher in the gratitude condition, 6.04 versus 5.58 on a scale ranging from 1 to 7, d = 0.40, 95% CI [0.12, 0.67]. There was no significant difference in average self-compassion in the self-compassion condition, 3.39 versus 3.43, d = -0.05, 95% CI [−0.32, 0.22].

TABLE 1. Manifest means, standard deviations, and intercorrelations for PA, Opt, Grat, SC, ESA, TG, and PST in the control (n = 106), optimism (n = 110), Gratitude (n = 105), and self-compassion (n= 104) conditions
Measure PA Opt Grat SC ESA TG PST M SD
Control and optimism condition
1. PA - .50*** .32*** .57*** −.06 .30** .31** 3.48 0.88
2. Opt .64*** - .42*** .58*** −.07 .31*** .31** 3.92 0.93
3. Grat .17 .48*** - .39*** .00 .68*** .08 5.91 1.16
4. SC .67*** .61*** .23* - .01 .32*** .38** 3.60 0.70
5. ESA −.19 −.06 .10 −.11 - .18 .06 2.86 0.72
6. TG .27** .48*** .57*** .28** .31** - .12 5.26 1.04
7. PST .34** .38** .04 .31** .01 .16* - 3.96 3.04
M 3.29 3.55 5.58 3.43 2.76 5.08 2.32
SD 0.85 0.97 1.23 0.70 0.70 0.93 2.16
Gratitude and self-compassion condition
1. PA - .60*** .28** .62*** -.18 .21* .34** 3.30 0.84
2. Opt .63*** - .51*** .67*** −.13 .52*** .27** 3.64 1.07
3. Grat .39*** .59*** - .39*** .24* .52*** .18 5.68 1.22
4. SC .65*** .47*** .32*** - −.18 .36*** .27** 3.39 0.72
5. ESA −.08 .05 .08 −.06 - .17 −.08 2.87 0.72
6. TG .46*** .60*** .69*** .38*** .21* - .17 5.34 0.91
7. PST .16 .24* .25* .13 .08 .14 - 2.20 2.03
M 3.43 3.71 6.04 3.42 2.76 5.14 3.73
SD 0.87 0.93 1.07 0.70 0.77 0.98 2.91
  • Note: The first section shows Pearson correlations for the control condition below the diagonal and for the optimism condition above the diagonal. Manifest means and standard deviations for the control condition are presented in the rows and for the optimism condition in the columns. The second section shows Pearson correlations for the gratitude condition below the diagonal and for the self-compassion condition above the diagonal. Means and standard deviations for the gratitude condition are presented in the rows and for the self-compassion condition in the columns.
  • Abbreviations: ESA, emotional self-awareness; Grat, gratitude; Opt, optimism; PA, positive affect; PST, positive, self-relevant thoughts; SC, self-compassion; TG, trait gratitude.
  • * p < .05.
  • ** p < .01.
  • *** p < .001.

Table 2 presents the (unstandardized) latent mean differences between the optimism, gratitude, and self-compassion conditions and the control condition. Based on the latent mean scores and there 95 % confidence intervals, the statistical conclusions were the same as for the observed means. The latent mean differences with respect to the control condition were 0.36, 95% CI [0.11, 0.61] for the optimism, 0.51, 95% CI [0.14, 0.87] for the gratitude, and −0.05, 95% CI [−0.24, 0.15] for the self-compassion condition. The estimates reported here are derived from the three multi group models (A–C) computed in the second step of our analysis after establishing measurement invariance.

TABLE 2. Latent mean differences, standard deviations, and latent covariances and correlations for PA, Opt, Grat, and SC in the control (n = 106), optimism (n = 110), gratitude (n = 105), and self-compassion (n = 104) conditions
Measure (A) optimism condition
1. PA 2. Opt 3. ESA M, SD
1. PA 0.36*** (0.08) 0.55*** −0.03 (0.05) −0.06

0.190, 88

[−0.03, 0.40]

2. Opt 0.47*** (0.09) 0.69*** −0.04 (0.07) −0.07

0.36, 0.93

[.011, .0.61]

3. ESA −0.09 (0.06) −0.19 −0.04 (0.07) −0.06 - 0.10, 0.09 [−0.08, 0.29]
(B) Gratitude condition
1. PA 2. Grat 3. TG M, SD
1. PA - 0.43*** (0.12) 0.47*** 0.50*** (0.14) 0.58***

0.14, 0.11

[−0.07, 0.36]

2. Grat 0.17 (0.12) 0.18 - 1.00*** (0.24) 0.82***

0.51, 0.19

[0.14, 0.87]

3. TG 0.24* (0.10) 0.33** 0.87*** (0.20) 0.72*** -

0.10, 0.16

[−0.22, 0.42]

(C) compassion condition
1. PA 2. SC 3. ESA M, SD
1. PA 0.34*** (0.08) 0.68*** −0.09 (0.06) −0.18

0.01, 0.11

[−0.20, 0.21]

2. SC 0.36*** (0.07) 0.73*** - −0.09 (0.06) −0.19

−0.05 0.10

[−0.24, 0.15]

3. ESA −0.09 (0.06) −0.19 −0.05 (0.05) −0.11 -

0.12, 0.10

[−0.07, 0.30]

  • Note: Within each section, we displayed the latent covariances and correlations for the control condition below the diagonal and for the intervention condition above the diagonal. Covariance estimates with standard errors in brackets and standardized solutions in bold. The latent means in the control condition were set to zero. Latent mean differences and standard deviations in the intervention condition are presented in the columns along with the corresponding confidence intervals. All models assume measurement invariance across conditions
  • Abbreviations: ESA, emotional self-awareness; Grat, gratitude; Opt, optimism; PA, positive affect; SC, self-compassion; TG, trait gratitude.
  • * p < .05.
  • ** p < .01.
  • *** p < .001.

Effect on affect

According to our second hypothesis, we expected that participants report higher positive affect after all positive interventions. Table 1 reveals that average observed positive affect measured on the total scale ranging from 1 to 5 was 3.48 in the optimism, 3.43 in the gratitude, and 3.30 in the self-compassion condition compared with 3.29 in the control condition. Standardized differences were d = 0.22, 95% CI [-0.05, 0.48], d = 0.16, 95% CI [−0.12, 0.43], and d = 0.01, 95% CI [−0.26, 0.28], respectively.

Accordingly, Table 2 shows that the (unstandardized) latent mean differences between the optimism, gratitude, and self-compassion conditions, and the control condition was 0.19, 95% CI [−0.03, 0.40], 0.14, 95% CI [−0.07, 0.36], and 0.01, 95% CI [−0.20, 0.21], respectively. None of the mean differences was significantly different from 0.

Moderation analyses

According to our third hypothesis, we expected that the effects of the optimism and self-compassion interventions depend on trait emotional self-awareness and that the effects of the gratitude condition depend on trait gratitude. This hypothesis assumes interactions between the intervention conditions and the personality variables. Because interaction is a symmetrical concept, these proposed interactions would imply that the regression slopes between the dependent latent variables and the potential latent moderator variables differ between the intervention and control groups. If there are no differences between the intervention and control groups in the regression coefficients, there will be no moderation (interaction) effect. To test our hypothesis, we compared two multiple group models against each other. The first model is depicted in Figure 1. It freely estimates the regressions between the proposed moderators and the outcome variables in the intervention and the control conditions. According to our hypothesis and assuming a linear moderation effect, the size of the negative regressions in the optimism condition should be larger than in the control condition because we expected individuals lower in emotional self-awareness to benefit more (or individuals higher in emotional self-awareness to benefit less or even experience adverse effects). Results for Model (a) in Figure 1 reveal that the size of the (unstandardized) regression for positive affect was actually smaller in the optimism compared with the control condition, B = −0.06 versus B = −0.21. For optimism, the coefficients were B = −0.07 versus B = −0.08. To test whether these differences were statistically significant, we calculated a second model under the assumption of equal regression coefficients across groups (i.e., assuming that the relation between emotional self-awareness and the outcomes is the same in the optimism and the control condition). A scaled urn:x-wiley:17580846:media:aphw12326:aphw12326-math-0015 difference test showed that the fit of the second model was not significantly worse than the fit of the first model, urn:x-wiley:17580846:media:aphw12326:aphw12326-math-0016 = 1.00, urn:x-wiley:17580846:media:aphw12326:aphw12326-math-0017 = 2, p = .606, indicating that there is not any significant moderation effect. We repeated the steps to test the expected moderation effects in the gratitude and self-compassion conditions. According to our hypothesis, the size of the positive regressions in the gratitude condition should be larger than in the control condition because we expected individuals high in trait gratitude to benefit more. Results for Model (b) in Figure 1 and Table 2 reveal that the coefficients for positive affect were B = 0.43 versus B = 0.26 and for gratitude B = 0.89 versus 0.95. The differences were not statistically significant, urn:x-wiley:17580846:media:aphw12326:aphw12326-math-0018 = 2.59, urn:x-wiley:17580846:media:aphw12326:aphw12326-math-0019 = 2, p = .274, showing that there is no significant moderation effect. Finally, we assumed that the size of the negative regressions in the self-compassion condition should be smaller than in the control condition because we expected individuals high in emotional self-awareness to benefit more. Results for Model (c) in Figure 1 and Table 2 reveal that the coefficients for positive affect were B = −0.20 versus B = −0.21 and for gratitude B = −0.19 versus −0.10. The differences were not statistically significant, urn:x-wiley:17580846:media:aphw12326:aphw12326-math-0020 = 3.07, urn:x-wiley:17580846:media:aphw12326:aphw12326-math-0021= 2, p = .215, indicating that there is no significant interaction.

Exploratory and additional analyses

We also investigated intervention effects on positive self-relevant thinking (Mongrain & Anselmo-Matthews, 2012), explored variables that potentially influence the effectiveness of our interventions (person-activity fit; Proyer et al., 2015; Schueller, 2011), and used text analysis to further investigate emotional and cognitive processing during the writing process (Guastella & Dadds, 2006).

Positive self-relevant thoughts

Univariate analysis of variance results indicates significant differences in positive self-relevant thoughts across groups, F(3,421) = 13.51, p = .000, urn:x-wiley:17580846:media:aphw12326:aphw12326-math-0022 = 0.09, 90% CI [0.05,0.13].2 Specifically, Table 1 reveals that participants in the control condition self-rated on average 23% of their thoughts as both positive and self-relevant compared with 39% in the optimism condition, 37% in the gratitude condition, and 22% in the self-compassion condition. Post hoc comparisons reveal significant differences between the optimism and the control conditions, diff = 1.63, 95% CI [0.73, 2.54, the gratitude and the control conditions, diff = 1.41, 95% CI [0.50, 2.33], the optimism and the self-compassion conditions, diff = 1.75, 95% CI [0.84, 2.66], and the gratitude and the self-compassion conditions, diff = 1.53, 95% CI [0.61, 2.45]. There were no significant differences between the self-compassion and the control conditions, diff = −0.12, 95% CI [−1.04, 0.80], as well as the optimism and gratitude conditions, diff = 0.22, 95% CI [−0.69, 1.13].

Person-activity fit

Results first show that participants liked the positive interventions better than the daily activities control. Specifically, the average score for liking on a scale ranging from 1 to 7 was 4.91 in the control condition compared with 5.36 in the optimism condition, d = 0.31, 95% CI [0.04, 0.58], 5.47 in the gratitude condition, d = 0.38, 95% CI [0.11, 0.65], and 5.40 in the self-compassion condition, d = 0.36, 95% CI [0.09, 0.63].3 Second, participants in the optimism and gratitude conditions reported that they had benefited more than participants in the control condition, 4.70 versus 4.25, d = 0.28, 95% CI [0.02, 0.55] and 5.42 versus 4.25, d = 0.77, 95% CI [0.49, 1.04], respectively. There was no significant difference between the self-compassion and the control condition, 4.58 versus 4.25, d = 0.22, 95% CI [−0.05, 0.49]. Third, participants perceived the self-compassion intervention as more difficult than the control intervention, 2.83 versus 2.28, d = 0.34, 95% CI [0.06, 0.61], whereas there were no significant differences between the optimism and the control condition, 2.53 versus 2.28, d = 0.15, 95% CI [−0.12, 0.42], and the gratitude and the control condition, 2.65 versus 2.28, d = 0.23, 95% CI [−0.04, 0.50].

Text analyses

Participants on average wrote 341 words in the control condition, which exceeded the average word counts in the optimism condition, 245, d = 0.51, 95% CI [0.24, 0.78], the gratitude condition, 247, d = 0.56, 95% CI [0.29, 0.84], and the self-compassion condition, 196, d = 0.96, 95% CI [0.68, 1.25]. The data contained four extreme values (optimism condition: 2, gratitude condition: 1, control condition: 1) identified as values that fall above 3 standard deviations above the third quartile or participants who wrote more than 931 words (mean for all: 257.69, median for all: 224). Statements about the significance of group differences do not differ between the results obtained from the complete data and results obtained after removing outliers, which were 332 versus 226 words, d = 0.73, 95% CI [0.45, 1.00], in the optimism condition, 332 versus 240 words, d = 0.63, 95% CI [0.35, 0.90], in the gratitude condition, and 332 versus 196 words, d = 0.98, 95% CI [0.70, 1.27], in the self-compassion condition (also see Aguinis et al., 2005; Dickerhoof, 2007). Text analysis results using the LIWC program show that participants in the optimism and gratitude conditions used more positive emotion words than participants in the control condition. As the table in Appendix S4 reveals, the average amount of positive emotion words was 27% in the control condition compared with 55% in the optimism condition, d = 1.28, 95% CI [0.99, 1.57] and 63% in the gratitude condition, d = 1.78, 95% CI [1.46, 2.10]. Participants in the self-compassion condition did not use significantly more positive emotion words than control participants, 31% versus 27%, d = 0.22, 95% CI [−0.06, 0.49]. However, the use of positive emotion words was positively associated with positive affect after the self-compassion writing session, r = .26, p = .007, whereas there was virtually no association in the control condition, r = .01, p = .905. In addition, participants in the gratitude and self-compassion conditions used more negative emotion words compared with the control condition, 16% versus 10%, d = 0.63, 95% CI [0.36, 0.91] and 36% versus 10%, d = 1.80, 95% CI [1.48, 2.13], respectively. Participants in the optimism condition did not use more negative emotion words, 10% versus 10%, d = 0.08, 95% CI [−0.18, 0.35]. Interestingly, the use of negative emotions words was negatively associated with positive affect in the control condition, r = −.28, p = .004, but not in the optimism condition, r = −.06, p = .513. One reason for this may be that the word count program we used did not count negated emotional expressions (e.g., “I will not be afraid”). Moreover, participants in the optimism, gratitude, and self-compassion conditions used more insight words compared with control participants, 30% vs. 18%, d = 0.99, 95% CI [0.71, 1.28], 24% versus 18%, d = 0.50, 95% CI [0.23, 0.78], and 30% versus 18%, d = 0.87, 95% CI [0.58, 1.15], respectively. Participants in the gratitude and self-compassion conditions also used more causal words, 16% versus 11%, d = 0.56, 95% CI [0.29, 0.84] and 20% versus 11%, d = 0.90, 95% CI [0.61, 1.18], respectively. There was no significant difference between the optimism and the control condition, 12% versus 11%, d = 0.10, 95% CI [−0.17, 0.37]. However, the use of causal words among BPS participants was positively associated with gratitude ratings after the intervention, r = .19, p = .049, whereas the association was negative in the control condition, r = −.27, p = .005. Finally, in line with the observation that participants in the optimism and gratitude conditions reported more positive self-relevant thoughts, text analysis results show that participants in the optimism and gratitude conditions used more positive emotion words to describe their thoughts compared with control participants, 11% versus 8%, d = 0.62, 95% CI [0.34, 0.89] and 16% versus 8%, d = 0.55, 95% CI [0.28, 0.83], respectively. There was no significant difference in the self-compassion condition, 7% versus 8%, d = -0.06, 95% CI [−0.33, 0.21].

DISCUSSION

The aim of the present study was to investigate cognitive-affective responses to online positive-psychological interventions and to explore differential effect patterns. Such knowledge is fundamental to the effective use and further development of positive-psychological interventions, which seems desirable considering their widespread application in digital formats (see Diefenbach, 2018; Hone et al., 2014, for reviews).

Cognitive-affective responses

In line with earlier studies, we found that the BPS intervention increased optimism (Heekerens & Eid, 2020) and that the gratitude letter increased gratitude immediately after the writing tasks (Davis et al., 2016). This finding provides further evidence that online positive-psychological interventions can facilitate specific adaptive cognitions (Quoidbach et al., 2015; also see Lyubomirsky & Layous, 2013). As in one previous study, effects on optimism and gratitude did not differ when comparing the BPS intervention group with the gratitude exercise group (Huffman et al., 2014). Other than expected differences in positive affect between the intervention conditions and the control task did not reach statistical significance. This is surprising given that numerous studies have demonstrated increases in positive affect following the BPS intervention (Heekerens & Eid, 2020), the gratitude letter exercise (Davis et al., 2016), and self-compassionate writing (Kirby et al., 2017). More so, and at odds with earlier studies (e.g., Shapira & Mongrain, 2010), the self-compassionate writing showed no other beneficial effect on cognitions. One reason for small or absent effects in our study is that we used very brief interventions (15 min writing) that were administered once. In addition, the self-compassion exercise may have lacked explicit instructions to acknowledge experienced difficult events or emotions, which may be necessary to develop self-compassionate responses. The instructions we used made participants perceive the self-compassionate exercise as less helpful and more complicated than the gratitude letter exercise and the best-possible-self intervention. It could have been helpful if participants had the opportunity to ask questions regarding the instructions (Dreisoerner et al., 2020). Particularly self-compassion interventions may require multiple administrations to be effective (e.g., 15 min daily meditation practice over the course of 8 weeks; Fredrickson et al., 2008; or 2 h weekly group sessions over the course of 8 weeks; Neff & Germer, 2013). In addition, our results indicate an increase in positive self-relevant thoughts immediately after the BPS intervention and the gratitude letter exercise (Mongrain & Anselmo-Matthews, 2012), which aligns well with the increase observed in levels of optimism and gratitude following these interventions.

Moderation effects

At odds with earlier studies, we found that the cognitive-affective responses to the gratitude letter did not differ depending on the level of baseline trait gratitude. This finding neither supports one experiment finding that individuals higher in trait gratitude report more positive emotions following gratitude interventions (Watkins et al., 2003, study 4), nor one experiment finding that individuals lower in trait gratitude report more positive emotions (Harbaugh & Vasey, 2014). In addition, we found that differences in baseline trait of emotional self-awareness do not affect cognitive-affective responses to the BPS intervention, which contradicts results from two experiments showing that individuals low in emotional processing particularly benefitted from the BPS intervention (Austenfeld et al., 2006; Austenfeld & Stanton, 2008). Reasons for the divergent results include differences in the frequency of intervention exposure (single vs. repeated administrations) or the outcomes under investigation (optimism and positive affect vs. negative affect). For example, individuals who generally pay little attention to their own emotions may require longer interventions to derive an additional self-regulatory advantage and succeed in regulating negative affective states.

Text analyses

We examined participants' writing and asked them to list 10 current thoughts after the interventions with the aim of exploring cognitive-affective processing. As expected, participants in the optimism and gratitude conditions used more positive emotion and insight words than participants in the control condition (Heekerens & Heinitz, 2019; Owens & Patterson, 2013). It might be that these interventions allow participants to draw connections between their present life and future dreams or reflect their relationships with meaningful others, which may facilitate self-exploration and understanding (King, 2002). Evidence from the expressive writing paradigm suggests that participants who used more positive emotion and insight words while writing about traumatic experiences gained most from the writing sessions (see Pennebaker, 2011, for a review). Building on this, results from the current study suggest that positive-psychological interventions may accomplish the same, providing a vital alternative to reactivating negative experiences to increase psychological health (King, 2001). Interestingly, participants in the gratitude condition also used more negative emotion words in their writings compared with control participants. This finding highlights that some positive-psychological interventions do require participants to confront unpleasant emotions while experiencing positive emotions at the same time (Killam & Kim, 2014). In addition, participants in the gratitude condition used more negative emotion words in their writings compared with control participants and although participants in the BPS intervention did not show an increased use of negative emotion words, the use of such words was unrelated to positive affect, whereas in the control condition a negative relationship was observed. This finding suggests that some positive-psychological interventions do require participants to confront unpleasant emotions and that positive-psychological interventions might help to facilitate an adaptive integration of negative emotional states, probably through simultaneously experiencing positive emotions (Killam & Kim, 2014).

Limitations and future research

Several limitations and proposals for future research should be mentioned. First, there was only one occasion of measurement after the intervention and our design did not permit conclusions regarding follow-up effects. Future studies should apply longitudinal designs and test how positive-psychological interventions differentially affect various outcomes over time (Maxwell & Cole, 2007). Second, our study revealed that the interaction effects are very small, and in most analyses much smaller, than typically found in moderator regression analyses (Champoux & Petres, 1987; Chaplin, 1991). Therefore, the sample size of our study was too small to detect such small interaction effects with sufficient power. Future studies can use the sizes of these estimated interaction effects to calculate the appropriate size in a priori power analyses (see Aguinis et al., 2005, for a deeper discussion). Third, the self-reports we used limit our results to conscious aspects of the constructs under investigation. Although there is compelling evidence that the subjective indicators we used are meaningful (e.g., Oswald & Wu, 2010), future studies should also evaluate positive-psychological interventions based on more objective metrics (e.g., real-time measures; Alexandrova, 2005; Kahneman, 2000). Fourth, participants' motivation to complete the interventions might primarily stem from the payment they received. Research shows that motivation influences the effectiveness of positive-psychological interventions and effect sizes are likely larger in samples of individuals who actively seek to become happier (Lyubomirsky et al., 2011; Parks et al., 2012). Fifth, although our results support the notion that positive-psychological interventions can affect specific outcomes (e.g., optimism or gratitude), longitudinal mediation studies are needed to establish whether increases in these variables explain subsequent increases in well-being (see Heekerens & Heinitz, 2019; Heekerens et al., 2019, for examples). Such studies should also investigate the role of positive self-relevant thinking (Mongrain & Anselmo-Matthews, 2012) and provide text analyses to deepen our understanding of cognitive and emotional processing during the writing sessions (Pennebaker, 2011).

ACKNOWLEDGMENTS

The first author of this study received a scholarship from the Friedrich Naumann Foundation for Freedom and is deeply grateful for the financial and personal support.

    CONFLICT OF INTEREST

    The authors declare that they have no conflict of interest.

    ETHICS STATEMENT

    The authors declare that the current study meets ethical standards and was approved by the Ethics Committee of the Freie Universität Berlin. All participants signed informed consent prior to participation.

    AUTHOR CONTRIBUTIONS

    JH and BM designed the experiments. JH conceived and performed the experiments. JH, ME, and BM analyzed the data. ME and KH contributed to reagents/materials/analysis tools. JH wrote the paper.

    ENDNOTES

    • 1 Parceling in structural equation modeling refers to a technique that aggregates individual items into “parcels” and uses those parcels, instead of items, as the indicators of latent constructs (Matsunaga, 2008). In this study, we randomly aggregated the 12 items of the MDMQ (affect) and the 10 items of the SCS (self-compassion) into each three parcels.
    • 2 For reasons of simplicity, we report ANOVA results here. We additionally performed sensitivity analyses for thought ratings using negative binomial regressions. Statements about the significance of group differences do not differ between the results of the more advanced methods and the normal approximations.
    • 3 Again, we reported standardized mean difference and confidence intervals here because we assumed normal approximations due to our comparably large sample. Sensitivity analyses were conducted for all tests. Specifically, group differences for rating scales were tested using probit regressions and for text analyses (percentages) using zero inflated beta regressions. Statements about the significance of group differences do not differ between the results of the more advanced methods and the normal approximations.

    DATA AVAILABILITY STATEMENT

    The data that support the findings of this study are available on request from the corresponding author, JH. The datasets are not publicly available because they contain information that could compromise the privacy of research participants.