It's our final blog post of 2023! I'm sharing a twofer here; a description of a study you can work with, which also comes with an invitation to participate and try the interventions yourself. The project is called Big Joy, and is administered by the Greater Good Science Center (GGSC).
The GGSC describes some results from the Big Joy project as
an interactive, web-based program that invites anyone interested to try a series of micro-acts for enhancing joy, one each day for one week. Big JOY also enables people to log their daily activities and experiences and to measure their impact on well-being.
The Big Joy project has involved about 84,000 participants online so far, from 207 countries. You can participate, too, by clicking here. This blog post will ask you to think about the data they present in this article to test the Big Joy project's effectiveness.
The Big Joy project invites people to participate in a small intervention a "micro act" every day for a week. Here are a few examples of these micro-acts, some of which you've read studies about on this blog before:
- Tune in to what matters: Rank important personal values such as kindness, humility, and accountability and write about how they appear in your life.
- Make a gratitude list: Think, reflect, and list anything you feel grateful about in your life.
- Dwell in awe: Watch an awe-inspiring video, like the world’s natural wonders.
- (and others)
In this article, GGSC present some pre- and post-data from 22,000 participants in the project. Jump over to the site and scroll a bit until you see a graph labeled "Weekly pre-post: Emotional well-being". About the graph, the authors of the post write that
Emotional well-being jumped 26%. Each dot in the graph above represents one person’s score for overall emotional well-being and is a composite of their self-rated life satisfaction, happy feelings, and meaning in life. From before (left column) to after (right column) Big JOY, most of the dots move up, which indicates a rising score—which, analyzed collectively, came to an increase of 26% from before to after Big JOY.
Let's dive into this finding.
a) What is the dependent variable in this result? What do its levels appear to be?
b) Is there an independent variable here? If so, what are its levels?
c) What kind of study does this appear to be: Posttest only? Pretest/posttest? Concurrent measures? Repeated measures? or One-group, prettest/posttest?
d) For now, do you think that the graph they present can support the claim that the Big Joy project caused emotional well-being to improve?
e) It's a spoiler, but the answer to question c) is that this design is a One-group, pretest/posttest, otherwise known (in Chapter 11) as the Really Bad Experiment. Here is a list of some of the internal validity threats in Chapter 11 that the study might be vulnerable to. Describe why (or why not), for each one:
Testing
Placebo
Instrumentation
Maturation
History
Attrition
Now, although GGSC presented a really bad experiment about their Big Joy project, this doesn't mean the intervention itself is based on flawed research. In fact, the interventions regarding gratitude journals, awe, and talking to other people have all been supported by true experiments. (Follow the links for blog posts from everydayresearchmethods about these topics.)
f) Do you think there's any potential harm to presenting a poor design? How might this article affect people's impression of psychology research or of research on well-being?
g) In their article, the GGSC presented limitations. Read the passage and ask yourself, do they acknowledge their "study"'s internal validity problems? If not, how would you rewrite this limitations section to include them?
While very encouraging, there are some limitations to interpreting the results shared above. Most importantly, the people giving these responses came to Big JOY voluntarily, which usually means that they were interested in and enthusiastic about the topic, which could bias them to report favorably on its impact. People whose responses are included in this analysis did not receive any material incentive or reward for participating in Big JOY, which suggests that they were personally motivated to complete all seven days, which is a sign of enthusiasm that can make claims about desirable outcomes murky. Were people saying they felt happier, more capable, and connected because they liked the experience or because doing the micro-acts causes change? We can’t answer this.