It is time for another joint blog post by Beth Morling and Jess Hartnett! It’s about a recently-published article that allows you to think about everything from the ethical use of AI to factorial ANOVA.

Part I: Student Learning Activity

When we scroll through the positive and negative comments below someone’s blog or Instagram post, we might find the negative comments funny or the positive comments naive. It’s secretly entertaining, in a way, to see what people write. 

But what if we are reading the comments on our own posts? How might that affect our moods?  A recent publication in the journal Nature: Scientific Reports tested what would happen. 

The scientists had participants in their study pretend to create little blog posts by asking them In to select one blog post among two options, and imagine they had written it.  Then they clicked a button that would “publish” them. A few minutes later, participants got to read some feedback on their posts, delivered in a simulated comments section. The 10 comments they got on each post were either all positive, all negative, or all neutral. 

a) Before reading on, make a prediction: How do you think reading such comments would make you feel? And, which type of comment–positive or negative–would have a stronger effect on your mood?
Finally, do you think there might be gender differences in how people react to the comments? Will one gender react more strongly than the other?

The scientists tested their hypotheses using a factorial design.

There is a popular press summary of this study here.  But you can handle reading some text from the actual empirical journal article. Here’s how the researchers themselves described their study in the paper’s Abstract: 

This study investigates how negative comments on social media affect adults’ anxiety and mood. In an experimental study involving 128 adult participants (85 female, 43 male), individuals were asked to share blog posts on a simulated internet forum. Subsequently, they were exposed to either negative, neutral, or positive comments, and their [positive] mood and anxiety levels were measured using validated scales [mood and anxiety ratings could range from 1 (low) to 4 (high)]. Results showed that negative comments significantly increased anxiety and decreased [positive] mood compared to neutral or positive comments, while gender did not show any significant effects.

b) Re-read the text above and focus on identifying the main variables of the study. Classify each variable on the table below. Here’s a hint: The study had two IVs (that is, two factors) and two dependent variables. In addition, one of the two IVs (one of the factors) was actually a Participant Variable–a variable that acts like an IV but whose levels are actually not manipulated. 

Variable name	Levels of this variable	Were the levels of this variable manipulated or measured?	Was this an Independent variable (IV)? A participant variable (PV)? Or a dependent variable?	For IVs and PVs: was it independent groups or within groups?

c) Looking back at your table, identify the type of design.  Your answer should take this format: “This was a ___ X ___ [independent groups/within groups/mixed] factorial design” (for example you might say, “This was a 2×2 mixed factorial design”).

d) How many cells, or different groups, does this design create?  (For example, a 2×2 mixed factorial design would create 4 cells).

Now, re-read this quote from the abstract that describes the results:

Results showed that negative comments significantly increased anxiety and decreased mood compared to neutral or positive comments, while gender did not show any significant effects.

It’s a good time to pop over to the article’s Figure 2 so you can see how these results are graphed. Figure 2 shows one DV (anxiety)on the left and the other DV (pleasant mood) on the right . 

e) Based on this description and the pattern in Figure 2, does the study show a main effect of comment type? Why or why not?
Does the study show a main effect for gender? Why or why  not?
How about an interaction of gender and comment type? Is there a difference in differences?

f) Look carefully at Figure 2 and see if you can detect whether the positive or the negative comments made the bigger difference on mood. Explain your answer. (Here’s a hint: the “neutral” condition plays a role in answering this question.)

g) Bonus– challenge question! Here’s an additional detail about the study. Participants pretend-posted four different blog posts.  And, depending on the condition they had been randomly assigned to, they received 10 comments on each post (that’s 40 total comments). Because comment valence was an independent-groups IV, that means that participants who were in the positive condition read 40 positive comments, and those in the negative condition read 40 negative comments, and so on. What do you think about this manipulation? You might consider the strength of the manipulation as well as ecological realism (i.e., similarity to the real-world). 

Part 2: Ideas for Instructors

Here are some ideas for how Instructors could use this article in class.  

1. After describing the results, you could discuss the real-life implications of its findings:
   a)Ask your students to think about and discuss their own snarky internet habits. Are there any forums or social media sites they follow for the negative comments??
   When Dr. Hartnett used this in class, she learned a lot about the subreddits r/WallStreetBets and r/fatsquirrelhate, as well as the ongoing beef between Cardi B and Nicki Minaj
   
   b) Based on the findings from this research, do your students think that internet snark is good or bad for their moods?
   
   c) What could this imply for the internet’s ability to control the emotions of the masses? 

2. You could discuss the authors’ use and documentation of A.I. to create research material for this study.

After the initial shock of the AI revolution, people began discussing the potential of AI as a personal assistant. This study provides an example of using AI to generate research stimuli. Specifically, they used AI to draft the pretend blog posts that participants used, and used it to draft the positive/negative/neutral comments on the blogs. The article also serves as an example of how to properly cite the AI prompts used to generate research materials. The authors described their process in the Methods section and shared their complete prompts on Open Science Framework. 

In class, you could invite your students to visit OSF and view the actual documentation used by the authors. You could then discuss why it is essential to cite AI in this way (and why the prompt process is crucial for successful use of AI).

3. OSF/open science/data sharing

As mentioned in the previous question, the authors have shared their data on OSF (https://osf.io/ef4nd/overview). You can use this in many ways:

a) Introduce your students to OSF. Talk about the advantages of pre-registration hypotheses. Discuss how sharing research materials encourages transparency and replication.
b) Speaking of shared research materials…these researchers shared their data, allowing you and your students to analyze and interpret the data.
c) Read and discuss the specific prompts that the authors used to generate research materials in AI. This can segue into a broader discussion about properly citing AI in their own work.

Reference

Ai, Y., von Mühlenen, A. (2025) An experimental online study on the impact of negative social media comments on anxiety and mood. Scientific Reports, 15, 26642. https://doi.org/10.1038/s41598-025-10810-8
^^It’s open access!