I think if you asked most students if they think women professors are worse teachers than men professors, they’d say “no” emphatically. None of us wants to be biased–we want our ratings of faculty members to reflect their actual teaching ability.

That’s why this study might come as a (disappointing) surprise. The research was covered by PsyPost here.(The empirical article is linked here.) You’ll see that the study’s purpose emerged out of the context of Italian University philosophy professors:

Our project began with personal and anecdotal evidence. In our daily experience, we noticed that students often behaved differently depending on whether they were interacting with men or women professors and these patterns varied with the gender of the students themselves”….We wanted to understand whether these impressions were backed up by systematic data and we were surprised to discover that, in the Italian context, no such data existed. This absence in itself seemed telling and it convinced us that we needed to design and conduct our own studies in order to investigate the phenomenon directly.

The research team conducted two separate experiments. In both studies, participants were philosophy students or recent graduates from Italian universities. The students were asked to evaluate short lecture excerpts that were identical in content but varied in the gender of the professor attributed to them.

In the first study, 95 participants read four lecture excerpts on philosophical topics, including Aristotle’s ethics and Bourdieu’s concept of symbolic power. [Students thought each excerpt was from a different professor.] Each excerpt was randomly assigned a fictitious male or female name. [Excerpts were counterbalanced across the assigned gender of the professor.] After reading each passage, participants answered questions about the lecture’s clarity, interest, perceived competence, self-confidence, care, and overall engagement they felt toward the lecturer. 

Here is a description of the results:

In the first study, male participants consistently rated lectures more favorably when they were attributed to a man. This was true across several key dimensions including clarity, interest, competence, self-confidence, and perceived benefit. Men also showed a greater willingness to take a full course with a male professor. The only area where they rated women higher was in perceived care, consistent with stereotypes that associate women with nurturing roles.

In contrast, women participants in the first study showed little gender bias in their evaluations […].

1. Re-read the text above and focus on identifying the main variables of the study. Classify each variable on the table below. Here’s a hint: The study had two IVs (that is, two factors) and several dependent variables. In addition, one of the two IVs (one of the factors) was a Participant Variable–a variable that acts like an IV but whose levels are actually not manipulated. 

Variable name	Levels of this variable	Were the levels of this variable manipulated or measured?	Was this an Independent variable (IV)? A participant variable (PV)? Or a dependent variable?	For IVs and PVs: was it independent groups or within groups?

b. This was a 2×2 factorial design. Consult your table above and decide: Was it a 2×2 independent groups factorial? A 2×2 within groups factorial? Or was it a 2×2 mixed factorial? Explain your answer.

c. In this design, take a look at the gender factors. There are two!  One of them is a true IV and one is a PV.  Can you explain why?It might help to label one of them, ‘Target Gender” and the other one “Perceiver Gender”.  

d. Sketch a graph of the results, using “clarity” as your DV. Put gender of professor on the x-axis and gender of participant as two different colors/line types.

Let’s keep going: There was a second study. Here is the description:

The second study involved 92 participants and used the same lecture excerpts, but this time delivered as audio recordings by voice actors selected to represent typical male and female vocal characteristics. Again, participants evaluated each lecture on the same seven dimensions…

[The results indicaed that] both male and female participants rated male professors higher across nearly all dimensions, including clarity, interest, competence, and self-confidence. Women were still rated more highly on care. 

e. Make sure you know the difference in method between the first and second study. Which one do you think has a better method? Why?

f. This was also 2×2 factorial design. Same as before, decide: Was it a 2×2 independent groups factorial? A 2×2 within groups factorial? Or was it a 2×2 mixed factorial? Explain your answer.

g. Sketch a graph of the results, using “clarity” as your DV again. Put gender of professor on the x-axis and gender of participant as two different colors/line types.

h. Speculate: Why might the two studies have produced different results?

i. Which of these studies has more ecological validity—that is, which is more similar to the real world, and why?

j. The study was conducted on Italian students—do you think this matters? Why or why not?

k. A skeptical reader might say the following: “Maybe the women professors really were less clear than the men professors, and that’s why they were rated lower?” What do you say in response—how does the study’s design rule out this critique? Which of the four Big Validities is this question getting at?

l. Similarly, a critical reader might say, “Maybe the students who rated the male professors as being more clear were just smarter students—could that explain their ratings?”  How does the study’s design rule out this critique?