Research Interests
Psychology of Science - Facilitating Belief Change
Calibrating beliefs with reality is a challenging task. People are adept at explaining outcomes in ways that do not conflict with (or even reinforce) preexisting beliefs. For instance, when someone has an experience that runs counter to their expectations or beliefs, they may dismiss that experience as not diagnostic for that belief or shift their expectations after the fact to align with what they experienced.
In one line of research, I have investigated pre-commitment to predictions as an intervention to counter these processes and facilitate belief change. In several studies, I have had participants evaluate their own abilities before and after testing those abilities. Before completing the test, participants make predictions about their expected performance. I have found that those who explicitly pre-commit to their predictions (e.g., report them before the test) update their evaluations of their abilities to be more in line with their performance on the test. Conversely, those who are allowed to report their predictions after completing the test update their evaluations less and instead alter their predictions to be more in line with their performance on the test. This suggests that pre-commitment to predictions may aid in calibrating beliefs with reality by limiting one's ability to reconstrue predictions in line with observed outcomes.
Calibrating beliefs with reality is a challenging task. People are adept at explaining outcomes in ways that do not conflict with (or even reinforce) preexisting beliefs. For instance, when someone has an experience that runs counter to their expectations or beliefs, they may dismiss that experience as not diagnostic for that belief or shift their expectations after the fact to align with what they experienced.
In one line of research, I have investigated pre-commitment to predictions as an intervention to counter these processes and facilitate belief change. In several studies, I have had participants evaluate their own abilities before and after testing those abilities. Before completing the test, participants make predictions about their expected performance. I have found that those who explicitly pre-commit to their predictions (e.g., report them before the test) update their evaluations of their abilities to be more in line with their performance on the test. Conversely, those who are allowed to report their predictions after completing the test update their evaluations less and instead alter their predictions to be more in line with their performance on the test. This suggests that pre-commitment to predictions may aid in calibrating beliefs with reality by limiting one's ability to reconstrue predictions in line with observed outcomes.
Metascience - Testing Moderators of Replicability
Replication is a key part of acquiring scientific knowledge. Replications can provide greater certainty about a phenomenon or identify conditions under which a phenomenon will not be observed. Both of these outcomes help scientists to calibrate their beliefs about phenomena and theories. Certain features of replication efforts may systematically moderate the replicability of prior findings across content areas of research.
One proposed moderator, time-of-semester variation, claims that effects may vary in their detectability throughout academic terms. A great deal of research relies on undergraduate participants, whose timing of participation is self-selected. This self-selection may lead to different “types” of undergraduates participating at different times (e.g., highly-motivated participants participate early in the academic term, whereas “slackers” wait until the end of the term) which could moderate psychological findings. To test this moderator, I led a crowdsourced replication project involving teams from 20 universities replicating 10 effects across an entire academic term (Ebersole et al., 2016). In support of time-of-semester participant differences, we did observe variation in participant characteristics over the semester (e.g., highly conscientious individuals do participate earlier than their less conscientious peers). However, these changes in participant characteristics did not impact the sensitivity of experimental paradigms. Effect detectability remained largely stable throughout the semester. Overall, popular beliefs about time-of-semester variation seem to have some basis in reality - participant characteristics slightly vary over the semester. However, among our replications, these variations did not moderate effects.
I am currently investigating another proposed metascientific moderator of replicability - the role of expertise in replicability. As one possible example of this moderator, there were a subset of replications from the Reproducibility Project: Psychology (RP:P) that had methodological issues identified by the original authors prior to data collection but that were unresolved. All but one of these replications failed to replicate the original finding. I am currently leading a project with around 150 collaborators from 90 universities around the world that will follow up on these studies. For each study, we are conducting a replication using the methods used in RP:P and a second replication using methods that have undergone formal expert review. If expertise in designing studies moderates their results, we would expect to see a higher rate of replicability from our revised paradigms compared to those used in RP:P. The results of this project will help us to better calibrate beliefs about phenomena following replications and perhaps provide guidance on the best ways to design replication studies.
Replication is a key part of acquiring scientific knowledge. Replications can provide greater certainty about a phenomenon or identify conditions under which a phenomenon will not be observed. Both of these outcomes help scientists to calibrate their beliefs about phenomena and theories. Certain features of replication efforts may systematically moderate the replicability of prior findings across content areas of research.
One proposed moderator, time-of-semester variation, claims that effects may vary in their detectability throughout academic terms. A great deal of research relies on undergraduate participants, whose timing of participation is self-selected. This self-selection may lead to different “types” of undergraduates participating at different times (e.g., highly-motivated participants participate early in the academic term, whereas “slackers” wait until the end of the term) which could moderate psychological findings. To test this moderator, I led a crowdsourced replication project involving teams from 20 universities replicating 10 effects across an entire academic term (Ebersole et al., 2016). In support of time-of-semester participant differences, we did observe variation in participant characteristics over the semester (e.g., highly conscientious individuals do participate earlier than their less conscientious peers). However, these changes in participant characteristics did not impact the sensitivity of experimental paradigms. Effect detectability remained largely stable throughout the semester. Overall, popular beliefs about time-of-semester variation seem to have some basis in reality - participant characteristics slightly vary over the semester. However, among our replications, these variations did not moderate effects.
I am currently investigating another proposed metascientific moderator of replicability - the role of expertise in replicability. As one possible example of this moderator, there were a subset of replications from the Reproducibility Project: Psychology (RP:P) that had methodological issues identified by the original authors prior to data collection but that were unresolved. All but one of these replications failed to replicate the original finding. I am currently leading a project with around 150 collaborators from 90 universities around the world that will follow up on these studies. For each study, we are conducting a replication using the methods used in RP:P and a second replication using methods that have undergone formal expert review. If expertise in designing studies moderates their results, we would expect to see a higher rate of replicability from our revised paradigms compared to those used in RP:P. The results of this project will help us to better calibrate beliefs about phenomena following replications and perhaps provide guidance on the best ways to design replication studies.
Implicit Social Cognition
Many individuals genuinely desire and purport to have egalitarian values and attitudes toward members of other groups. However, these beliefs that we are fair and unbiased are not always supported by indirect measures of attitudes and judgments. I have investigated the gaps between explicit attitudes (preferences that can be consciously reported and endorsed) and implicit attitudes (preferences outside of conscious awareness and control) and behavior, identifying where such gaps occur and developing new ways to reliably detect biases.
As one demonstration, my collaborators and I have investigated patterns of implicit and explicit attitudes toward racial and religious group in large online samples (Axt, Ebersole, & Nosek, 2014). While explicit attitudes do not follow reliable patterns, we have found that implicit attitudes follow reliable hierarchies. Among both racial and religious groups, individuals implicitly preferred their group most, but then showed biases in favor of more dominant groups in society (e.g., White people, Christians) over less powerful or more maligned groups (e.g., Black and Hispanic people, Muslims). These hierarchies existed for people of all races and religions. The pervasiveness of these patterns suggests that social status may be represented in implicit cognitions on a societal level.
Implicit attitude measures have provided researchers with adaptable and validated tools for indirectly assessing intergroup bias. By comparison, there is a relative lack of behavioral or judgment measures that can be flexibly adapted to many intergroup contexts and reliably measure bias. To help address this issue, my collaborators and I have developed a repeated judgment task in which participants make a series of accept or reject decisions for admitting candidates to a fictional group (e.g., Axt, Ebersole, & Nosek, 2016). Importantly, this task allows for the social categories of those candidates to be easily manipulated, allowing researchers to measure the leniency with which participants admit members of each group. This task has been applied to measure intergroup bias in the contexts of race, university membership, and physical attractiveness and has the flexibility to measure bias in many other domains.
Many individuals genuinely desire and purport to have egalitarian values and attitudes toward members of other groups. However, these beliefs that we are fair and unbiased are not always supported by indirect measures of attitudes and judgments. I have investigated the gaps between explicit attitudes (preferences that can be consciously reported and endorsed) and implicit attitudes (preferences outside of conscious awareness and control) and behavior, identifying where such gaps occur and developing new ways to reliably detect biases.
As one demonstration, my collaborators and I have investigated patterns of implicit and explicit attitudes toward racial and religious group in large online samples (Axt, Ebersole, & Nosek, 2014). While explicit attitudes do not follow reliable patterns, we have found that implicit attitudes follow reliable hierarchies. Among both racial and religious groups, individuals implicitly preferred their group most, but then showed biases in favor of more dominant groups in society (e.g., White people, Christians) over less powerful or more maligned groups (e.g., Black and Hispanic people, Muslims). These hierarchies existed for people of all races and religions. The pervasiveness of these patterns suggests that social status may be represented in implicit cognitions on a societal level.
Implicit attitude measures have provided researchers with adaptable and validated tools for indirectly assessing intergroup bias. By comparison, there is a relative lack of behavioral or judgment measures that can be flexibly adapted to many intergroup contexts and reliably measure bias. To help address this issue, my collaborators and I have developed a repeated judgment task in which participants make a series of accept or reject decisions for admitting candidates to a fictional group (e.g., Axt, Ebersole, & Nosek, 2016). Importantly, this task allows for the social categories of those candidates to be easily manipulated, allowing researchers to measure the leniency with which participants admit members of each group. This task has been applied to measure intergroup bias in the contexts of race, university membership, and physical attractiveness and has the flexibility to measure bias in many other domains.