Should I allow my direct effect to be moderated?

A Blog Companion to “How Does Model (Mis)Specification Impact Statistical Power, Type I Error Rate, and Parameter Bias in Moderated Mediation” by Jessica L. Fossum, Amanda Kay Montoya, and Samantha Anderson

Recently, a paper lead by my former graduate student, now Assistant Professor of Psychology at Seattle Pacific University, Dr. Jessica Fossum, was recommended at Peer Community in Registered Reports (PCI-RR). You can read the preprint here. The paper evolved from Jessica’s dissertation while in the QRClab. While I (Amanda Montoya) am writing this blog, I want to emphasize that Jessica did the lion’s share of intellectual and practical work on this project, and I am just so happy to have been along for the ride. Additionally, I want to acknowledge Dr. Samantha Anderson (ASU), who was on Jessica’s committee and then continued the work with us through this process with PCI-RR. Throughout this blog (and the paper), we use the model numbering system from the PROCESS macro (Hayes, 2021; https://processmacro.org/index.html).

There is a question I get a lot when I’m teaching about moderated mediation, that underlies the inspiration for this paper. While I had an intuition about the answer, I thought it would be valuable to get an empirical answer. In a moderated mediation analysis, we can allow any of the paths in the mediation model to be moderated. Often researchers have strong hypotheses about certain paths in the mediation pathway (either X-M, M-Y, or both) as being moderated (hence they estimate a moderated mediation model). But, the direct path from X-Y (controlling for M) is often less of a focus, and researchers are unsure whether they should allow it to be moderated.

This conundrum leads to a lot of questions via email or in class like “Should I estimate Model 7 or Model 8?”, “What’s better: Model 14 or Model 15?” or “How do I choose between Model 58 and 59?” The difference between each of these pairs is whether or not the direct effect is moderated.

In this paper, we explore what happens when the true (population) model is one of these, but the model we use to analyze it is not the right one (misspecification). While the paper looks at the implication of all sorts of misspecification, for the purpose of this blog I want to focus specifically on this question about the specification of the direct effect.

When considering this question about whether to allow the direct effect to be moderated, there are two hypotheticals that we may want to consider:

1.       What if the direct effect is truly moderated in the population?

2.       What if the direct effect is not moderated in the population?

For each of these hypotheticals, we must consider what the cost is, if we get the answer wrong.

In this context, Fossum et al. (2026) focuses on the index of moderated mediation, which is used to test whether moderated mediation occurs. For this index there are two outcomes of interest: power and bias. Power considers the case where the mediation IS moderated, and how often do we accurately detect that moderated mediation (true positive). Bias examines how far the estimate is from the population value on average across many simulated datasets. Typically, regression coefficients are unbiased, so evidence of bias in this case would be a concern.

Interestingly, what we found was that the impact of omitting a truly moderated path, depends on which paths in the true model and analysis model were allowed to be moderated. So, for example, if the path from X-M is moderated (Model 8), the answer may be different than if the path from M-Y is moderated (Model 14).

Model 7 vs. 8 (X –> M Moderated)

Truly moderated direct effects

If the direct effect is moderated in the population, but we do not allow it to be moderated in the analysis, what happens? Specifically, if the path from X-M and the direct effect are moderated in the population (Model 8) but we use Model 7 (which only moderates the path from X-M, not the direct effect) to analyze the data. We find that power to detect the index of moderated mediation is similar when using the correct model (Model 8) and the simpler model (not moderating the direct effect, Model 7). In fact, at small sample sizes (e.g., 100) the simpler model has higher power to detect the index of moderated mediation. So, it could even be better to use the simpler model, especially if sample size is small. But, this additional power comes at a cost: a small amount of parameter bias. While the correctly specified model (analysis using Model 8) resulted in no bias, the simpler model (Model 7) resulted in a small amount of bias. This bias was not sufficiently large to pass our preregistered threshold of problematic bias (20%).

Non-moderated direct effects

If the direct effect is not moderated in the population (i.e., Model 7 is true), but we fit a more complex model that allows the direct effect to be moderated (i.e., Model 8), what is the cost? Interestingly, we find that including this extra moderated direct effect does not lead to any parameter bias, but it can lead to slightly lower power to detect the index of moderated mediation. This makes sense because there’s an extra parameter floating around the in the model, which reduces degrees of freedom and increases sampling variability.

The Take Away

So what’s the conclusion: When considering models where the X-M path is moderated, if the direct effect is truly moderated omitting it results in a small amount of parameter bias which can lead to slightly higher power especially when sample size is small. Would I recommend omitting a moderated direct effect on purpose, to get this mild power benefit: no. But it’s good to know that in this case the choice makes a pretty minimal difference. On the flip side, if the moderated direct effect is extraneous, it does not bias the parameters, but can lead to a loss in power.

So what to do? Mostly, my takeaway from this study was in the case of a moderated X-M path, adding or omitting a direct effect has pretty minimal impact. Arguably, if you do not have a strong hypothesis about whether it exists, it may be best to just leave it out.

However, the same cannot be said for models where the M-Y path is moderated

Model 14 vs. 15 (M –> Y Moderated)

Truly moderated direct effects

In the case of a model where the M-Y path and the direct effect are moderated (Model 15), omitting the moderated direct effect (analysis using Model 14) can have notable impacts. We found that this specific pattern leads to concerning levels of parameter bias in the index of moderated mediation. This makes sense because the M-Y relationship and X-Y direct effect are estimated in the same model. So misspecifying one part will affect the other. Given the patterns of moderation that we generated (all paths were positive), this misspecification actually led to higher statistical power, but if coefficient sizes were mixed it terms of magnitude it’s very possible that this could also lead to lower power.

Non-moderated direct effects

The alternative, where the direct effect is not moderated but the M-Y path is (Model 14), then analyzed by including the moderated direct effect (Model 15), was less problematic. Like what we saw with the X à M path moderated models, adding an extra moderated direct effect does not increase bias but it does reduce power.

The Takeaway

In models where the X-M path is moderated, omitting an existing moderated direct effect or including a non-existence moderated direct effect has a fairly minimal impact. But the same is not true for models where the M à Y path is moderated. This makes sense because X à M is estimated independently from the direct effect, whereas M à Y is estimated in the same regression model. The risk in the case of models with the M à Y path moderated, is that if it exists and we exclude it, parameters can become very biased, and this can impact statistical power.

So, in the case of the M-Y path moderated, I would recommend to air on the side of inclusion. For researchers unsure whether to include a moderated direct effect, include it. The cost of excluding it if it exists is too high and having something extra in the model is not too problematic, especially if you have a large sample size.

Impact of...	When the direct effect...	Models with X —> M Moderated	Models with M —> Y Moderated
Including moderated direct effect	Does not Exist	No parameter bias Slightly lower power	No parameter bias Slightly lower power
Excluding moderated direct effect	Exists	Small parameter bias Slightly higher power	High parameter bias Higher or lower power depending on direction of effects

Wrapping Up

There is much more in the paper about different patterns of misspecification. We also looked at the case where both the X-M and M-Y path is moderated, which behaves more like the case where M à Y is moderated. This suggests that any model with a moderated M à Y path might benefit from inclusion of the moderated direct effect, unless the research team has strong evidence (independent of the sample) to suggest that path is not moderated.

While we did not discuss this much in the paper, I want to explicitly say that I discourage the practice of fitting models and then omitting non-significant moderated paths. This biases the p-values of the other paths in the model. Especially if this process is not transparently reported it can misrepresent the models as confirmatory, which leads to many of the types of problems that underly the replication crisis. I have worked with teams before that don’t have clear ideas about which paths are moderated, and I typically recommend doing a kind of split half analysis if the sample size is large enough: split the data in half and with the first half fit a very general model (e.g., Model 59) and use defined criteria (e.g., effect size) to determine which paths will be moderated in the second half of the data. This avoids biasing p-values and other issues with inference. Alternatively, two studies could be run in sequence if one study’s sample size is not sufficiently large to warrant splitting the data.

In the end, my answer to the question of whether you should allow the direct effect to be moderated is “it depends.” If you have strong independent evidence that it should or should not, then follow that evidence. Likely if that was the case then you wouldn’t be asking the question, so for those who are asking the question: If your analytical model allows only the X à M path to be moderated, inclusion or exclusion of a moderated direct effect has a somewhat limited impact, especially if your sample size is large. But for those with an analytical model that moderates the M à Y path, lean toward including the moderated direct effect as this will reduce parameter bias and undue impacts on statistical power.

To cite the paper: Fossum, J. L., Montoya, A. K., & Anderson, S. F. (2026, January 13). How Does Model (Mis)Specification Impact Statistical Power, Type I Error Rate, and Parameter Bias in Moderated Mediation? A Registered Report. Retrieved from osf.io/preprints/psyarxiv/prqsg_v3