Posts Tagged “power”

Running Power Simulations with the Power of R!

Psychology, Science · · 4 comments

As one of the resident “stats people” (as opposed to “regular people”) in my department, one of the most common questions I get asked is about power analyses. This is partly because more psychology journals are requiring explicit discussions about statistical power. The problem is that if you do studies that require analyses anything more complicated than a t-test or correlations, things can get a little bit hairy. You can do calculations for analyses like regression, but the effect sizes used for these calculations (like Cohen’s f2) are often uncommon, to the point where you might not even be clear if you’re using the correct test or calculating the effect size correctly. And then, as you move up in complexity to analyses like multilevel models, SEM, etc., there isn’t even an analytic solution. That’s where power simulations come in.Continue Reading

R-index, unbiased, varying num. studies

Evaluating the R-Index and the P-Curve

Psychology, Science · · 4 comments

Several years ago, Uri Simonsohn (along with Leif Nelson and Joe Simmons) introduced the psychology community to the idea of p-hacking, and his related concept for detecting p-hacking, the p-curve. He later demonstrated that this p-curve could be used as an estimate of true effect size in a way that was better at correcting for bias than the common trim-and-fill method.

Now, more recently, Ulrich Schimmack has been making a few waves himself, using his own metric called the R-index, which he has stated is useful as a test of how likely a result is to be replicable. He has also gained some attention for using it as what he refers to as a “doping test”, to identify areas of research—and researchers themselves—that are likely to have used questionable research practices (QRPs) that may have inflated the results. In his paper, he shows that his R-index indicates an increase in QRPs from research in 1960 to research in 2011. He also shows that this metric is able to predict the replicability of studies, by analyzing data from the Reproducibility Project and the Many Labs Project.Continue Reading

Line in the sand

A Line in the Sand

Society · · Leave a comment

Every once in a while, a feminist here or there likes to come along and draw a line in the sand, declaring one side as “real feminism” and the other side as fake feminism, or kowtowing to the patriarchy, or…well, fill in the blank with your preferred negative term. Many feminists rightfully get uneasy about these sorts of things, myself included (who am I to say that your feminism is not “real”?). I typically think that labels are nebulous to begin with, and are generally used to describe some rough cluster of beliefs, values, and opinions. But I’d like to go against my uneasiness and perhaps my better judgment and draw a line in the sand for a moment, in the hopes of delineating one critical aspect of feminism. (Don’t worry; I’ll force myself to soften up a little bit at the end.)Continue Reading