The replication crisis is real, but I’m going to give some pushback on the “ssssh” like it’s some kind of conspiracy “they” don’t want you to know about™. We live in an era of unprecedented and extremely dangerous anti-intellectualism, and pushing this as some kind of conspiracy is honestly really gross.
The entire reason the crisis became known is because scientists have and are having the integrity to try to reproduce results from existing studies. They want the science in their field to be sound, and they’ve been extremely vocal about this problem from the minute they found it. This wasn’t some “whistleblower” situation.
Arguably a major reason why it took so long for this to come to the fore is because government agencies which administer grants focus much less on reproducing previous experiments and more on “new” stuff. This would ironically be much less of a problem if more funds were allocated for scientific research (i.e. so they weren’t so competitive that researchers feel the need to publish “new” research lest their request be denied). This “ssssh” rhetoric makes the voting public want the exact opposite of that because it tells them that their tax dollars are being funneled into some conspiratorial financial black hole.
This happens in large part because concrete, reproducible research on humans is extremely hard, not because the researchers lack integrity and just want to publish slop. In CS, I can control for basically everything on my computer and give you a mathematical proof that what I wrote works for everything every time. In physics, I can give exact parameters for my simulation or literal schematics for my device. A psychological or sociological experiment is vastly more difficult to remove confounding variables from or to properly document the confounding variables in.
This doesn’t invalidate soft sciences like anti-intellectuals would want you to believe. While some specific studies may not be reproducible, this is why meta-analyses and systematic reviews are so important in medicine, psychology, sociology, etc.: they give the “average” of the existing literature on a specific subject, so outliers get discovered, and there’s far more likelihood that their results are correct or close to correct.
This is actively being worked on, and researchers are more aware of it than ever – making them more cognizant of the way they design their experiments.
You understand that the “hard sciences” are also affected by this crisis, correct? “Soft science” is a borderline meaningless term that stigmatizes entire fields of science to the sole benefit of anti-intellectuals.
Even when we take into consideration that the problem is currently worse in sciences like psychology, economics, sociology, etc.: “these results support the scientific status of the social sciences against claims that they are completely subjective, by showing that, when they adopt a scientific approach to discovery, they differ from the natural sciences only by a matter of degree.” Social sciences are science.
You don’t belong to “the hard sciences crowd”; you belong to a Sheldon Cooper-esque stereotype who devalues work you don’t understand.
Love the write-up, well done. These issues are huge, complex, fascinating, and depressing. It’s always worth defending science, and you’re right - this is basically the opposite of a conspiracy. Experts are actively screaming “something is wrong here!”
But, yeah, wow. What a shit take. Psychology is not science, from someone married to a psychologist? Soft sciences aren’t science?
Correct me if I’m wrong, but Charles Darwin’s science was as soft as it gets! He didn’t have p-values, he had pretty birds with funky beaks. One of the most important scientists to ever live, and his masterpiece did not have a single quantitative model.
Just because psychology got irrationally stuck on Freud for so long doesn’t mean it’s not science. We all learn about Lamarckian inheritance and think it’s goofy as shit, doesn’t mean we dismiss the entire field of biology.
No, the difference in the replication crisis between the soft “sciences” and the hard is enormous. The soft are basically producing results equal to making coin tosses.
You have clearly never actually done “hard sciences” research in any meaningful way if this is your take. And computer science does not count as a science at all, it is more like engineering. Mathematics is a “hard science” that can be implemented through computer science, and physics is a “hard science” that can be implemented through electrical engineering (and as a subset computer engineering).
But even then mathematics is closer to philosophy and logic than any of the physical sciences. The physical sciences like physics, chem, bio are very different due to their experimental nature, and how sensitive they can be to specific conditions of the experiments. And the more complex the system being studied is, the harder it is to control variability which is why the social sciences like psychology and economics are working on incredibility difficult problems in systems we do not currently fully understand, and are more vulnerable to difficult reproducing and replicating the conclusions.
This is in contrast to computer science where we fully understand the system because humans have built it, and it is a machine built on the principles discovered by physicists and implemented by electrical engineers to run calculations that are created by mathematicians.
The replication crisis is real, but I’m going to give some pushback on the “ssssh” like it’s some kind of conspiracy “they” don’t want you to know about™. We live in an era of unprecedented and extremely dangerous anti-intellectualism, and pushing this as some kind of conspiracy is honestly really gross.
I don’t consider Psychology to be a scientific discipline - I belong to the hard sciences crowd.
My wife is a psychologist.
I wonder if she regrets her marriage, or if she’s trying to fix you.
Hey, maybe it’s both!
You understand that the “hard sciences” are also affected by this crisis, correct? “Soft science” is a borderline meaningless term that stigmatizes entire fields of science to the sole benefit of anti-intellectuals.
Even when we take into consideration that the problem is currently worse in sciences like psychology, economics, sociology, etc.: “these results support the scientific status of the social sciences against claims that they are completely subjective, by showing that, when they adopt a scientific approach to discovery, they differ from the natural sciences only by a matter of degree.” Social sciences are science.
You don’t belong to “the hard sciences crowd”; you belong to a Sheldon Cooper-esque stereotype who devalues work you don’t understand.
Love the write-up, well done. These issues are huge, complex, fascinating, and depressing. It’s always worth defending science, and you’re right - this is basically the opposite of a conspiracy. Experts are actively screaming “something is wrong here!”
But, yeah, wow. What a shit take. Psychology is not science, from someone married to a psychologist? Soft sciences aren’t science?
Correct me if I’m wrong, but Charles Darwin’s science was as soft as it gets! He didn’t have p-values, he had pretty birds with funky beaks. One of the most important scientists to ever live, and his masterpiece did not have a single quantitative model.
Just because psychology got irrationally stuck on Freud for so long doesn’t mean it’s not science. We all learn about Lamarckian inheritance and think it’s goofy as shit, doesn’t mean we dismiss the entire field of biology.
No, the difference in the replication crisis between the soft “sciences” and the hard is enormous. The soft are basically producing results equal to making coin tosses.
You have clearly never actually done “hard sciences” research in any meaningful way if this is your take. And computer science does not count as a science at all, it is more like engineering. Mathematics is a “hard science” that can be implemented through computer science, and physics is a “hard science” that can be implemented through electrical engineering (and as a subset computer engineering).
But even then mathematics is closer to philosophy and logic than any of the physical sciences. The physical sciences like physics, chem, bio are very different due to their experimental nature, and how sensitive they can be to specific conditions of the experiments. And the more complex the system being studied is, the harder it is to control variability which is why the social sciences like psychology and economics are working on incredibility difficult problems in systems we do not currently fully understand, and are more vulnerable to difficult reproducing and replicating the conclusions.
This is in contrast to computer science where we fully understand the system because humans have built it, and it is a machine built on the principles discovered by physicists and implemented by electrical engineers to run calculations that are created by mathematicians.