[ 3 / biz / cgl / ck / diy / fa / ic / jp / lit / sci / vr / vt ] [ index / top / reports ] [ become a patron ] [ status ]
2023-11: Warosu is now out of extended maintenance.

/sci/ - Science & Math

View post   
File: 121 KB, 659x729, reproducibilitycrisis.jpg [View same] [iqdb] [saucenao] [google]
11967828 No.11967828 [Reply] [Original]

Where did it all go so wrong?

>> No.11967839

>>11967828
but can they reproduce the experiments conducted by this research?

>> No.11967862

>>11967828
Vested interest by multinational pharmaceutical and chemical companies. for instance: very little advancement in heart meds for 20+ years, keep tweaking analogs so they can keep pushing patented meds instead of perfectly useful (often more effective) non-patented ones

>> No.11967869

Why are they lumping in all sciences?

>> No.11967892

>>11967828
Publish or perish leads to an environment where fudging results is better than having no publishable results. This leads to our current situation.

>> No.11967893

Number one problem from the ground: the peer review process. You have tiny little page limits (8-12 pages seems like a lot, but it's fucking nothing when you're exploring a complex problem). So you cut corners where you can (background, minutae of the procedure) in hopes of meeting the arbitrary page limit. Then, you argue with the other grad students reviewing your paper about shit that they don't understand because they couldn't be bothered to read the fucking manuscript, and you end up cutting even more to fit in some bullshit to address their asinine questions and push through the approval process. On top of all that, there is no room in any reputable journal for "incremental" work, so you have to hit the reader over the head with all the reasons your approach is amazing and novel in every paragraph, further reducing space for actual content that is relevant to your work.

It's no surprise that there is not room for critical information required to reproduce the experiment, when everything else about the process incentivizes not including it.

>> No.11967900

>>11967828
when getting into college became a political statement instead of only something reserved for the people who should be there
thank the civil rights movement

>> No.11967970

>>11967893
This. It really shows in the higher impact journals where the methods, the most critical part of good science, are entirely relegated to SI

>> No.11967978

>>11967828
This is almost exclusively applied to soft sciences.

>> No.11968382

>>11967828
Idk man, I prove trivial theorems so it's not very hard to reproduce my proofs.
t. theoretical physicist

>> No.11968496

>>11967893
>page limits
This isn't the 90s anymore, very few journals have page limits these days

>> No.11969130

>>11967828
>medicine
That’s your problem. This has been known for years
>2017
Well there you go. The problem is that certain pharmaceutical companies and medical researchers did something called p hacking to publish. This was not all science, this was medical journals. You can go read up on it.

>> No.11969545

>it's peer reviewed, therefore it is truuue!

>> No.11969551

This happens from time to time even in Robotics (easy experiments to delimit, negligible uncertainty), it's unnerving.

>> No.11969643

>>11967828
Most of the unreproducible experiments are social science and psyhology. Hard science is as reproducible as ever

>> No.11969658

>>11967828
see >>11967862
But also the communist influences on think tanks and control of info didn't help.

>> No.11969663

>>11968496
Yeah, only the reputable ones do

>> No.11969699

>>11967978
Not true, it is just significantly worse in the soft sciences than the hard ones.
I have read many new release materials research papers that have been absolute dogshit and have been pushed through to preserve funding for the department they came from. Unrepeatable experiment, incomplete synthesis routes, variables that were unaccounted for. I also did a lot of work for grad students performing materials research and the lack of consistency of how they conduct their experiments is very concerning.
E.g.
>dude is doing phase experiments with ceramics over long periods at high temps
>tell them that operating the furnace at its max temp for that long will likely cause a failure at some point prior to the time he wanted
>didnt care, did it anyways
>1/4 the way though one thermocouple dies, temp starts to fluctuate by a couple hundred C which significantly impacts growth mechanisms
>later that week other thermocouple dies and the furnace goes cold
>him and the PI said fuck it 'we dont have the money to redo it'
>full send the thesis without accounting for any of the fuckery that occured and 'extrapolated' information that wasn't there
>gets published

>someone tries to repeat experiment in future
>I wonder why my results dont match!!!

Not to mention they did not account for the diffusion of the crucible material into the tested material that was fairly significant at those timescales and temperatures.
But what do I know, I don't have a PHd

>> No.11969863

>>11967893
>On top of all that, there is no room in any reputable journal for "incremental" work, so you have to hit the reader over the head with all the reasons your approach is amazing and novel in every paragraph
Trying to sound more amazing than you are also leads to glossing over or downplaying issues when writing about your experiments. The better and more solid your shit sound more likely you'll get published or convince people that you're onto something. Then someone tries your experiments and runs into the same problems but doesn't just ignore them and instead deems the experiment doesn't work.

A lot would be fixed if we could kill off the largest scientific journals, move away from paywalled articles and shit like impact factor.

>> No.11969877

>>11967828
When all of science became a race to get grants driven by a bunch of retards in university administrations rather than the pursuit of knowledge for sake of knowledge.

>> No.11969930

>>11969643
>Hard science is as reproducible as ever
Big ol [citation needed] right there. I'm happy to be shown wrong, but I honestly think that this problem hasn't been charted systematically in the hard sciences.

>> No.11970764

>>11969699
Sounds like you might some personal experience with this subject.