[Edit 1/11/13]: It's been several days and the hashtag is continuing as strong as ever. There's now a twitter account devoted completely to overly honest methods, and a cute little cartoon. There's also been a blog trying to stir up trouble by writing about the potential for backlash, where the only backlash so far is this blog post itself. I'd like to think there's been no backlash because most people on Twitter have a healthy sense of humor and appreciation for the realities of science, rather than because not enough people have seen #overlyhonestmethods yet.
By the way, this list is only semi-complete; I've left out a few tweets that I don't understand, don't think are funny, or don't think say anything interesting. But it's still the most complete list I've seen so far. (Other lists include Huffington Post's, The UK Telegraph's, io9's, ASBMB Today, Scientific American Blogs', Beckie Port's, Occam's Typewriter's, Vit's, and Anne Buchanan's, which includes some interesting reflections on the whole phenomenon. Oscillatory Thoughts composed his own "confessions" in the same style for neuroimagers). A political science blog has collected social science ones here and here. My comments on selected tweets are in brackets.
1) Science involves laziness and human error, because it's conducted by tired, rushed, overworked human beings. In this respect, science isn't so different from any other career.
julezyme
Sara
Callori @SaraDoesScience: Sample was air dried at
room temperature because I put them in the heater but forgot to turn it
on. #overlyhonestmethods [another candidate for #adhdscience]
RT @GavinHub:
I didn't label the tubes, but by looking at the results, I'm pretty sure this
one is the control#OverlyHonestMethods [#adhdscience redux]
Bill
Manley @disgruntledbill: I used half as much
enzyme because I forgot to order more #overlyhonestmethods [#adhdscience]
Dominik @comfort_eagle: I had to exclude 2 subjects
from further EEG analysis because I forgot to press the "record"
button . #overlyhonestmethods [I'm pretty sure this #adhdscience actually happened with one participant in the experiment I'm currently analyzing, as no data got recorded from them].
janisue jones @janisuejones: We used 50μl of positive control instead of 30μl because it is going bad and we forgot to order more. #overlyhonestmethods
Jennifer
Schmitt @JenSchmitt618: That survey question
was written that way because we finished the questionnaire at 2 am and left for
the field at 7 am#overlyhonestmethods
Paul
Macklin @MathCancer: This figure is not
really relevant to the method, but making it took way too much effort to leave
out. #overlyhonestmethods
Prerana
Shrestha @prerana123: all behavior
procedures were carried out in the order of least stressful to most stressful
for the experimenter.#overlyhonestmethods
Emily
Burns @emilyredhead: #overlyhonestmethods a
16h incubation was carried out because that's how many hours there are between
5pm and 9am.
Alex Chase @aechase: Slices were left in a
formaldehyde bath for over 48 hours, because I put them in on Friday and refuse
to work weekends.#overlyhonestmethods
Geeky
Girl Engineer @gkygirlengineer: Samples were incubated
for however long our time-wasting, productivity-sucking lab meetings
lasted. #overlyhonestmethods
julezyme @julezyme: The reaction mix was
incubated for as long as it took me to go to the toilet and check my
email. #overlyhonestmethods
Kirkwood
Lab @ UOIT @KirkwoodLab: Study lake selection was
based on phys-chem criteria and the absence of annoying, barking dogs from
local cottages#overlyhonestmethods
Dr Caz @fruitflyCaz: I copy-pasted most of
this methodology section from my previous paper on a similar topic #OverlyHonestMethods [I've done this before...]
Melanie
Slabinski @MissSlabby: “@The_Shed: After the biological sample was
dropped on the floor, the 3 second rule was applied. #overlyhonestmethods”
Scott
Paquette @scottpaq: Only papers available
online were cited in the lit review. I haven't found the campus library
yet. #overlyhonestmethods
Will Morgan@willtmorgan: The
instrument was inoperable during this period because somebody hit one of the
switches with their arse.#overlyhonestmethods
Dr. Twittenheimer@DrTwittenheimer: A modified protocol was implemented because a certain graduate student
seems unable to follow simple instructions.#overlyhonestmethods
Blas M.
Benito @BlasMBenito: We used method A
instead of method B because my cat pooped on method B #overlyhonestmethods
Georgina_Milne @Georgina_Milne: "The sample was biased
because the plants were growing in a thorn bush and I didn't want to stick my
hand in too far"#overlyhonestmethods
Adam J.
Mosa @MosaFacts: we studied the effects
of the rhinovirus on adult neurogenesis because I sneezed inside the sterile
hood #overlyhonestmethods
Jamie Quinn @jmquinn: No SES [socioeconomic status] data were available
for the sample, as the testers forgot to ask. #overlyhonestmethods
[And on a similar note...] @peds_id_doc: #overlyhonestmethods Side effects were limited 2 headache & nausea, because they were the only ones we asked about.
overlyhonestmethods
Allie Kosela
Louise Saul @AikiSaul: incubation times vary
depending on how long a coffee/lunch break is needed #overlyhonestmethods
David
Clarke @Mozarticus: the crystal was grown
via slow evaporation of dmso.... or i couldnt be bothered cleaning the nmr tube
for 3 mths #overlyhonestmethods
Adam
Shellard @AdamShellard: Statistical analysis was
carried out on excel #overlyhonestmethods [included here because this basically reflects laziness].
Michelle
Turco @chelleomi: Crystals formed after 5
months... because I forgot to check between month 1 & 5 #overlyhonestmethods
Jamie Quinn @jmquinn: we don't have the code
used for the analyses because one of us is too good to use code and instead
uses dropdown menus#overlyhonestmethods
Lija Greenseid @Lija27: The interviews were
transcribed verbatim, except for when I babbled about things that were totally
off topic #overlyhonestmethods
Alex @voting: Standard room pressure
was assumed because nobody could be bothered to fight with the barometer. #overlyhonestmethods
Mariza de
Souza @marizageek: Further investigation
could have been carried out if I hadn't knocked the apparatus over and
misaligned it #overlyhonestmethods
Gregor
Miller @radianity: The experiment was stopped
after the postdoc collapsed in tears from the realisation that he'll never
become faculty#overlyhonestmethods
Alex @voting: This research topic was
chosen because I could essentially just rehash various pieces of work from the
past few years.#overlyhonestmethods
Kane_WMC_Lab @Kane_WMC_Lab: We would have reported
stimulus size in terms of visual angle, rather than cm, but we always forget
how to do that!#overlyhonestmethods
Alix Conway @conwizzle42: Sites were randomly selected by how far I felt like walking that day#overlyhonestmethods
Sasha Feoktistov @thisisdemocracy: "we adapted prev. est. protocol for our novel experiments" we had no clue what we were doing so we copied smarter ppl#overlyhonestmethods
Helen Lee Lin, Ph.D. @helenleelin: #overlyhonestmethods We only cite papers that are available as full-text online because going to the library and Xeroxing is hard.
@sethzenz: Additional
variables were not considered because everyone involved is tired of working on
this paper.#overlyhonestmethods
Donaldo D.
Canales @psycho_ddc: Mostly citing
meta-analyses in this manuscript because I don't have time to read 500+
papers #overlyhonestmethods
Teri
Anne Stanley @TeriAnneStanley: We used Female Rats
because the technician checked the wrong box on the order form #overlyhonestmethods
Amrita
Neelakantan @amster04: rodent traps laid only on
the ground as too many volunteers fell off the trees while trying to set them
up in branches #overlyhonestmethods
Cool Science Videos @CoolScienceVids: This method was essentially described by Jones et al (1964), but we didn't notice it and so are publishing it anyway#overlyhonestmethods
A.C. Papageorgiou
Arnaud Garcon
Marc
Maxson @marcmaxson: we didn't test as many
clams as oysters because we're pretty sure someone found the samples and ate
them #overlyhonestmethods
JörgB @BldyJck: We tried to measure
only axons.. but honestly, it's f**king hard to distinguish them from dendrites
with our microscope#OverlyhonestMethods [I've wondered about this sort of thing].
Xylotheca @Xylotheca: Measurements were
accurate to 6 places... + or - error caused by vibration due to people walking
past. #overlyhonestmethods [A perfectly controlled environment, sadly, is almost never possible. I work with human subjects and would love to be able to make measurements accurate to 6 places plus or minus anything].
Bernadette
Wright @WrightEval: The site visit
locations were chosen because we thought they would make fun vacation spots #overlyhonestmethods
[And of course....]: NCSU Linguistics Lab
Kel Hinkle
James
Grecian @JamesGrecian: species datasets were
really selected because they were all I could get hold of 1 month before my PhD
thesis deadline#overlyhonestmethods
Elise
Steven
Graves @revwhitewolf: RT @paulcoxon The spectrometer was made by a
company which no longer exists, in a country which also, no longer exists#overlyhonestmethods
Viet Le
Miguel_delF @yo_Antitwitter: We didn't read half (in
real any) of papers we cite because to print them is to expensive #overlyhonestmethods [To better understand the costs, scientists can cite anywhere from dozens to over a hundred papers, ranging from 2 or 3 to 40 or 50 pages depending on the type of paper and the discipline].
Anne Osterrieder@AnneOsterrieder: Buffers were prepared with the stock solutions inherited from the PhD
student who left the lab a couple of years ago.#overlyhonestmethods
thecellularscale@cellularscale: We used software X because it is free.#overlyhonestmethods
Erin
Kaufman @ELKaufman1: Sample was biased b/c
patients were at 3 diff hospitals, gas prices were over $4 and I wasn't getting
reimbursed.#overlyhonestmethods
Allison D. Gamblin
Ana B
@devillesylvain Sylvain Deville: we didn't read half of the papers we cite because they are behind a paywall #overlyhonestmethods #OA
Dr.Angela Pfammatter
James Baker
GK Lefkowitz
Piv Gopalasingam
2b) Because of the expense of supplies, scientists often borrow them from each other--and have to take on faith that these supplies really are what they're supposed to be.
julezyme
dr leigh
Chris Gallop
3) Scientists use equipment or programming code that breaks or otherwise doesn't function as intended--generally at the most inconvenient times. The vicissitudes of weather and dangerous animals fill the same function for field/nature researchers.
Steven
Graves @revwhitewolf: RT @Camaceron1 pH remained constant
throughout entire experiment (pH meter ran out of battery half way through)#overlyhonestmethods
Julian Rubino
Maria Sundaram
Angela
Amanda
Warr @Mandabirdstweet: Bird counts were not
carried out on this day because there was a lion at the site and I didn't want
to die. #overlyhonestmethods
Donaldo D. Canales
Sam
Urmy @ElOceanografo: "Data were not
collected during these 2 weeks because a gopher chewed into the buried power cable
and exploded."#overlyhonestmethods [That one has to be true, because you just can't make up those details. Also, "The Exploding Gophers" might make a good band name].
4) Science involves limited numbers of subjects, which are often uncooperative. There's good reasons for the much-maligned "convenience samples."
Madhusudan Katti
Jon Brock @DrBrocktagon: Participants were 19 undergraduate students aged 18 to 23 plus my mum, who's 57 #overlyhonestmethods [Apparently undergrads do actually run their mom in studies, and professors seem to think it's a good thing. So long as it's just undergrads, no problem].
a small
bear @hoorayabear: There were 14 participants
in this study because that is how many people I could convince to take
part. #overlyhonestmethods
LNT @LNT89: Mice were treated with
200uL subcutaneously. Apart from that wriggly one, not sure how much he got...
100uL? #overlyhonestmethods
Barbara
Bell @baulabarb: we modified the
marking/identification method for use with snapping turtles because we wanted
to keep our fingers.#overlyhonestmethods
Dr Caz @fruitflyCaz: We're not going to
mention how many marshmallows were used to bribe the monkeys to do our
bidding #overlyhonestmethods
Lucy Wyatt @LinkaNeo: Research organisms
weren't reproducing as fast as I need them to so I tried playing Barry White
while in lab. #overlyhonestmethods
Barbara Bell
Brett Schaerer
Jon
Brock @DrBrocktagon: Four participants completed
5000 trials each. Coincidentally, their initials are the same as those of the authors#overlyhonestmethods [In some fields, such as vision science, it's actually accepted for study authors to be subjects, on the (questionable) assumption that everyone sees pretty much the same way. However, they're generally not the only participants].
eastcoastkids
Gregor
Miller @radianity: Participants were selected
based on their level of desperation for free chocolate bars #overlyhonestmethods
Robyn
Powell @DontDoDinosaurs: Participants selected from
poor university students who will do anything we ask for ten bucks #overlyhonestmethods
Jessica @BennyandMaia: Test subjects were freshman
males. Only volunteered after I dressed slutty. Nobody volunteers to smell
shit. #overlyhonestmethods
Ian Beatty
Angela
Anne Cregan
NCSU Linguistics Lab
Yani Malai
Abdullah @YaniMalai: In this study we had a
LARGE number of ASD participants (n=10); matched on the enthusiasm of their
parents.#overlyhonestmethods [This is sort of two-in-one as for most types of studies, ten participants is not a large number].
Dr. Nyk @DrKushalNaik: #overlyhonestmethods All
patients gave consent because we removed the 'I do not give consent.' checkbox
from the informed consent document. [This one crosses the line into an outright misdeed, I think].
5) Science hasn't stopped relying on serendipity since penicillin was discovered in the mold growing on a scientist's old sandwich left lying around the lab. Good scientists take advantage of this.
[This says it all]: Georgina
Jessica
Wright @Jessrw_NYC: We discovered binding
to a heat shock protein because we overgrew our culture and did the experiment
anyway. #overlyhonestmethods#truestory
NatC @SciTriGrrl: We used the reagent in
Experiment 1 due to a lazy mistake by an undergrad. But hey! Serendipity
works! @dr_leigh#overlyhonestmethods
Marcus G @NoMstn: we accidently
discovered figure 7 and then filled in figures 1-6 to that it would all make
sense in the paper #overlyhonestmethods
Keated @Keated: #overlyhonestmethods This
interesting result turned up in my last ever sweep, so I have no idea whether
it's a fluke or not
Mason A. Rouser
Tal Yarkoni @talyarkoni: we discovered the anxiogenic properties of our new drug accidentally, while trying to fuck with labmate Steve's coffee#overlyhonestmethods
6) Scientists love clever wordings, puns, fun journal titles, cool noises, pretty graphs, and cute animals as much as the rest of us. Humor and aesthetics are an important, if undervalued, motivator in science.
Claire
Gomer @ClaireGomer: I chose the study species
due to their complex social interactions, and because they're dead cute. #overlyhonestmethods
Rebecca
Gill @DrRebeccaGill: Wrote this paper on
the effect of petitioner gender on success just to see the variable
"petsex" in print. #overlyhonestmethods
isa @isaontheway: Like how I chose the
title of my thesis because I wanted every word to start with a
"C".
Macrophagic
Cheng H.
Lee @chenghlee: MT @drugmonkeyblog 2 days to isolate the
protein, 5 weeks to generate the hilarious double-entendre name for the gene.#overlyhonestmethods
Colleen @straightedgersx: When observing baby
chickens (chicks) behavior we named them Chicken Little, Chicken Pot Pie &
Chicken Ala King#overlyhonestmethods
Nash Turley
ootastic
Pasquale @mansquatchie: I chose to use the
particular solvent because it smelled better than the others. #overlyhonestmethods
Nils Hoppe @nilshoppe: "Team
brainstormed excellent project acronym and then built grant proposal around
it" #overlyhonestmethods
Rachel
Beattie @RachelLBeattie: We chose to study
preschoolers because that gave us a really good excuse to buy lots of stickers #overlyhonestmethods
7) Scientists often don't understand statistics very well, and use procedures that aren't appropriate for the data being analyzed.
Cheng H. Lee
Nicolas
Ward @UltraNurd: We fit a normal
distribution even though we know the data is multimodal, because it's easier to
calculate in Excel.#overlyhonestmethods
Marga
Corameta @MargaCorameta: We applied this statistical
analysis method because is the only one we know how to use in SPSS #overlyhonestmethods [For those unfamiliar with the program, SPSS is a program involving no coding whatsoever, just drop-down menus...with a confusing array of different procedures you can use, whose names and attribute don't necessarily perfectly match their equivalents in the stats textbooks. However, this is actually one of the easiest methods you can use to analyze data, so this confession is pretty sad].
Blas M.
Benito @BlasMBenito: Our dataset was
non-normal, heterocedastic, and almost stochastic, but what the hell, we used
linear regression#overlyhonestmethods
Scott
Paquette @scottpaq: A qualitative data approach
was chosen since our graduate stats course has permanently scared us away from
numbers#overlyhonestmethods
Jessica
Bregant @JessieBregant: Tests were chosen after
thorough analytical review. We ran everything we could make SPSS do, then reviewed
for p<.05.#overlyhonestmethods
miss_sapphyre @miss_sapphyre: This statistical method was chosen because it's the only one my MD coauthors understand. #overlyhonestmethods
8) Scientists aren't always honest. They'll bend the procedure or, particularly, the stats, to make the results look as good as possible.
Dr Krystal
Nik Papageorgiou @upmicblog: n=3 for positive data. The other n=16 that didn't work were "experimental error". #overlyhonestmethods
Dr Caz @fruitflyCaz: This methodology is full of
jargon so you won't work out we just tweaked things until they worked #OverlyHonestMethods
Adam
Watts @fireecologist: Four outliers were excluded
from analysis, because the third author thought they "messed with our
R-squared." #overlyhonestmethods [Outliers are people whose results were either way higher or way lower than the rest of the group. The third author is complaining that the strength of a correlational analysis has been reduced by outliers].
Shinobi @shinobi42: We removed some of this data from our sample because our model looks better without it. #overlyhonestmethods
Nikki
Botting @NicolaBotting: A substantial number of
participants were then excluded on relatively random grounds so that results
match predictions #overlyhonestmethods
Adrian O'Hara
Iain
Gilmour @Iain_Gilmour: Once got advice to
plot data that correlated as large solid squares, non-correlated data as faint
open circles. #overlyhonestmethods
Nial Wheate @NialWheate: Only a section of the
NMR spectrum is shown, because there was evidence of small impurities in the
other sections #overlyhonestmethods
9) Scientists follow conventions, too. They claim that this allows them to compare their results with others. True, but I think it also reflects that scientists are subject to the same social pressures as the rest of us.
dr leigh@dr_leigh: we used [program] because doesn’t everyone else? #overlyhonestmethods
Emily
Randy
McLain @randoramma: Used [method] as that's
what we've always done, the only way we know to do it, and God help us if we
change anything!#overlyhonestmethods
Michelle
Dawson @autismcrisis: We know this autism
severity measure doesn't work, but a bunch of other people used it so we will
too #overlyhonestmethods but
true story
Jon Brock
10) Reviewers can be more annoying than helpful, and scientists invest a good deal of time and energy in placating them and stroking their egos.
Gregor
Miller @radianity: Half of the article's
literature review cites irrelevant research because one of those authors might
review the paper#overlyhonestmethods
Are you serious
AprilPawluk
Jon
Brock @DrBrocktagon: Further analyses were
performed to appease Reviewer 3 #overlyhonestmethods
Kane_WMC_Lab @Kane_WMC_Lab: We thank reviewer 3 for
suggesting this alternative analysis, but we really don't mean it. #overlyhonestmethods
Pepijn Vemer @PepijnVemer: I added this reference during revision, because I believe one of the authors is a reviewer. #overlyhonestmethods
Dr Lyndon Walker @DrLyndonWalker: My references seem out of date because this is the 10th journal I have submitted the article to #overlyhonestmethods
Scott
Paquette @scottpaq: The motivation for
this paper was to increase our citation count, pre-tenure. #overlyhonestmethods
Joe Coules @Joe_Coules: We was going to run a
replication but nobody funds or publishes replications. So we did this
instead #OverlyhonestMethods
Rob Ford
Patricia
lockwood @thepsychologist: We changed the title to an
undescriptive pun as it was more likely to get published in a high impact
journal #overlyhonestmethods
biochem belle@biochembelle: We decided to use Technique Y because it's new and sexy, plus hot and cool. And because we could. #overlyhonestmethods
Dr. Paul
Blas M. Benito
Nikki Botting
Michelle Turco
Crommunist @Crommunist: This test was chosen because my supervisor is trying to pioneer it as part of her tenure bid #OverlyHonestMethods
Jon Brock
Felicity
Callard @felicitycallard: Methods were
creatively embellishd by PI, who said that if we accurately reported what we'd
done, wd never be published #overlyhonestmethods
overlyhonestmethods @overlyhnstmthds: submitted to a new
journal b/c editors wanted more experiments and the only one who can do it was
an undergrad who left #overlyhonestmethods
Régis
Alenda @regis_alenda: My 16-years old son
hadn't actualy done much for this paper, but he is listed as an author so that
it'll make the news.#overlyhonestmethods
Paul Bretherton
Neuroskeptic @Neuro_Skeptic: The first author
didn't write this Methods section and doesn't understand half of it. #overlyhonestmethods
Dr Caz @fruitflyCaz: Our first author didn't
conduct this research so this methodology isn't actually accurate #OverlyHonestMethods
Julie Scott
Jen @JenDVM2012: The first author had
no part in data collection since he can't be in the lab without
supervision #overlyhonestmethods
Neuroskeptic @Neuro_Skeptic: The first author didn't
write this Methods section and doesn't understand half of it. #overlyhonestmethods
12) Part of science is not knowing what results you're going to get before you start, so you often get surprising results or ones that don't make sense. You usually have to rethink your hypotheses, redo parts of your procedure, or otherwise improvise. In my opinion, this flexibility is a sign of good science, but for some reason, journals want to publish articles that suggest the results and explanation were perfectly planned all along. This creates a misleading picture of how science works--hence, the existence of an #overlyhonestmethods tag.
dr leigh @dr_leigh: we did experiment 2
because we didn't know what the fuck to make of experiment 1 #overlyhonestmethods [I bet this happens sometimes in every lab].
[This point is actually quite profound.] Christina
Palka @Luckiturtle: successful experiments are
actually failed experiements combined with a new hypothesis #overlyhonestmethods.
[And on the same lines]: Duncan Hill @drduncanhill: #overlyhonestmethods a common route to new avenues of research is getting an experiment wrong, seeing something interesting and studying it
[And on the same lines]: Duncan Hill
Dr Caz @fruitflyCaz: This isn't the study we
intended on doing but the results supported this angle better #OverlyHonestMethods
Michael
Hale @Hey_MickeyH: The methods were
described in the 3rd person passive voice to create an illusion that everything
went according to plan#overlyhonestmethods
Pavito @pdodonov: We performed these fancy
analyses because our original results (and hypotheses) just weren't interesting
enough.#OverlyHonestMethods
@ProfLikeSubst Proflikesubstance: Though it all sounds tight now, we had no sweet fucking clue what we were doing when this started#overlyhonestmethods
Stuart R Jackson
[Here's the cynical way of looking at this situation]: Skeptical @RC_Skeptic: Our Introduction is pure post-hoc bullshit, but it tells a nice story, doesn't it? #overlyhonestmethods
A.C. Papageorgiou @ACPapageorgiou: The "aim of the
study" was defined after writting the "discussion" part of this
manuscript... #overlyhonestmethods
One blogger has argued that the errors discussed here result from perverse incentives in science. They don't help, for sure, and probably contribute to #7-11. But even with the best incentives, human error and laziness, technological failure, limited resources, and the sheer serendipity of science will still exist. #Overlyhonestmethods will always be with us.A.C. Papageorgiou
nrayko @nrayko: The literature review was
written after all the research was completed and analyzed so the results would
make sense.#overlyhonestmethods [Doesn't everyone do this? No, seriously, don't they? You always end up adding new literature you would never have thought to include after finding results anyway, so why not just write the whole literature review after running the experiment?].
It has been some time since I visited website with such high quality information. sigma antibody
ReplyDelete