Comment on “Maintaining Scientific Integrity in a Climate of Perverse Incentives and Hypercompetition”

I just read this worrying paper summarizing a big problem we should all be aware: “Maintaining Scientific Integrity in a Climate of Perverse Incentives and Hypercompetition“.

I don’t have a perfect solution to change the system for good, but I have an easy patch to help your integrity and the integrity of your group. And I say this because I am very conscious that I am (we all are) weak and when under pressure, the easier person to fool is yourself. This means, that even if you don’t want to cheat consciously, behaviors like p-hacking, ad hoc interpretations and not double checking results that fit your expectations are hard to avoid if you are on your own. So this is the patch: Don’t do things alone. You can fool yourself, but it’s harder to fool your team-mates. And as a corollary, don’t let your students do field work, data cleaning, analysis, etc… alone. Somedays I may be tired and tempted to be more sloppy during fieldwork, for example, but if I have a team-mate with me, it’s easier overcome the situation as a team. In our lab, one way to do this is using git collaboratively. Git tracks all steps of your research since data entry. The first thing we do when we have raw data is to upload it to git and check it ideally at least among 2 of us. This creates a permanent record and avoids the temptation of editing anything there if results are not what you expected later on. Same with data cleaning, and analysis. When those steps are shared and your actions are tracked, it’s easier to be honest. Just to be clear, this mechanism doesn’t work as the threat of a “big brother” that is watching you, is more a feeling of teamwork, where you want to live for the team expectations.

Advertisements

Marie Curie, concessions, and pressure to publish.

I have to admit I didn’t know much about Marie Curie a few days ago (other than the “trivia” facts such as that she discovered the radioactivity and was the first women winner of a Nobel prize). But I just read a book* about her and I really loved it. Oh my god, she was unique in a thousand ways. The book is written by Rosa Montero, and uses Marie Curie’s diary written after Pierre Curie death to talk about very personal things including death, gender balance, society pressures, self-esteem, and many other main topics in life. So it’s not a typical biography, but an excuse to reflect on important things. I won’t go into details, but I highly recommend it.

And while reading the book I found a quote by Pierre Curie that reflects at perfection my actual feeling in science.

“Besides, we must make a living, and this forces us to become a wheel in the machine. The most painful are the concessions we are forced to make to the prejudices of the society in which we live. We must make more or fewer compromises according as we feel ourselves feebler or stronger. If one does not make enough concessions he is crushed; if he makes too many he is ignoble and despises himself”

I do think finding this balance is what kept you (and your science) alive in this world.

Which brings us to the last point. I just discussed a result with my PhD student. It is not significant (p = 0.08), but the effect size is quite big (probability something happening goes from 0.6 to 0.2), but the sample size is small (n < 20). The unavoidable question raised. “It’s 0.08 marginally significant?”, “can we say there is an effect?” My reply was that in a perfect world we would use this data to frame a hypothesis. Then, we would collect 30 more independent data points and test it for real. But the project is almost over, he needs to defend the PhD soon and we are not in a perfect world. So we make concessions. And we will try to publish what we have and cross our fingers hoping that someone else will validate our finding. But we don’t concede too much either, and we should make sure to discuss the result appropriately. A potential large effect size, but very variable and based on a limited sampling size. Or in other words, we will try to avoid the p-value dichotomy once more.

*The book is edited only in Spanish, French, Dutch, and Portuguese… for once, sorry English speakers!

Happy women and girls in science day!

Today we celebrate an important day. We celebrate equality in science! Hence, I want to make a post highlighting a few great researchers I have the privilege to work with. I was lucky enough to interact with lots of great female scientists and my stereotype of a scientist’s is not an old white man. I know this is not common, and this is why it’s important to show that there are plenty of awesome female researchers like the ones I met, specially to ensure young girls have a diversity of role models.

So here there are four great researchers in different career stages. First, my PhD Advisor, Montse Vilà. She investigates the impacts of invasive plants and was my first contact with a real scientist. Second, my PostDoc advisor, Rachael Winfree, from whom I learnt a million things. She investigates how bee diversity determines ecosystem functioning. Third, Romina Rader, who studies non bee pollinators. We met as postdocs and has become one of my usual collaborators. Finally, Ainhoa Magrach, which I hired as Postdoc last year and it was super-stimulating to work with. She is now studying the impact of global change on biodiversity. I could name many more because almost half of my coauthors are great female scientists, but I’ll stop here. Today several initiatives are highlighting the awesome work that women do in ecology, for example here (spanish) or here, so check it out and spread the word, specially among young girls and schools!

 

Bugs, kids and swimming pools

I accidentally found a great way to make kids into natural history/entomology during summer holidays.
All you need is a catch-net, Chinery’s Insect of European insects book (or any other general insect book for your region) and a swimming pool.
Kids catch drowned insects in the pool with the net (quite frequent drownings in our case*, I have to say, and I wonder how high are the death rates caused by swimming pools globally!). Collecting that way is already quite fun. Most bugs are still alive and we let them dry gently. It’s quite rewarding to see them fly away after a couple of minutes, which is wonderful because kids are now bug-rescuers, not bug-killers! Additionally, wet bugs are slow and let you see them and try to id them (i.e. at least to family or genus). So this is a kill free, easy way to see bugs
If you want the scientific bonus, we also recorded the frequency of each species, time of the day and if it was dead of alive. You can then explore the data. We found 52 specimens of 22 morphospecies. Of the several insect groups recorded (wasps, bees, ants, flies, beetles, moths, harvestmen (opiliones), heteropterans, milipids, …), the most numerous were wasps. Interestingly, among the two most common species, Polistes has higher mortality rates (4 out of 8 ;50%) than Vespula (3 out of 9; 33%). The coolest insect was probably the stick bug!
*If not much insects fall in the pool, open the filter and you should find dozens of mostly dead bodies.

P-hacking and Paul Feyerabend

P-hacking, or researcher degrees of freedom, it’s a worrying issue in science. Specially, because p-hacking is not a black and white issue. On the blackest side there is deliberate p-hacking with the only purpose to advance your career. This is bad, but I hope it’s rare*. The grey area is more intriguing, because it concerns researchers not doing it consciously. I used to think that this include researchers that never had a proper statistical training, with too much pressure to publish too small datasets or that fool themselves thinking that this new analysis/subset of the data is what he/she should be testing in first place, so it doesn’t matter really the 200 previous analysis/subsets (which is false, they matter!). This is equally bad for science (even if the motivation is not as bad).

But then I read “against method” of Paul Feyerabend**. Despite some passages are really slow and repetitive, I liked it. A big part of the book explains Galileo Gallilei story. Galileo changed the paradigm based in incomplete theory, iffy data and measurement tools, and lots of propaganda. He used more its intuition than a proper scientific method. He was still right and most of his ideas were confirmed years later.

And that rang a bell. I’ve heard before scientists saying things like “well, we can’t measure it accurately, but trust me I know the system and this is what is happening”. From here to do a bit of conscious or unconscious p-hacking to support your hypothesis there is a small step. This researchers are using intuition, hours of thought and lots of knowledge. This scientists are putting forward their ideas. Ideas in which they believe, but they can’t just prove unequivocally with the data at hand because of the complexity of the problem.

Paul Feyerabend said that “everything goes” if it advances science. I am not justifying p-hacking to support something that it’s hard to  prove but you think is true, but after reading Feyerabend I am also less worried about adding some subjectivity to the scientific method, because being completely objective and following the method strictly may also slow down science. Maybe the middle ground is being able to recognizing when something is an opinion, and not facts, and avoid sticking a p-value to this opinion, but defend it anyway in the light of the data available and try to push forward the agenda to get better data, better methods, or whatever you need to support it. It’s complicated.


*people that only want to advance their careers choosed politics in first place, not science, right?  I know this is probably a wrong assumption.

**In a nutshell he praises that an objective scientific method is unattainable and rarely applied, and that we should free ourselves from using it as the single tool to do science. I liked for example the idea of aiming to create a plethora of theories (with no historical constraints or resistance from the status quo to accept compatible alternative explanations) that can cohabit and let time to do the thinning a posteriori. More on wikipedia.

Preferring a preference index II: null models

This is a guest post by my PhD student Miguel Ángel Collado. My last post on preferring a preference indexes was not satisfactory to us, so we have better options now. Read Miguel Ángel solution below.


We are working on the ecological value of various habitats or sites. In addition to different classical biodiversity indexes, we want to know if we have some sites that are not specially diverse, but they have some ecologically important species attached to them, we could measure this through preference analyses, using null models to compare with our data.

We can define “preference” for an species if the presence of this species on a given site is bigger than expected by random. A way to know this is comparing to null models and establishing an upper threshold for preference, and a lower one for avoidance, this way we would know whether some species of interest have affinity for some sites or just use them as expected.

To see an example of this