When Fish Teach us MRI-Stats

11/02/2017 New in our Guess the publication  students’ section. Which study is illustrated and why?

Solution: This week’s Guess the Publication was inspired by the 2009 poster and later publication of Bennett and colleagues ¹ . Their research team conducted an fMRI study using a dead Atlantic Salmon as the main research subject. No, we did not make this up. In fact, many everyday objects look quite beautiful when scanned using an MRI machine. But this story with the salmon was about more than admiring the amazing inner works of fish visualized by MRI machines. But what? Why would a group of skilled scientists invest time into scanning a dead fish? They would do so, because the real purpose of this experiment was to show that neural activation obtained by functional MRI can be spurious unless the appropriate correction methods are applied!

The problem of coincidental findings in neuroscience can result from a problem called multiple comparison problem. When analysing neuroimaging data using a whole-brain analysis approach we run a test for every voxel within this whole-brain mask. This means for every single one of tens of thousands of voxels a test is computed. As a result of running that many tests, the likelihood of getting a false positive finding increases drastically. The problem of multiple comparison can be visualized by the example of someone throwing a dice over and over again (see this youtube video for an easy description of the topic), until you receive the number you hope for. The more times you throw the dice, the more likely you get the number right. Or, in fMRI analysis, with every statistical test run in addition to the previous one (and we said it’s tens of thousands of tests), the likelihood of finding spurious activation increases. With every additional tested voxel in the brain. This means nothing else as that the likelihood of a certain result increases with the number of tries. There are different correction methods in neuroscience to overcome this problem. These methods for example include family-wise error correction (FWE) and the false detection rate (FDR). Both represent an expectancy, that part of the positive discoveries will be false positives. There are also new techniques available, including threshold free cluster enhancement techniques (TFCE), that might offer sensitivity while taking voxel- and cluster-based information into account. But how exactly each one of them work is another story.

To go back to our study – what does the “salmon-experiment” have to do with the topic of multiple comparisons? Bennett and colleagues showed, that at a threshold of p < 0.001 with no correction for multiple comparison it is possible to obtain neuronal brain activation, even in a dead salmon. These findings disappeared upon employing FWE or FDR corrections.

Take home message: Without proper correction methods you may be looking at nothing more than a red herring (or hot fishy air). Scientists spend so much time and effort on acquiring quality data in order to venture further than anyone before. How we handle that data afterwards is a big responsibility, especially, because of the chance of misinterpretation. And also, it is o.k. to think outside of the box – it could go a long way.

Fun fact: this study did win an Ig Nobel award. The Ig Nobel award is dedicated to research studies that first make you laugh, then think. The awards honours those thinking outside the box and we can only recommend to read through the awards page and recipient’s work. You can also read more about the dead salmon study through an entertaining blog post that appeared in Scientific American. If you do, you will understand that it is not a coincidence that there was a pumpkin in the trash bin of our drawing representing this study.

Original publication:

[1] Bennett, C. M., Baird, A. A., Miller, M. B., & Wolford, G. L. (2011). Neural correlates of interspecies perspective taking in the post-mortem atlantic salmon: an argument for proper multiple comparisons correction. Journal of Serendipitous and Unexpected Results, 1, 1-5.