The cost and complexity of assessing impact
The idea was a simple one. Using EICAT impact assessments of alien amphibians, could we get the costs of the studies and see whether more expensive studies resulted in higher impact assessments? Or, perhaps more worryingly, were cheaper studies resulting in lower impacts of invasive species?
But this paper turned out to be a huge undertaking.
First, my dedicated team of MeaseyLab amphibian fundis who were kicking their heals after the CIB ARM in November 2019, committed to scoring all unscored amphibians since we did the same exercise in 2015 (seehereif you aren’t familiar with that). Initially, we thought that this would be a short order, but we discovered that all those invasive amphibian people out there in the world had been working overtime. Specifically, they had generated about the same number of papers that we’d scored in 2015, but in only 4.5 years!
We drank a lot of coffee and all sat around together in my office to answer any potential queries regarding EICAT scoring of amphibians.
While James and Nitya certainly made the most noise, it turned out that Carla quietly did most of the assessments.
After that first exercise, we started begging. We approached everyone who had published in the last 4 years with a paper that had resulted in one of our EICAT scores and we gave them a questionnaire to work out how much money they had spent on the project that had produced that paper.
The questionnaire was approved by Stellenbosch University ethic’s committee, and was given the green light by our economics Prof: Elizabeth Pienaar.
Sharp eyed blog readers will recognise Elizabeth from the July 2019 postgraduate field trip where she taught her module on economics and invasive species (see here). Sadly, Elizabeth couldn’t join us in person, so we simply sent the data from all of our (very kind) contributors who had taken the time to produce costs for us. Their anonymity was assured. You know who you are, and we are very grateful for your help. We know that it was a big ask.
Once Elizabeth had converted all of those different currencies into some dollar price parity, we could run our models and see whether the price of the study predicted the level of EICAT impact for the amphibians.
Here comes the big reveal:
Yes - it did!
This is actually quite shocking, as what it means is that if you don’t have a lot of cash you are unlikely to find that your invasive species has a high impact. It follows that if your sitting in a part of the world where you don’t have much cash, then it might be extra difficult for you to make your case that your invasive species is actually a big problem.
Next, Nitya had the idea of looking at the complexity of these studies. Would studies that were more complex result in higher impact scores? He based the complexity on a new study by Christie et al (2019), who found that more complex designs resulted in more robust results. Nitya beavered away for several weeks scoring all of the papers. Finally, when they were all done, we were able to run our models.
And we found that - yes again! More complex study designs were indeed likely to result in higher impact scores.
This result goes hand in hand with our economic result above, as more complex study designs often cost more.
So please do go and read all about it:
Measey J, Wagener C, Mohanty NP, Baxter-Gilbert J, Pienaar EF (2020) The cost and complexity of assessing impact. NeoBiota 62: 279-299.https://doi.org/10.3897/neobiota.62.52261
