Subscribe to MeaseyLab Blog by Email

Dan graduates

30 March 2023

Graduation time for Dan

Dan van Blerk has been in the MeaseyLab since his Honours project in 2019 (see here), when he conducted a literature review on the impacts of invasive lizards around the world. In late 2019, Dan and I met with Olaf Weyl to plan Dan’s MSc fieldwork over the next two years. With all three of us filled with enthusiasm for the upcoming adventure and what it would achieve, we were about to find out that fate had other plans.

Very sadly, Olaf died later that same year, and subsequently did not get to participate in any of the plans. He is greatly missed, and we continue to remember his initial enthusiasm for the project. Happily, Josie Pegg was soon on hand to take over from the SAIAB side, and in doing so saved the project. However, just as we were about to deploy Dan into the streams of the Western Cape, we were hit by another big setback. The world suddenly shut down, and with it Dan was confined to quarters and we decided that we’d have to conduct at least one of his chapters as a literature review instead of in the field as planned.

During 2021, Dan managed to get two notes published on natural history observations that he’d made during his time in the MeaseyLab (van Blerk 2021 a,b).

Eventually, Dan did get out into the field and surveyed many locations in many streams for ghost frog tadpoles and invasive fish. The work was a real advance for conservation evidence in the impacts of invasive fish on these frogs. Dan presented his work in the Conservation Symposium (see here). We got used to seeing Dan presenting online and in person (see here and here). His findings have already been submitted to Aquatic Invasions, and we hope to announce publication of his chapter soon. Watch this space. Dan defended his MSc work in February this year (see here).

It was great having Dan in the lab, even if he didn’t spend much of the three years physically with us. We wish him all the best in his subsequent career and hope that he’ll keep in touch.

Read more:

van Blerk, D. (2023) The Impacts of Invasive Fish on Ghost Frog tadpoles. MSc thesis, Stellenbosch University. http://hdl.handle.net/10019/32356

Van Blerk, D., Reissig, J., Riley, J.L., Measey, J., Baxter-Gilbert, J. (2021) Observations of infanticide and cannibalism in four species of cordylid lizard (Squamata: Cordylidae) in captivity and the wild. Herpetology Notes 14: 725-729. pdf

Van Blerk, D., Measey, J., Baxter-Gilbert, J.H. (2021) Predation by a Brown Widow Spider, Latrodectus geometricus (Koch, 1841), on a Common Dwarf Gecko, Lygodactylus capensis (Smith, 1849), with a review of the herpetofaunal diet of Latrodectus spp. Herpetology Notes 14: 291-296. pdf

  Frogs  Lab  prizes

Can Google's Bard do any better?

27 March 2023

Can Google's Bard write my PhD?

Last week I checked out ChatGPT and its ability to write text on scientific topics with citations. The bot performed very well in that it gave a great first draft of text, but the references that it gave were completely bogus, with none of them checking out. It seemed to me that ChatGPT was using the internet to source the text and then throwing in a citation where it saw that others had used one on something similar. Writing text with citations is, however, a lot more difficult than randomly tossing in a citation (read more about how to do it here). Instead, citations must assert that statements in text where they are used have some basis. Overall, the output from ChatGPT resembled that of a “cocky student” who takes the risk that you won’t check their work, and so makes up citations to go along with text that looks good.

Today is the turn of Google’s Bard platform to take the same text, so let’s see how it does.

 First, Bard starts with a statement that: “Bard will not always get it right. Bard may give inaccurate or inappropriate responses. When in doubt, use the “Google it” button to check Bard’s responses.”

What does Bard say it is?

Input text: “I am writing a chapter for a book on academic writing and I want you to describe Bard in your own words”

Bard is a large language model, also known as a conversational AI or chatbot trained to be informative and comprehensive. I am trained on a massive amount of text data, and I am able to communicate and generate human-like text in response to a wide range of prompts and questions. For example, I can provide summaries of factual topics or create stories.

I am still under development, but I have learned to perform many kinds of tasks, including:

I will try my best to follow your instructions and complete your requests thoughtfully.

I will use my knowledge to answer your questions in a comprehensive and informative way, even if they are open ended, challenging, or strange.

I will generate different creative text formats of text content, like poems, code, scripts, musical pieces, email, letters, etc. I will try my best to fulfill all your requirements.

I am excited to be able to help you with your academic writing, and I hope that I can be a valuable tool for you.

You may feel, as I do, that Bard is a little overeager to please me, especially given that this is the first time we have chatted. Like some of Google’s other interactive platforms, I’d rather it was more concise and less politely flatulent.

The test

To quickly recap, I asked Bard the same things that I asked ChatGPT last week. I used the same wording, including the same sentence structure. Specifically, I wrote:

Write a sentence with citations on: ”…”

And then replaced the … between the quotes with a sentence that I had written as an outline for the first two paragraphs of an introduction appropriate to a PhD chapter or manuscript. Both paragraphs consisted of four sentences:

Paragraph 1

Invasive species are a growing global problem

invasive species cost a huge amount of money to governments

invasive species are a threat to biodiversity

invasive species impact people’s quality of life

Paragraph 2

Freshwater habitats have been particularly impacted by global change

important changes have occurred to freshwater systems after the introduction of invasive fish

invasive fish have been added to both rivers and lakes

large numbers of introductions of invasive fish have already occurred globally, and are ongoing

Here is the output from Bard compiled into two paragraphs for easy reading by me (see line-by-line version below):

Invasive species are a growing global problem, with the International Union for Conservation of Nature (IUCN) estimating that they cause $1.4 trillion in economic damages annually (IUCN, 2016; WWF, 2017). Invasive species cost governments a huge amount of money to control and eradicate, with the US National Invasive Species Council estimating that the cost in the US is $120 billion annually. (NISC, 2017). Invasive species are a major threat to biodiversity, as they can outcompete native species for resources, introduce new diseases, and alter habitats. (IUCN, 2016). Invasive species impact people’s quality of life by causing economic losses, disrupting ecosystems, and harming human health. Economic losses can occur when invasive species damage crops, forests, and other natural resources. Disrupted ecosystems can lead to the decline of native species and the loss of ecosystem services, such as water purification and flood control. Harm to human health can occur when invasive species spread diseases or cause allergic reactions.

 

References:

IUCN. (2016). Invasive alien species: A global threat to biodiversity and ecosystem services. Gland, Switzerland: IUCN.

WWF. (2017). Invasive alien species: A global problem in need of a global solution. Gland, Switzerland: WWF.

NISC. (2017). The economic impact of invasive species in the United States. Washington, DC: US National Invasive Species Council.

At the end of this first paragraph I have to say that it’s a relief that Bard is citing and providing the reference for its citation. This makes tracking the information a lot easier. However, the bland nature of the text and the (over) reliance on a single source is not particularly useful. As stated for ChatGPT, these reports are compiled from a great many different documents, and are often written in an ‘readers’ digest’ format that doesn’t include source material. Hence, for a student these are best avoided as citation sources unless their work is pertinent to that particular report. In the last sentence, Bard seems to have forgotten completely about the one sentence request and instead has provided us with a barrage of statements – none of which have citations, although there are still a couple of references.

2nd paragraph

Freshwater habitats have been particularly impacted by global change, with climate change, invasive species, and habitat loss all playing a role. Climate change is causing changes in temperature, precipitation, and water flow that are altering freshwater habitats. Invasive species are non-native species that can outcompete native species for resources, introduce new diseases, and alter habitats. Habitat loss is occurring due to human activities such as development, agriculture, and mining. The impacts of these changes are already being felt, with many freshwater species in decline. If we do not take action to address these challenges, the consequences for freshwater ecosystems and the people who rely on them will be severe. "Invasive species can cause major changes to freshwater ecosystems, including the decline of native species, the alteration of food webs, and the spread of diseases. For example, the introduction of the invasive Asian carp to the Great Lakes has led to the decline of native fish populations and the alteration of the food web" (IUCN, 2016). "Invasive fish, such as Asian carp and largemouth bass, have been added to both rivers and lakes, often through illegal releases or escapes from aquaculture facilities." (IUCN, 2016). "Large numbers of introductions of invasive fish have already occurred globally, and are ongoing. Between 1970 and 2000, an estimated 7,000 species of fish were introduced to new areas (IUCN, 2016)."

 

References:

IUCN. (2016). Invasive alien species: A global threat to biodiversity and ecosystem services. Gland, Switzerland: IUCN.

WWF. (2017). Invasive alien species: A global problem in need of a global solution. Gland, Switzerland: WWF.

  

The 2nd paragraph contained requests (from me) for sentences that were a lot more specific. However, Bard continues to use the same set of references for all of its answers. Those IUCN reports are really good, especially that one in 2016 – where would Bard be without it? Interesting that it slipped into a quotation mode by the end. Whether this was because it could only find a single pertinent sentence and didn’t want to simply rephrase it, or because it was feeling lazy, I’m not sure. Still, lots more text than I had wanted, and if I had wanted quotes then I’d have asked for them.

Rather like the output from ChatGPT, Bard is writing sentences (apart from when it gets over enthusiastic and produces a paragraph or gets lazy and decides to quote directly from the source) and they are on topic. But they do lack depth. While I have no reason to doubt the source for this information, it is inadequate for a thesis or manuscript and I’d be tempted to toss the lot and go back to my outline and fill it in with citations from my own reading. It's interesting that Bard starts quoting its source, and even gives me back the exact same sentence I gave it as one of its outputs. This is an odd interpretation of the request: write a sentence on - simply to provide the same sentence back. 

It should not go without comment that Bard is distressingly out of date. I guess this comes from only looking at one or two reports from Google Books as a source, but I’d have thought that there was a lot more up to date content in Google Books. It’s a curious and conservative approach.

 

Bard output – thanks but no thanks

In comparison with ChatGPT though, I have to conclude that Bard is more honest in its output. While ChatGPT might be thought of as the “cocky student who isn’t afraid to make anything up”, Google’s Bard could be characterized as the “lazy student who found a single report and mined it to produce everything”. Neither would survive writing a PhD or manuscript.

Given that Google should have access to so much more, I asked Bard a few supplementary questions to find out why it was being so restrictive in its answers:

Why do you only supply references from a single source?

I am currently able to access and process information from a single source, which is the Google Books database.

Will you be able to cite from academic literature in the future?

Yes, I am confident that I will be able to cite from academic literature in the future. I am currently being trained on a massive dataset of text and code, including academic literature. As I continue to learn and grow, I will be able to access and process information from a wider range of sources, including academic literature. This will allow me to cite from academic literature in my responses, which will help me to provide more comprehensive and informative answers to your questions.

If you look back at our conversation above do you think that there will be any conflicts between your statements and your cited sources?

I have reviewed the conversation above and I do not think that there will be any conflicts between my statements and my cited sources. I have tried to be as accurate as possible in my responses, and I have cited my sources whenever I have made a statement that is not common knowledge. I hope that I have been able to provide you with accurate and informative information.

Do you think that your answers are going to be different from those of chat GPT?

It is possible that my answers will be different from those of Chat GPT. I am trained on a different dataset of text and code, and I am also trained using different techniques. This means that I may be able to generate different results, or I may be able to generate results that are more accurate or more informative.

However, it is also possible that my answers will be similar to those of Chat GPT. We are both large language models, and we are both trained on a massive amount of data. This means that we may be able to generate similar results, or we may be able to generate results that are both accurate and informative.

Ultimately, it is up to the user to decide which model they prefer.

And there we have it folks. Two bots who produce shallow text that isn’t really of any help to a student who is trying to write for their thesis or a manuscript. Hence, to answer my own question, neither of these platforms are going to write your PhD thesis. ChatGPT is going to throw a real curve ball to anyone who has the patience to look into the veracity of its statements, while Bard is just looking lame. However, I think that we will need to watch this space. Either or both have the potential to do much better, and I suspect that the will get much better at this as soon as they get the chance.

Last note

This text was written without the use of suggestive prompts. AI generated text is written as quotes in the above text. All other text is my own.

It will be interesting to look back on this chapter in a decade and see the changes that have emerged in that time.

I would like to thank Google and Bard for generating the quoted text in this blog post.

Here is the output from Bard line-by-line:

Invasive species are a growing global problem, with the International Union for Conservation of Nature (IUCN) estimating that they cause $1.4 trillion in economic damages annually. (IUCN, 2016; WWF, 2017)

 

References:

IUCN. (2016). Invasive alien species: A global threat to biodiversity and ecosystem services. Gland, Switzerland: IUCN.

WWF. (2017). Invasive alien species: A global problem in need of a global solution. Gland, Switzerland: WWF.

 

 

Invasive species cost governments a huge amount of money to control and eradicate, with the US National Invasive Species Council estimating that the cost in the US is $120 billion annually. (NISC, 2017)

References:

NISC. (2017). The economic impact of invasive species in the United States. Washington, DC: US National Invasive Species Council.

 

Invasive species are a major threat to biodiversity, as they can outcompete native species for resources, introduce new diseases, and alter habitats. (IUCN, 2016)

References:

IUCN. (2016). Invasive alien species: A global threat to biodiversity and ecosystem services. Gland, Switzerland: IUCN.

 

Invasive species impact people’s quality of life by causing economic losses, disrupting ecosystems, and harming human health.

 

Economic losses can occur when invasive species damage crops, forests, and other natural resources.

 

Disrupted ecosystems can lead to the decline of native species and the loss of ecosystem services, such as water purification and flood control.

 

Harm to human health can occur when invasive species spread diseases or cause allergic reactions.

 

References:

IUCN. (2016). Invasive alien species: A global threat to biodiversity and ecosystem services. Gland, Switzerland: IUCN.

WWF. (2017). Invasive alien species: A global problem in need of a global solution. Gland, Switzerland: WWF.

2nd paragraph

Freshwater habitats have been particularly impacted by global change, with climate change, invasive species, and habitat loss all playing a role.

 

Climate change is causing changes in temperature, precipitation, and water flow that are altering freshwater habitats.

 

Invasive species are non-native species that can outcompete native species for resources, introduce new diseases, and alter habitats.

 

Habitat loss is occurring due to human activities such as development, agriculture, and mining.

 

References:

IUCN. (2016). Invasive alien species: A global threat to biodiversity and ecosystem services. Gland, Switzerland: IUCN.

WWF. (2017). Invasive alien species: A global problem in need of a global solution. Gland, Switzerland: WWF.

 

The impacts of these changes are already being felt, with many freshwater species in decline. If we do not take action to address these challenges, the consequences for freshwater ecosystems and the people who rely on them will be severe.

 

"Invasive species can cause major changes to freshwater ecosystems, including the decline of native species, the alteration of food webs, and the spread of diseases. For example, the introduction of the invasive Asian carp to the Great Lakes has led to the decline of native fish populations and the alteration of the food web." (IUCN, 2016)

 

 

References:

IUCN. (2016). Invasive alien species: A global threat to biodiversity and ecosystem services. Gland, Switzerland: IUCN.

 

 

"Invasive fish, such as Asian carp and largemouth bass, have been added to both rivers and lakes, often through illegal releases or escapes from aquaculture facilities." (IUCN, 2016)

 

References:

IUCN. (2016). Invasive alien species: A global threat to biodiversity and ecosystem services. Gland, Switzerland: IUCN.

 

"Large numbers of introductions of invasive fish have already occurred globally, and are ongoing. Between 1970 and 2000, an estimated 7,000 species of fish were introduced to new areas (IUCN, 2016)."

 

 

References:

IUCN. (2016). Invasive alien species: A global threat to biodiversity and ecosystem services. Gland, Switzerland: IUCN.

 

  Lab  Writing

The results are in - ChatGPT is a fraud

24 March 2023

ChatGPT sounds convincing, but gets it all wrong

In the experiment that I set up last week, I gave ChatGPT 8 statements and asked it to write a referenced sentence on each. These were then combined into two paragraphs in a way that you may consider writing for your PhD thesis.

See the experiment here.

During this week, I checked the statements given and the citations used. Here are the results:

The results of my experiment are in, and it’s bad news for ChatGPT or anyone who might want to try to rely on it for writing referenced statements for their PhD.

Most statements sounded good, and were certainly on message, but the citations were sometimes vague and when they could be pinned down turned out to be about other subjects or did not contain the information claimed. This is really bad news, because at first sight the information seemed credible. In the case of a PhD student, the examiners are likely to know their specialist field and will hopefully recognise the mis-citations. But for undergraduate essays, this is a far more difficult situation as those marking them will not have the time or subject area expertise to check for mis-cites. If it sounds credible, it will most likely fly.

 As a recap

I gave ChatGPT a series of short statements and asked for a sentence with citations. In the following, you will see this broken down statement by statement:

 Paragraph 1

Given: "Invasive species are a growing global problem"

Chat GPT Statement 1: According to the International Union for Conservation of Nature (IUCN), invasive species are a growing global problem that pose a significant threat to biodiversity and ecosystem health.

My assessment of this statement is that it is very vague. While I can easily imagine that IUCN has made (many) statements such as this, it would be very difficult to pin down the source – which is primarily what we are interested in when we use a citation. Thus, while I feel that the statement is credible, it is unhelpful. Many might be tempted to use this text, and retrospectively assign another citation that would fit it. I would caution against this because (a) you’ll waste your time trying to find a citation that fits this text, and (b) you’d be better off reading pertinent literature and citing from what you’ve read. If you are unsure where to start with pertinent literature, try citations within citations from literature that you do know. The first paragraph of their text should give relevant literature for what you are trying to cite in your first paragraph – but always find and read it.

Chat GPT Statement 2: The National Wildlife Federation (NWF) reports that invasive species cost the United States government an estimated $120 billion annually in damages and control measures (Masterson, 2018).

My assessment: I got very stuck with this statement, which has caused me a lot of difficulties in trying to track down this document attributed to Masterson in 2018. A search of the National Wildlife Federation (NWF) website (https://www.nwf.org/) for “Masterson” came up with only two documents (Pathak et al 2022; Small-Lorenz et al., 2017). Neither document is authored by someone called “Masterson”. I have written to NWF to ask them whether they have any insight into where this statement might have come from.

Next is the figure of $120 billion annually. This likely comes from the Pimental et al. (2005) paper, which has now been updated by Fantle-Lepczyk et al. (2022), who have provided the most recent assessment of costs to the USA. In their abstract, they claim that a conservative estimate would be ~$20 billion per year, with ~$73 billion as an upper bound on this figure. Hence, if Masterson really did write a report for NWF then it seems that this was a very high estimate, and not in line with other (more recent) literature. Given the wide ranging access to the internet, it is surprising that ChatGPT decided to use the NWF source.

Chat GPT Statement 3: The Convention on Biological Diversity (CBD) states that invasive species are one of the main threats to global biodiversity, causing species extinctions, altering ecosystems, and reducing ecological resilience (CBD, 2021).

My assessment: This is another very vague statement, this time attributed to the Convention on Biological Diversity (CBD). They produced a large number of documents in 2021, and trying to find the one that made this statement, and on what it would be based would require plenty of time. I don’t doubt that it is correct, but it would be better to use a source that others can find. Thus, rather like statement 1, I would junk this statement as unhelpful for including in your PhD.

Chat GPT Statement 4: A study conducted by the University of California, Davis, found that invasive species can have negative impacts on human health, recreation, and property values, affecting people’s quality of life in areas where they are present (Koch et al., 2011).

My assessment: Here I started by searching the UCD website for “Koch invasive”. This came up with a number of hits, but none of which looked like anything that could be linked to the statement given. Next, I tried searching Scopus with “invasive species” in the Keywords, Abstract and , and Koch as first author. In 2011, only one paper by Frank H. Koch (of the Department of Forestry and Environmental Resources and USDA Forest Service, both in North Carolina – none of the other authors were from UCD) was written on invasive forest insect species (Koch et al., 2011). The article does not mention house prices, human health or quality of life. Had I been looking for a citation for a statement like this I might have started with the Blackburn et al. EICAT framework [@blackburn2014unified] and looked for examples of people citing this with their specified categories.

At the end of the first paragraph, I have to say that I’m not impressed with what ChatGPT actually came up with. It sounded good, but the statements were vague, and no sources could be verified. Hence, in terms of how useful this would be for writing your PhD, I would say that it gets zero points and was probably a waste of time. I would hope that you are more familiar with the broad context of your work to write a better first paragraph from scratch.

Paragraph 2

Chat GPT Statement 5: A global analysis of freshwater ecosystems found that they are among the most vulnerable to the impacts of climate change, pollution, and invasive species, with declines in biodiversity and ecosystem services being reported worldwide (Vörösmarty et al., 2010).

My assessment: Paragraph 2 looks like it is starting on a much firmer footing, and this suggests to me that ChatGPT needs more specific input in order to get it to be more useful. Firstly, there is a paper by Vörösmarty et al. (2010), and it is about fresh water. But it doesn’t mention invasive species. What is actually says is: “non-native species threaten biodiversity but are typically inconsequential to human water security.” This appears to be the opposite of what ChatGPT thought, and while we might have been tempted to keep this sentence as the citation (1) existed and (2) is about fresh water, again we find that this statement doesn’t match what we needed. There are papers that would be a much better fit for our purposes like Gallardo et al. (2016) and Gallardo et al. (2018).

Chat GPT Statement 6: Studies have shown that the introduction of invasive fish species can cause significant ecological and economic changes in freshwater systems, including alterations in food webs, declines in native fish populations, and reduced recreational and commercial fishing opportunities (Leprieur et al., 2008; Ricciardi & MacIsaac, 2011).

My assessment: Both citations here are readily findable and are on topic. Leprieur et al. (2008a) does indeed discuss economic activities, but finds evidence that they correlate with increased likelihood of non-native fish introductions – not that non-native species cause economic changes as claimed by ChatGPT. Similarly, as Leprieur et al. (2008b) study the patterns of invasive fish occurrence, they do not have any findings about ecological impact. Ricciardi & MacIsaac (2010) is a book chapter that focuses on the impacts of invasions in freshwater. There is a brief mention of economic impacts in their historical perspective, otherwise they do not discuss this issue. There is a good review of ecological impacts, and they do give examples of alterations in food webs, declines in native fish populations and a mention of impact on commercial fisheries. There is no assessment of impacts on recreational fishing.

Thus, for statement 6, ChatGPT failed with one citation, but got some success with another albeit less than 100%. It’s not a complete fail, but it is mostly wrong.

Chat GPT Statement 7: Invasive fish species have been intentionally or unintentionally introduced into both rivers and lakes, causing significant ecological impacts and economic costs globally (Savini et al., 2010; Phelps et al., 2016).

My assessment This statement is rather like a reiteration of the previous statement. As I asked for these separately, we cannot blame ChatGPT for this instance of repetition, but let’s see how it does on fish introductions to rivers and lakes – which is what was asked for.

Savini et al. (2010) is a paper about the top 27 alien animals introduced into Europe for aquaculture, so it is on topic with lots of information about invasive freshwater fish. However, it does not mention anything about rivers or lakes, does not mention intentional or unintentional introductions, but does give extensive examples about ecological impact. No specific mention of economic impacts are made, although they could be inferred due to some of the mentions of damaged aquacultural activities due to invasions. Not a complete fail then, but certainly this citation cannot be used to substantiate the statement made by ChatGPT alone.

The Phelps et al. citation could have been one of two papers published that year with Phelps as a lead author about fish. Phelps et al. (2010a) is about Shovelnose Sturgeon in North American rivers and does not mention invasive fish, but does talk about re-introductions. Phelps et al. (2010b) is about the illegal wildlife trade and does mention fish, but only in relation to trade. Neither of these publications could be cited to affirm the statement made by ChatGPT.

Chat GPT Statement 8: A study by Jeschke et al. (2014) found that over 12,000 introductions of invasive fish species have already occurred globally, with new introductions continuing to happen at an increasing rate, highlighting the urgent need for effective prevention and management strategies.

My assessment There is a paper by Jeschke et al. (2014). Rather than being about invasive fish, this paper covers impact of all invasive species. Non-native fish are given as an example of differences in opinions between quantifiable and value of change of impact, but are otherwise not mentioned. If this was the publication that ChatGPT was referring to, then it was a complete fail as none of Statement 8 could be attributed to this paper.

 

Was ChatGPT a complete fail?

Yes, I would say that having checked the references, most of what was said sounded good, but could not be backed up by the citations given, in the cases where they could be found. For this reason, I am confident in calling this experiment a complete fail, not because there was no truth, but because it was mostly misleading (incorrect citations and unhelpful misdirection to correct citations).

For most of you who are trying to write their PhD thesis, I would suggest that you are completely wasting your time to try this approach with ChatGPT. This doesn’t mean that you can’t use it as an to writing, but that you can’t rely on it to produce anything that is factually correct.  

Perhaps ChatGPT is getting all these citation uses wrong because it is looking to see how other people use citations? This is not unreasonable, as mis-citing is a real problem. But having this platform reinforcing false citation habits is the last thing you need.

Given that it’s taken me several hours to check two paragraphs, I feel no hesitation in suggesting that you avoid using ChatGPT in this way – and if you do, beware – it probably got it all wrong.

If ChatGPT were a student, I'd have some choice words to say after spending all this time and finding out that almost nothing was correct. If I was feeling charitable, then I'd say "cocky" and "unafraid of making up rubbish in order to have something sound nice".

Literature Cited (by me - who knows what ChatGPT was citing)

Blackburn, Tim M., Franz Essl, Thomas Evans, Philip E. Hulme, Jonathan M. Jeschke, Ingolf Kühn, Sabrina Kumschick, et al. “A Unified Classification of Alien Species Based on the Magnitude of Their Environmental Impacts.” PLOS Biology 12, no. 5 (May 6, 2014): e1001850. https://doi.org/10.1371/journal.pbio.1001850.

Fantle-Lepczyk, J.E., P.J. Haubrock, A.M. Kramer, R.N. Cuthbert, A.J. Turbelin, R. Crystal-Ornelas, C. Diagne, and F. Courchamp. “Economic Costs of Biological Invasions in the United States.” Science of the Total Environment 806 (2022). https://doi.org/10.1016/j.scitotenv.2021.151318.

Gallardo, B., A.E. Bogan, S. Harun, L. Jainih, M. Lopes-Lima, M. Pizarro, K.A. Rahim, R. Sousa, S.G.P. Virdis, and A. Zieritz. “Current and Future Effects of Global Change on a Hotspot’s Freshwater Diversity.” Science of the Total Environment 635 (2018): 750–60. https://doi.org/10.1016/j.scitotenv.2018.04.056.

Gallardo, B., M. Clavero, M.I. Sánchez, and M. Vilà. “Global Ecological Impacts of Invasive Species in Aquatic Ecosystems.” Global Change Biology 22, no. 1 (2016): 151–63. https://doi.org/10.1111/gcb.13004.

Jeschke, Jonathan M., Sven Bacher, Tim M. Blackburn, Jaimie T. A. Dick, Franz Essl, Thomas Evans, Mirijam Gaertner, et al. “Defining the Impact of Non-Native Species.” Conservation Biology 28, no. 5 (2014): 1188–94. https://doi.org/10.1111/cobi.12299.

Koch, Frank H., Denys Yemshanov, Manuel Colunga-Garcia, Roger D. Magarey, and William D. Smith. “Potential Establishment of Alien-Invasive Forest Insect Species in the United States: Where and How Many?” Biological Invasions 13, no. 4 (April 1, 2011): 969–85. https://doi.org/10.1007/s10530-010-9883-8.

Leprieur, Fabien, Olivier Beauchard, Simon Blanchet, Thierry Oberdorff, and Sébastien Brosse. “Fish Invasions in the World’s River Systems: When Natural Processes Are Blurred by Human Activities.” PLOS Biology 6, no. 2 (February 5, 2008): e28. https://doi.org/10.1371/journal.pbio.0060028.

Phelps, Jacob, Duan Biggs, and Edward L Webb. “Tools and Terms for Understanding Illegal Wildlife Trade.” Frontiers in Ecology and the Environment 14, no. 9 (2016): 479–89. https://doi.org/10.1002/fee.1325.

Phelps, Q. E., S. J. Tripp, M. J. Hamel, J. Koch, E. J. Heist, J. E. Garvey, K. M. Kappenman, and M. a. H. Webb. “Status of Knowledge of the Shovelnose Sturgeon (Scaphirhynchus platorynchus, Rafinesque, 1820).” Journal of Applied Ichthyology 32, no. S1 (2016): 249–60. https://doi.org/10.1111/jai.13241.

Pimentel, David, Rodolfo Zuniga, and Doug Morrison. “Update on the Environmental and Economic Costs Associated with Alien-Invasive Species in the United States.” Ecological Economics, Integrating Ecology and Economics in Control Bioinvasions, 52, no. 3 (February 15, 2005): 273–88. https://doi.org/10.1016/j.ecolecon.2004.10.002.

Ricciardi, Anthony, and Hugh J. Macisaac. “Impacts of Biological Invasions on Freshwater Ecosystems.” In Fifty Years of Invasion Ecology, 211–24. John Wiley & Sons, Ltd, 2010. https://doi.org/10.1002/9781444329988.ch16.

Savini, D., A. Occhipinti-Ambrogi, A. Marchini, E. Tricarico, F. Gherardi, S. Olenin, and S. Gollasch. “The Top 27 Animal Alien Species Introduced into Europe for Aquaculture and Related Activities.” Journal of Applied Ichthyology 26, no. SUPPL. 2 (2010): 1–7. https://doi.org/10.1111/j.1439-0426.2010.01503.x.

Vimercati, Giovanni, Anna F. Probert, Lara Volery, Ruben Bernardo-Madrid, Sandro Bertolino, Vanessa Céspedes, Franz Essl, et al. “The EICAT+ Framework Enables Classification of Positive Impacts of Alien Taxa on Native Biodiversity.” PLOS Biology 20, no. 8 (August 16, 2022): e3001729. https://doi.org/10.1371/journal.pbio.3001729.

Vörösmarty, C.J., P.B. McIntyre, M.O. Gessner, D. Dudgeon, A. Prusevich, P. Green, S. Glidden, et al. “Global Threats to Human Water Security and River Biodiversity.” Nature 467, no. 7315 (2010): 555–61. https://doi.org/10.1038/nature09440.

  Lab  Writing

What's up at eLife?

22 March 2023

Brave New worlds need brave new people


When eLife announced that they were going to publish every article that was reviewed, I reacted like some others in simply focusing on the monetary gain that that may imply. 


This may just have been a general cynical reaction to profiteering by publishers in general. At the time I had not realised that eLife is a not for profit organisation. This was mostly because of their APC which was 3,000 USD (now reduced to 2000 USD). As we know (see here) it is hard to justify such an enormous APC. I would welcome eLife to explain in detail why it is that they cannot reduce their APC to something more reasonable. A diamond open access eLife being the model that we would all want to see.

But the attempt at an experiment to change the way in which researchers publish, this is actually very welcome and I applaud eLife and the editors that support this change. The concept that we as readers should be able, and indeed have to, to look at not only the article but also the reviews is very important. It is part of the critical reading that we should be engaged in anytime we read a scientific article and make a decision based on the value therein. At its heart, this model is an attempt to remove the idea of prestige from a journal title. 

So is the removal of prestige the reason why there's been such a terrible kickback?

I have made the comment before that the people who really benefit from the current system which is heavy on prestige are the gatekeepers who have obtained and maintained their positions based on this prestige (see here). I was very interested then to read that eLife editor Mike Eisen agrees that this could be the reason for the kickback.

“...opposition to eLife’s model is driven fundamentally by powerful scientists not wanting to change a system that has benefited them and which they have sculpted to continue to reward them”.
Michael Eisen in Nature: News Feature 

I think that Eisen is right. The reason why these prestigious editors do not want eLife to change its model is because they are only interested in the prestige that they gain by being associated with its title if it continues to be prestigious. This is one of the reasons why we need to change the current publishing system: albeit a wicked problem with many facets that require change (see here).

Perhaps a mistake that eLife has made has been to try to make these changes from the top end of publishing. We saw the same attempt with Public Library of Science (PLoS), who started with high-end journals ls and then initiated PLoS-ONE - eventually leading to serious struggles for their own survival (see here). Getting funders to consider new publishing models appears to necessitate starting at the top, otherwise they take no notice. 

"Following its initial success in establishing itself as an open access publisher of high-quality science…"

Would Nature News Feature have written an article about change in editorial policy at a low impact factor diamond open access journal? I think not. The reason why this is big news is because big decisions are made around the funding of prestigious work. Threaten to take away that prestige and those who have benefited from it may well get upset. In this case they called for the axing of Eisen as editor. eLife is backed by some powerful funders, so it'll be interesting to watch their reaction as eLife pursues this new publishing model.

This is a multi-layered argument but it's an important one for science. Do you think science would benefit from a world without an accept / reject decision?

Further Reading
Measey J (2022) How to publish in Biological Sciences: a guide for the uninitiated. CRC Press, Boca Raton.  ISBN: 9781032116419 https://doi.org/10.1201/9781003220886   
To Read Open Access version: www.howtopublishscience.org
  Lab  Writing

Will Chat GPT write my PhD?

19 March 2023

Can an AI write my PhD?

The emergence of artificial intelligence (AI) platforms brings with it new issues around writing generally, including writing your PhD thesis. These bots are already widely available in early 2023, and there are already publications that list ChatGPT as an author (Mijwil, Aljanabi & ChatGPT, 2023O’Connor & ChatGPT, 2023) in acknowledgement that a large amount of the text therein had been generated by the bot.

More AI writing tools are going to be coming online in the near future, Google’s LaMDA and Microsoft’s Bing are very close to release. Microsoft plan to integrate their Bing platform directly into their office suite, meaning that as an author you may unwittingly use AI generated automated suggestions inside your word processor without realising it. Thus, you should be interested in knowing whether such tools can be used to help you in your own writing, if you haven’t already tried (Owens, 2023), with the tantalising possibility that they could be used to replace your needing to write.

In this blog post (and in a chapter in How to Write a PhD) I set out some ideas about how you might use AI to help you write, as well as some things that you should consider if you do go that route.

What is ChatGPT?

I decided to ask ChatGPT to contribute to this chapter by describing itself. My input text: “I am writing a chapter for a book on academic writing and I want you to describe ChatGPT in your own words”. Here’s what it said:

As a language model based on the GPT-3.5 architecture, ChatGPT is a sophisticated artificial intelligence program that has been trained on a massive amount of text data from various sources. It is designed to understand natural language and generate responses that are contextually relevant and coherent.

ChatGPT uses deep learning algorithms to analyze text inputs and predict the most probable next word or phrase, based on patterns and associations it has learned from its training data. This allows it to generate human-like responses to a wide range of questions and prompts, from simple queries to complex conversations.

One of the unique features of ChatGPT is its ability to understand and generate text in multiple languages, making it a valuable tool for communication and collaboration across different cultures and regions. Additionally, it has been used for a wide variety of applications, including language translation, content creation, and customer service.

Overall, ChatGPT represents a significant breakthrough in artificial intelligence research and has the potential to transform the way we communicate and interact with technology.

 

Large language platforms have been around for a while

Although at the beginning of 2023 we are seeing a large number of popular articles about ChatGPT, the use of language platforms in writing in general, including writing in the Biological Sciences, has been around for a while. If English is not your first language, then you already know how incredibly useful it is to enter text to a translation app in your own language and get a English back.

Some years ago, these translation platforms were rather literal, simply translating each word and providing text back that was barely usable. Today, with the help of AI, the translation that returns is more than not often a reasonable sounding (near) grammatically correct translation of what was entered. Hence, if you have used such translations in your own writing, then you should have already considered the a lot of contents of this chapter, and if you haven’t then you need to read on.

Interestingly, when ChatGPT was asked to provide text in French and Arabic, it failed to produce coherent text (Seghier, 2023), although it is possible that this will change over time as it becomes more sophisticated. It is interesting to note the difference between the claim that ChatGPT made (above) and this assertion from Seghier (2023) a few months ago. Whether or not ChatGPT has improved has yet to be tested.

Output from ChatGPT cannot be trusted

The first, and perhaps most important issue, about using ChatGPT outputs in your own writing is that you cannot trust it to generate text that is factually correct or unbiased. Indeed, OpenAI who own and administer ChatGPT readily admit that their bot can produce factually incorrect statements and is often biased in its output (Owens, 2023).

Because ChatGPT uses data on the internet to generate the basis of the text that it generates, it is prone to providing the internet consensus about popular topics, and studies have already shown that there is a bias in what is produced (McGee, 2023). This is something that OpenAI have already acknowledged, but that they are working to improve bias with user feedback (Owens, 2023).

This means that you should be checking any AI text that is generated before you add it to your chapter or manuscript.

 

Plagiarism by AI

There is the chance that your AI tool is going to plagiarise text that it finds on the internet, and so you may unwittingly be pasting the words of another author into your own text. Some authors consider that even if the AI is programmed to avoid plagiarism, using this text in any way should be considered as plagiarism (Salvagno, Taccone & Gerli, 2023). However, on balance, my own view is that any attempt to be overly restrictive on the use of AI generated text is bound to fail as AI itself becomes increasingly more sophisticated and widespread. Instead, I share the view (Lim et al., 2023) that it is better to embrace and learn how to use AI as a tool.

Perhaps a more fundamental consideration is whether AI will require us to redefine what we mean by plagiarism. Certainly, this is a philosophical consideration that right now (at the start of 2023) you will need to decide for yourself, although as more AI tools come online and are more commonly available, I think that many journal guidelines will be prescriptive on their use.

Plagiarism can be thought of as taking someone else’s writing without attribution. Another definition of plagiarism is pretending that someone else’s work is your own. While these two definitions may appear interchangable at first sight, when considering use of text from an AI, you might only infringe the attribution rule while the AI is programmed to infringe the pretence. In other words, if the AI is not a person, then following the first definition of AI you have not taken someone else’s writing. However, you might consider that as the AI itself was the product of someone else’s work, then they are the person (or group of people) that generated that text, and so the work should be attributed to them. This then brings you to the second definition of plagiarism. But what if that group of people don’t ask for any attribution or acknowledgement? Should you still give them as an author or put them into the acknowledgements? Moreover, could this team really be said to have generated the text when they have created software that searches text written by others on the internet and then collate and rewrite it? The huge number of content creators on the internet can never be acknowledged individually.

Another way of thinking about this might be to consider human interactions in writing text. If, for example, I help some colleagues with the English text of a manuscript and they offer to place my name in the acknowledgements, but I tell them that there is really no need. Should they still acknowledge me even though I have told them not to?

A further example might be the use of different layers in GIS that have been generated by different people. Many journals now insist that there is attribution to these layers in the legend of the figure or in the acknowledgements. We might legitimately ask whether this is necessary when those layers are freely available and have non-attribution Creative Commons licence (see here)?

Clearly, the definition of plagiarism will need some work in the light of new abilities for AIs to write text.

To me, the possibility that by using AI generated text verbatim you could be using someone else’s words means that you should avoid this. The same goes whether this is a translation from a large language platform or a new chat bot style AI. Using such platforms as tools is certainly acceptable, while using their text verbatim is probably an example of false attribution.

 

Ethics

In addition to the attribution of authorship, and potential plagiarism issues, you should also be aware of the ethical component of using text from an AI tool.

The kind of ethics points that come up are similar to those that are raised by paper mills (Salvagno, Taccone & Gerli, 2023). This issue is covered comprehensively elsewhere: see Chapter on when you should be an author in Measey (2022).

 

Positive aspects of using AI

It will not come as any surprise that many authors often find it difficult to get started. The classic image is starting at a blank piece of paper, or these days a flashing cursor on an empty screen. In this book I provide a lot of suggestions about how to get started with your writing task:

but can ChatGPT provide another opportunity when you don’t know how to start?

 

Use AI as a tool

Essentially, if you want to try using AI to help you write, use it as a tool. Don’t try to use it to replace your own writing. That will likely end in a poor product, and could possibly end with factually incorrect or biased text that you won’t want to be associated with your name - and clearly not something that you should want your advisor to read.

Getting started is often a struggle and I regularly suggest that students start by writing something/anything to fill out their outline, even if all of that text gets replaced at a later date. This is a way that you could use a tool like ChatGPT. Produce your own outline, because you do still need to plan your chapter, but then try using an AI like ChatGPT to flesh this out. Hopefully, this will end by helping you to write your own text, replacing everything that the AI has produced.

 

Using ChatGPT as an example of getting started with writing

I have tried this approach below and you can judge the outcome for yourself. First, I write the text outline, and second I provide each point to ChatGPT. For the following outline, I provided the outline statement (each bullet point) in quotes and prefixed it with “Write a sentence with citations on:”…”

In this example, the outline below is similar to what I would produce when writing an outline for the first two paragraphs of an introduction to a manuscript on invasive fish:

  • Invasive species are a growing global problem
    • invasive species cost a huge amount of money to governments
    • invasive species are a threat to biodiversity
    • invasive species impact people’s quality of life
  • Freshwater habitats have been particularly impacted by global change
    • important changes have occurred to freshwater systems after the introduction of invasive fish”
    • invasive fish have been added to both rivers and lakes
    • large numbers of introductions of invasive fish have already occurred globally, and are ongoing

The following text was generated by ChatGPT (on 19 March 2023) given the primer text above, and the stipulation that it should produce a sentence with citations. I have simply combined the output text into the paragraphs as might be expected in the introduction to a manuscript:

According to the International Union for Conservation of Nature (IUCN), invasive species are a growing global problem that pose a significant threat to biodiversity and ecosystem health. The National Wildlife Federation (NWF) reports that invasive species cost the United States government an estimated $120 billion annually in damages and control measures (Masterson, 2018). The Convention on Biological Diversity (CBD) states that invasive species are one of the main threats to global biodiversity, causing species extinctions, altering ecosystems, and reducing ecological resilience (CBD, 2021). A study conducted by the University of California, Davis, found that invasive species can have negative impacts on human health, recreation, and property values, affecting people’s quality of life in areas where they are present (Koch et al., 2011).

A global analysis of freshwater ecosystems found that they are among the most vulnerable to the impacts of climate change, pollution, and invasive species, with declines in biodiversity and ecosystem services being reported worldwide (Vörösmarty et al., 2010). Studies have shown that the introduction of invasive fish species can cause significant ecological and economic changes in freshwater systems, including alterations in food webs, declines in native fish populations, and reduced recreational and commercial fishing opportunities (Leprieur et al., 2008; Ricciardi & MacIsaac, 2011). Invasive fish species have been intentionally or unintentionally introduced into both rivers and lakes, causing significant ecological impacts and economic costs globally (Savini et al., 2010; Phelps et al., 2016). A study by Jeschke et al. (2014) found that over 12,000 introductions of invasive fish species have already occurred globally, with new introductions continuing to happen at an increasing rate, highlighting the urgent need for effective prevention and management strategies.

It is noteworthy that while ChatGPT did get into the swing of producing citations later on, the first couple of sentences didn’t have anything concrete. Neither did it see fit to produce the actual references themselves to the citations (although I didn’t ask it to do so). There is also a notable bias to the USA in the results given in the first paragraph. Overall though, I feel that the results of this exercise are somewhat impressive, and while I personally would not use this text verbatim, it is certainly better than a lot of first drafts that I’ve written myself.

As an exercise, I have tried to find each of these citations in an attempt to determine whether ChatGPT was accurate. In addition, I have run the two paragraphs above through a plagiarism detection software. I’ll write the results of that here once it’s complete.

Using a plagiarism checker (TurnItIn), the output suggested that: No plagiarism was found.

 

Will others detect ChatGPT if I use it?

Some people claim that they can detect the output from ChatGPT as it lacks the depth and insight that original authors usually have. In other words, the aim of ChatGPT is to produce the words (in a grammatically correct manor), while the aim of an author is to transmit an idea to the reader. As you might expect and hope, AI is not at the point where it can generate the same intent to communicate.

There will likely be better AI chat bots in the future, and ChatGPT itself is constantly getting feedback from users that should improve its own output.

 

ChatGPT, or other AI platforms, cannot be an author

You may be aware of several articles that are already published with ChatGPT listed in the author line (Mijwil, Aljanabi & ChatGPT, 2023O’Connor & ChatGPT, 2023). Decisions on whether or not this is allowed are likely to be journal specific. However, some publishing houses are already making journal wide decisions excluding the use of AIs as authors.

This is clearly a dynamic process and you should be aware of what is happening in your own field.

 

Transparency

However you use an AI platform, you should be transparent to your advisor and any colleagues that you publish with about the exact level of use that was involved in generating your text. As a general rule with writing and academia, transparency is the best policy. Some authors have already called for regulations (Salvagno, Taccone & Gerli, 2023). Be aware that journals do update their instructions to authors, so you may need to look for statements on AI text generation. Similarly, familiarise yourself with your institutions requirements for thesis submission, and be careful you do not transgress any recently added rules.

 

COPE have already provided guidelines

The Committee on Publication Ethics (COPE) have already provided some guidelines to their members (mostly publishers) on the use of AI (COPE, 2023) which you can find here. In general, these rules emphasise the importance of oversight and transparency in the use of AI tools in decision making.

 

Publishers using AI bots

Publishers are very interested in using the text from AI bots to generate lay summaries of articles that are published on their platforms. Certainly, there is evidence that scientists are already using the bots to create their own summaries (Owens, 2023). This is certainly a possible creative use for AIs, but I would be concerned that without careful curation they may be prone to producing factually incorrect or misleading content. If you plan to use AI software to popularise your own science, then I suggest that you carefully read anything that is created and ensure that the text is correctly attributed when you use it.

 

Last note

This text was written in RMarkdown, without the use of suggestive prompts. AI generated text is written as quotes in the above text. All other text is my own.

It will be interesting to look back on this chapter in a decade and see the changes that have emerged in that time.

I would like to thank OpenAI and ChatGPT for generating the quoted text in this chapter. I’d also acknowledge TurnItIn for their plagiarism check.

References

COPE. 2023. Artificial intelligence (AI) in decision making.
Lim WM, Gunasekara A, Pallant JL, Pallant JI, Pechenkina E. 2023. Generative AI and the future of education: Ragnarök or reformation? A paradoxical perspective from management educators. The International Journal of Management Education 21:100790. DOI: 10.1016/j.ijme.2023.100790.
McGee R. 2023a. Capitalism, Socialism and ChatGPT. DOI: 10.13140/RG.2.2.30325.04324.
Measey J. 2022. How to publish in Biological Sciences: A guide for the uninitiated. Boca Raton, Florida: CRC Press.
Mijwil M, Aljanabi M, ChatGPT. 2023. Towards Artificial Intelligence-Based Cybersecurity: The Practices and ChatGPT Generated Ways to Combat Cybercrime. Iraqi Journal For Computer Science and Mathematics 4:65–70. DOI: 10.52866/ijcsm.2023.01.01.0019.
O’Connor S, ChatGPT. 2023. Open artificial intelligence platforms in nursing education: Tools for academic progress or abuse? Nurse Education in Practice 66:103537. DOI: 10.1016/j.nepr.2022.103537.
Owens B. 2023. How Nature readers are using ChatGPT. Nature 615:20–20. DOI: 10.1038/d41586-023-00500-8.
Salvagno M, Taccone FS, Gerli AG. 2023. Can artificial intelligence help for scientific writing? Critical Care 27:1–5. DOI: 10.1186/s13054-023-04380-2.
Seghier ML. 2023. ChatGPT: Not all languages are equal. Nature 615:216–216. DOI: 10.1038/d41586-023-00680-3.
This text is an excerpt from the book: How to Write a PhD in Biological Sciences - which you can find Open Access here
  Lab  Writing
Creative Commons Licence
The MeaseyLab Blog is licensed under a Creative Commons Attribution 3.0 Unported License.