How to use generative AI in research responsibly

Basic principles and practical implications

Why This Matters 
Generative AI is increasingly used in research for tasks like summarizing literature, drafting text, writing code, or exploring data. This provides new opportunities and can help you to work more efficiently. At the same time, this also raises new questions about what these tools mean for research integrity. To have an educated opinion on this matter, on one hand we need to understand the risks of using generative AI for research for us as individuals, but also for academia and society at large (also see my previous blog post on generative AI). On the other hand, we need to consider (or rethink) traditional definitions of the concept of research integrity. These are topics too large for an individual to solve. 

To make it easier for you as a researcher to make responsible decisions today about using generative AI in your workflows, guidelines are slowly being created. Some institutions already have their own, others do not. The most comprehensive guidelines out there are the Living guidelines on the responsible use of generative AI in research by the European Commission. These guidelines should help you to minimize the risk and maximizing the potential of generative AI. 

What is behind these guidelines? 

As a researcher, you commit to the principles of good scientific practice. These are also operationalized in guidelines, such as guidelines published by the DFG in Germany or the ALLEA. Most institutions have their own guidelines, but they all share key values and principles. 

The European Code of Conduct for Research Integrity defines the core values as follows: 

• Reliability in ensuring the quality of research, reflected in the design, methodology, analysis, and use of resources. 
• Honesty in developing, undertaking, reviewing, reporting, and communicating research in a transparent, fair, full, and unbiased way. 
• Respect for colleagues, research participants, research subjects, society, ecosystems, cultural heritage, and the environment. 
• Accountability for the research from idea to publication, for its management and organisation, for training, supervision, and mentoring, and for its wider societal impacts. 

These core values are also translated to the responsible use of generative AI. 

The worst violation of these values is scientific misconduct, often summarized in the FFP framework: 

·       Fabrication – making up data or results 

·       Falsification – manipulating data 

·       Plagiarism – using others’ work without attribution

While generative AI facilitates the fabrication and falsification of data, generally it takes bad intentions to actually do so. Plagiarism on the other hand, is debated heavily as an unintentional risk of generative AI, in particular when it comes to the creation of new ideas and writing text. 

How to translate the values of research integrity to generative AI 

Reading through various guidelines and recommendations on the responsible use of generative AI, there are a few main general points that are made that are related to the core values: 

• Reliability: AI outputs must be critically verified. Be aware of technical limitations, such as inherent model biases. 
• Honesty: Be transparent about the use of generative AI. 
• Respect: Do not upload personal data to commercial AI systems unless compliant with data protection laws. 
• Accountability: AI tools cannot take responsibility and thus cannot be authors. Researchers remain liable for all content produced using AI. 

What this means in practice

Transparency: Reporting the use

The DFG states here: “When making their results publicly available, researchers should, in the spirit of research integrity, disclose whether or not they have used generative models, and if so, which ones, for what purpose and to what extent.”

Journal guidelines argue along the same lines. An exception of this is often the use of grammar and spell checks.

Now the question is, how to report the use. Here, some journals are quite vague and some have more in-depth guidelines, such as JAMA. They state: 

“Authors should report in the Acknowledgment section the following: 

• Name of the AI software platform, program, or tool 
• Version and extension numbers 
• Manufacturer 
• Date(s) of use 
• A brief description of how the AI was used and on what portions of the manuscript or content 

Confirmation that the author(s) take responsibility for the integrity of the content generated.”

As journal requirements may change and become stricter, documenting thoroughly (and keeping the prompts and raw outputs) can be helpful in later providing the requested information.

Be careful with image generation 

Picture generation and amendments are often not permitted by journals (with some exceptions for when AI tools are explicitly part of the methodology). This is what some journals say about that: 

•       Elsevier: “We do not permit the use of Generative AI or AI-assisted tools to create or alter images in submitted manuscripts.” 

•       Nature: “While legal issues relating to AI-generated images and videos remain broadly unresolved, Springer Nature journals are unable to permit its use for publication.” 

•       PLOS: “The use of AI tools and technologies to fabricate or otherwise misrepresent primary research data is unacceptable.” 

•       JAMA: “The submission and publication of images created by artificial intelligence, machine learning tools, or similar technologies is discouraged, unless part of formal research design or methods, and is not permitted without clear description of the content that was created and the name of the model or tool, version and extension numbers, and manufacturer.” 

Using generative AI in peer-reviewing is not appreciated 

Another yet no-go is using generative AI for peer-reviewing. The DFG states: 

The DFG states here: “The use of generative models in the preparation of reviews is inadmissible due to the confidentiality of the assessment process. Documents provided for review are confidential and in particular may not be used as input for generative models.” 

When you review, check with the journal or funder’s specific guidelines whether and how you can use generative AI. 

Try to use data protection compliant tools 

Whenever you can, use trustworthy tools. Ideally, your institution should provide tools that allow processing sensitive data without sharing them with commercial entities. 

In Germany, a data compliant tool is chat-ai.academiccloud.de. This is hosted in Germany, but be aware that the data you upload there still leaves your local computer and using this tool may not be in line with your data protection rules (especially when you are working with sensitive data).

Takeaway 

Generative AI is rapidly evolving and at this point (and maybe never), there is no universal rulebook about its use in research — what we have are defined values, growing expectations of transparency, and clear red lines (such as fake papers and image fabrication). 

As a researcher, you should: 

• Stay informed of current guidelines. They are aimed at helping you.
• Reflect on your use of AI and document it so thoroughly that you can comply with the ever-changing journal and funder’s requirements. 
• When in doubt, prioritize transparency and accountability.