From the book

What does a good prompt look like for a researcher?

· By Robert Scales · 4 min read

Short answer

A good research prompt asks ChatGPT to help you map a question, its terms, subtopics, methods and search strings, rather than to hand you finished citations, and it forbids invented sources outright. Every reference that ends up in your work comes from a database or a paper you opened yourself.

Researchers ask ChatGPT for two very different things, and only one of them works well. The first is help with thinking: breaking a broad question into parts, listing the terms different fields use for the same idea, suggesting study designs, drafting search strings, spotting gaps in an argument. The model is genuinely useful here. The second is facts about the literature: "Give me ten peer-reviewed papers on this topic with citations." That is where it fails, sometimes badly.

The reason is how a language model works. It writes the most likely next words given everything before them. A citation is just a very plausible pattern of author names, a year, a title and a journal, so the model can produce one that looks perfect and does not exist. In June, a judge in New York fined two lawyers $5,000 each after they filed a brief citing court decisions ChatGPT had made up, including one it insisted was real when they asked. UNESCO's quick start guide on ChatGPT in higher education, published in April, puts the general rule plainly: ChatGPT has no way to verify the information it gives.

So a good research prompt plays to the first strength and closes the door on the second.

The parts of a research prompt

A Beginner's Guide to ChatGPT suggests telling ChatGPT your research field, the question you are working on and the kinds of sources you need, then saying how you want the results presented. For a scoping task, that becomes five parts:

  1. Role and field. Name the discipline so the vocabulary fits.
  2. The question and its limits. Population, setting, time frame, what you are leaving out.
  3. What you want from the model. Concepts, terms, methods, search strings, open questions: things you can check.
  4. The source rule. No specific papers, authors or statistics unless you supplied them.
  5. The format. A table or numbered sections you can work through.

A worked example: scoping a literature review

Here is a prompt for a graduate student starting a review on whether street trees reduce summer heat in cities:

Prompt
You are an experienced research librarian in urban environmental science.

My question: how much do street trees and urban tree canopy reduce summer heat at street level in mid-sized North American cities, and who benefits most?

Help me scope the literature. Give me:
1. The 5 to 7 subtopics a review on this question should cover.
2. For each subtopic, the key terms and synonyms different fields use (for example, ecology, urban planning, public health).
3. The common study designs and measurements, with one sentence on the strength and weakness of each.
4. Two Boolean search strings I can paste into a database such as Web of Science or Scopus.
5. Three open questions or debates I should look for.

Rules: do not name specific papers, authors, journals or statistics. If you are unsure whether a term is standard, say so. Present parts 1 to 3 as a table.

Every item in that output is something you can test in a few minutes: run the search strings, see whether the terms return relevant work, and keep what holds up. If an answer does not match what you find in the database, the database wins.

Once you have real papers, the model becomes useful again, this time with text you supply. Paste in an abstract and ask for structure:

Prompt
Below is the abstract of a paper I am reviewing. Extract: the city and climate, the study design, how heat was measured, the main finding, and any limitation the authors state. Quote the sentence each item comes from. If an item is not in the abstract, write "not stated".

Abstract: """[paste abstract]"""

Asking for the exact sentence behind each item makes errors easy to spot, and "not stated" gives the model an honest way out instead of a reason to guess.

Try it: Pick a question from your own field and run the scoping prompt with the source rule included. Then paste both search strings into a database you use and count how many of the first 20 results are relevant. Revise the terms and try again.

Where the model helps, and where it doesn't

TaskChatGPT is usefulDo this yourself
Breaking down a questionYesDecide the final scope
Terms and search stringsYesTest them in the database
Summarizing a pasted abstractYes, with quotesRead the paper
Finding papers and citationsNoSearch databases and reference lists
Recent findingsNo; the free version's training data stops in 2021Check current sources

The book recommends working in rounds: start general, read the answer, then narrow the request with more context. Research scoping fits that rhythm well. Each round of searching tells you something to feed back into the next prompt.

How do you keep sources honest?

Make one rule and keep it without exceptions: no reference goes into your work unless you have found it in a database, a library catalogue or a reference list, and opened it. A title from ChatGPT can be a lead to search for, never a citation. If the search finds nothing, assume it does not exist.

Second, separate the model's text from the source's text. When you ask for summaries, give it the source and ask it to quote the sentences it relied on, as in the extraction prompt above. Then spot-check those quotes against the paper. Paraphrase without a quote is where small distortions slip in, such as a finding stated more strongly than the authors did.

Third, keep a record. Save your prompts, the dates you ran them, and which outputs you used and how. UNESCO's guide notes that whether an AI can count as an author is still debated, and that some publishers argue a model cannot take responsibility for a paper. A short log makes it easy to describe your use of AI accurately in a methods section, and to show that every source in your bibliography came from you, not from the model.

Sources

  1. A Beginner's Guide to ChatGPT — Holistic Hybrid, April 18, 2023
  2. Lawyers who cited fake cases invented by ChatGPT must pay — The Register, June 22, 2023
  3. ChatGPT and Artificial Intelligence in higher education: Quick Start guide — UNESCO IESALC

Get the free Guide to Artificial Intelligence

Sign up for the weekly newsletter and receive the free guide: what changed in AI this week, one lesson, one prompt to try. Unsubscribe in one click.

How we handle your address

Back to top

Display

Your choices are saved in this browser only. No data is sent.

Text size
100%
Colours
Reading
More settings
Accessibility statement