How do you use ChatGPT to test your own ideas?

Most of us are poor critics of our own plans. We know why we like an idea, and that makes it hard to see where it breaks. ChatGPT, the free chatbot OpenAI opened to the public on November 30, is surprisingly useful here, not because it knows better than you, but because it will argue any side you ask for, at length, without getting tired or offended.

The trick is to ask for disagreement deliberately. Left alone, a chatbot tends to take your framing as given and help you along. OpenAI notes that when a request is unclear, ChatGPT usually guesses what you meant rather than asking. If you write "here is my great idea, help me with it", you will get help, not scrutiny. You have to request the scrutiny.

The steelman-then-critique method

Two terms from debate are useful. A strawman is a weak version of an argument, set up so it is easy to knock down. A steelman is the opposite: the strongest, most charitable version of an argument, the one its best defender would make. Testing an idea well means first building its steelman and then attacking that, not a flimsy version.

ChatGPT can play both roles in one conversation. Asking it to make the best case first has two benefits. You see whether the idea can be stated clearly at all, and the critique that follows has to deal with the idea's real strengths instead of its easiest weaknesses.

A worked example: a mobile bike-repair van

Suppose you are thinking of running a bike-repair service from a van that visits customers at home in a town of about 10,000 people. Here is a prompt that sets up the whole exercise:

I want to test a business idea. Please do this in three parts.

Idea: a mobile bike-repair service run from a van. I visit customers at home or at work in a town of about 10,000 people. Tune-ups, flat repairs and brake adjustments, booked online.

Part 1. Steelman: write the strongest honest case for this idea in five bullet points.
Part 2. Critique: now act as a skeptical small-business advisor. Give the five most serious problems with the case you just made. For each one, say what fact or test would show whether the problem is real.
Part 3. List three questions I should answer before spending any money.

The phrase "what fact or test would show whether the problem is real" is the most important line. It forces each objection to be concrete. An illustrative reply might include points like these:

None of these answers is proof. They are a checklist of things to find out. That is exactly what you want from this exercise.

How to judge the answers

ChatGPT writes fluent text in a confident tone whether or not it is right. OpenAI's own launch post warns that it sometimes gives answers that sound plausible but are wrong or make no sense. Early testers, as The Verge reported, found it presenting invented details as fact. So treat every objection as a hypothesis, and sort them into three piles:

• Checkable and relevant. You can confirm or rule it out with a phone call, a count or a small test. Keep these; they are the real value.
• Generic. "Competition may be strong" or "marketing could be expensive" apply to every business ever started. Ask a follow-up: "Make objection 3 specific to a town of 10,000 people."
• Factual claims you cannot verify. If it states a statistic about the cycling market or names a regulation, do not rely on it. Its knowledge of events after 2021 is limited and it shows no sources, so check such claims elsewhere.

It also helps to rerun the critique. OpenAI says the model is sensitive to small changes in wording and that repeated attempts can produce different answers. Ask for a new response once or twice. Objections that appear every time are worth more attention than ones that show up once.

How do you ask for the strongest counterargument?

Be explicit about strength, role and format. "What are the downsides?" invites a gentle, balanced list. Instead, name who is arguing and how hard: "Argue against this as a sharp critic who wants to persuade me to drop it. Give only the single strongest argument, in one paragraph, and do not soften it." Asking for one argument rather than five pushes the model to pick its best point instead of padding the list.

You can raise the bar further by asking it to anticipate your defense: "State the counterargument, then my most likely reply, then why that reply is not enough." This gives you a short debate on the page and shows whether your favorite rebuttal actually holds.

Finally, push back when an answer is weak. ChatGPT is designed to work in a dialogue, so you can say "That objection is generic. Give a stronger one that a person with ten years in this trade would raise." If it agrees too quickly when you dispute a point, ask it to argue the other side of that point again. The goal is not to win the argument with a chatbot, but to leave the conversation with a short list of questions only the real world can answer.