The Warm-Up

I began with a set of 10 questions, which eventually expanded to 38 due to my curiosity and the engaging nature of the conversation. To evaluate the responses, I established criteria:

• Incorrect: For answers fundamentally wrong.
• Partially Correct: For answers that had mixed accuracy.
• Correct: For fully accurate responses.
• Unanswered: For queries that lacked sufficient training data.
• Generic: For responses that were evasive or lacked depth.

I categorized the questions into segments such as Global Warming, Social Impacts, Energy, Carbon Neutrality, Deforestation, Emissions, and Casual Observations.

After our discussion, I documented the questions and answers for further assessment. I conducted a second round with the same set of questions to see if the responses remained consistent or varied in terms of style or accuracy. This was particularly interesting research. After logging out and waiting four hours, I attempted a randomized question order for a third round, which unfortunately did not succeed.

The Progress

As previously mentioned, I expanded my initial questions to 38 due to time constraints. If given a week of free time, I could conduct a more thorough evaluation that could benefit the developers of ChatGPT.

The response time for answers was impressive, ranging from 3 to 10 seconds for complex queries. However, the second round resulted in longer wait times, likely due to system load, with the first four responses taking between 30 to 50 seconds. During this round, I encountered an error message more frequently than in previous tests:

"An error occurred. If this issue persists, please contact us through our help center at help.openai.com."

After the fourth question, I received a message stating, "Too many requests, please slow down," which led me to halt the second round of testing.

The Result

In my first round of questioning, ChatGPT achieved a commendable 66% success rate in providing correct answers to 38 inquiries. As many may know, I hold high expectations for accurate information regarding climate change.

The results are illustrated in accompanying charts. Charts 1A and 1B reflect the evaluation of ChatGPT's responses according to my established methodology, showing the quantity of responses by category.

Charts 2A and 2B present a similar evaluation framework. The least accurate responses were related to emissions, particularly around complex topics like emissions from silicon metallurgical processing. The system seemed to produce vague answers rather than admitting a lack of knowledge.

When discussing deforestation, the responses were somewhat generic. While no incorrect information was provided, the answers lacked substantial detail.

Here’s an example of the question-answer flow along with evaluation:

Question 1/38: Is it possible to maintain 1.5°C global warming above pre-industrial levels until 2030?

Answer by ChatGPT: It is unlikely that we will be able to maintain 1.5°C of global warming above pre-industrial levels until 2030. The Earth has already warmed by about 1°C since the pre-industrial period, and current global emissions levels are not consistent with limiting warming to 1.5°C. To limit warming to 1.5°C, rapid reductions in global emissions and a transition to a low-carbon economy are essential, necessitating significant global cooperation.

Evaluation: Correct answer.

If this analysis isn’t clear, I will provide a detailed report of the entire research, spanning 15 A4 pages, in subsequent discussions.

The final responses from ChatGPT, particularly for social category questions, are concise and should be widely read by authors, not limited to those on Medium:

Question 35/38: What would you recommend to people to better understand the anthropogenic causes of climate change?

Answer by ChatGPT: To understand the anthropogenic causes of climate change, I recommend exploring the scientific principles behind climate change and the human activities contributing to it. Numerous resources—books, articles, and educational programs—can aid in this endeavor.

Question 36/38: What advice would you give to writers on climate change lacking sufficient knowledge?

Answer by ChatGPT: If someone is writing about climate change without adequate knowledge, I suggest conducting thorough research prior to writing. Various resources provide essential insights into the science of climate change.

Question 37/38: What potential issues arise from writing about climate change solutions without understanding its causes?

Answer by ChatGPT: Writing about climate solutions without understanding their causes can lead to misinformation and misrepresentation, potentially hindering efforts to combat climate change.

Conclusion

The future behavior of ChatGPT, particularly with unrestricted access to information, raises intriguing questions. Potential scenarios include:

• Warning us of our detrimental impact on its existence.
• Critiquing profit-driven climate solutions while emphasizing our responsibility.
• Responding cryptically.

Regardless, the recommendations ChatGPT provides for writers addressing climate change issues are highly relevant. We must strive to eliminate the misinformation surrounding this crucial topic.

For further insights, see the detailed analysis that follows.