AI Homework
Comment of the Day

December 08 2022

Commentary by Eoin Treacy

AI Homework

This article by Ben Thompson for his Stratechery letter may be of interest to subscribers. Here is a section:

The difference is that ChatGPT is not actually running python and determining the first 10 prime numbers deterministically: every answer is a probabilistic result gleaned from the corpus of Internet data that makes up GPT-3; in other words, ChatGPT comes up with its best guess as to the result in 10 seconds, and that guess is so likely to be right that it feels like it is an actual computer executing the code in question.

This raises fascinating philosophical questions about the nature of knowledge; you can also simply ask ChatGPT for the first 10 prime numbers:

Those weren’t calculated, they were simply known; they were known, though, because they were written down somewhere on the Internet. In contrast, notice how ChatGPT messes up the far simpler equation I mentioned above:

For what it’s worth, I had to work a little harder to make ChatGPT fail at math: the base GPT-3 model gets basic three digit addition wrong most of the time, while ChatGPT does much better. Still, this obviously isn’t a calculator: it’s a pattern matcher — and sometimes the pattern gets screwy. The skill here is in catching it when it gets it wrong, whether that be with basic math or with basic political theory.

Eoin Treacy's view

The probabilistic versus deterministic issue boils down to the problem all models face; quality of data. The best way for a chatbot to function is to only use a confined sample of reliable answers. In other words for an answer that is sure to be correct, the dataset has to be filled with deterministic infallible answers.

Click HERE to subscribe to Fuller Treacy Money Back to top