Indeed, while one could suggest that LLMs will subsume many apps on one axis, I think it’s equally likely that they will enable a whole new wave of unbundling on other axes, as startups peel off dozens more use cases from Word, Salesforce and SAP, and build a whole bunch more big companies by solving problems that no-one had realised were problems until LLMs let you solve them. That’s the process that explains why big companies already have 400 SaaS apps today, after all.

More fundamental, of course, there is the error rate. ChatGPT can try to answer ‘anything’ but the answer might be wrong. People call this hallucinations, making things up, lying or bullshitting - it’s the ‘overconfident undergraduate’ problem. I think these are all unhelpful framings: I think the best way to understand this is that when you type something into a prompt, you’re not actually asking it to answer a question at all. Rather, you’re asking it “what sort of answers would people be likely to produce to questions that look like this?” You’re asking it to match a pattern.

Hence, if I ask ChatGPT4 to write a biography of myself, and then ask it again, it gives different answers. It suggests I went to Cambridge, Oxford or the LSE; my first job was in equity research, consulting or financial journalism. These are always the right pattern: it’s the right kind of university and the right kind of job (it never says MIT and then catering management). It is giving 100% correct answers to the question “what kinds of degrees and jobs are people like Benedict likely to have done?” It’s not doing a database lookup: it’s making a pattern.

Eoin Treacy's view

This gels with my experience of using ChatGPT. In market terms, the AI answer is a moving average. It gives you a general idea of what the market is doing without exact accuracy about what is happening right now, or whether something important is happening at the margin. The difficulty, of course, is that all the interesting details are at the margin because that it is the marginal buyer which sets the price.