Tags
Here's my thought on 💭 What ChatGPT is NOT - Tech Raven Blog
Do you remember regression models from college: given some data, you find a best fit line that allows you to predict Y given X. At the end of the day, ChatGPT, and LLMs in general, are the same thing as the regression model – it’s just that ChatGPT is the largest and fanciest model we currently have to model language and information.
I really am coming to the idea of calling it a "word calculator", this seems to be the most succinct description of llms that the lay person can comprehend and relate to.
ChatGPT does not hallucinate or become unhinged
I think Steve goes much deeper on this in his intervew on fafo.fm. They describe it more as a pleaser or "yes man" essentially all the companies that are building these models want to give the "best" answer, better than their competitors. With this comes the risk of it being completely wrong, they are designed to always give an answer.
On that episode they talk about how each word has a statistical probability tied to it and in some visualizations you can see the confidence, in almost every example of "hallucination" you will find some set of very poor confidence followed by BS. Thats not to say that every time there is low confidence there is BS though, that is why this is such a hard problem.
ChatGPT is not a replacement for search
I've become really bad at this. This is slowly becoming my starting point for a lot of research, not my only research, but the first wet finger in the air. Most often done with low effort, low risk tasks.
In ChatGPT, just like in the shelf full of the Encyclopedia Britannica I had when I was a kid, information on topics has been reduced to good summaries and presented in logical format. In this case we have removed the human authors and replaced them with a very large neural network.
I really like this description and it matches up with my last comment.
Note
This post is a thought. It's a short note that I make about someone else's content online. Learn more about the process thoughts
This post was a thought by Waylon Walker see all my thoughts at https://waylonwalker.com/thoughts