It can’t not hallucinate. It’s just predicting (not even selecting) next tokens. It doesn’t know what it knows and what it doesn’t know. It can’t introspect. It just gives probabilities for all possible tokens in it’s vocabulary based on the context window and the inference engine selects the next one (based on it’s settings). Without having the correct answer in the context window it can just make a prediction based on it’s (fixed) neutral net parameters and these are severely limited, even for big models. What I mean is, they basically “learned” the whole Internet and compressed the whole thing into some hundred billion or a few trillion parameters. That’s an insane compression ratio. This compression is lossy. For niece information and the results are similar to the “unimportant” details in highly compressed JPGs, you can make out the general image but fine details are just a mush. The LLM itself doesn’t know this, it just gives wrong predictions.
For what it does I think the result is extremely impressive but the way it works is severely limited.
From what I understand from what it told me about itself, it can be wrangled further. That is what the paid for versions are. The info can be sandboxed and then other agents verify the correctness of the info from very specific, known to be solid, sources. This is very expensive and still not fool proof. Am I wrong in thinking this bubble is going to pop hard?
It’s definitely going to pop hard, because those “verifying agents” are just more models computing correlation with sources, not actually verifying anything.
It can’t not hallucinate. It’s just predicting (not even selecting) next tokens. It doesn’t know what it knows and what it doesn’t know. It can’t introspect. It just gives probabilities for all possible tokens in it’s vocabulary based on the context window and the inference engine selects the next one (based on it’s settings). Without having the correct answer in the context window it can just make a prediction based on it’s (fixed) neutral net parameters and these are severely limited, even for big models. What I mean is, they basically “learned” the whole Internet and compressed the whole thing into some hundred billion or a few trillion parameters. That’s an insane compression ratio. This compression is lossy. For niece information and the results are similar to the “unimportant” details in highly compressed JPGs, you can make out the general image but fine details are just a mush. The LLM itself doesn’t know this, it just gives wrong predictions.
For what it does I think the result is extremely impressive but the way it works is severely limited.
From what I understand from what it told me about itself, it can be wrangled further. That is what the paid for versions are. The info can be sandboxed and then other agents verify the correctness of the info from very specific, known to be solid, sources. This is very expensive and still not fool proof. Am I wrong in thinking this bubble is going to pop hard?
It’s definitely going to pop hard, because those “verifying agents” are just more models computing correlation with sources, not actually verifying anything.