I once built a thinking machine out of dominos. Mine added two bits together. Matt Parker’s was way bigger, and could do 8 bits. Children have made thinking machines in Minecraft out of redstone. Thinking machines aren’t very hard.
My calculator can extrapolate 5 when I give it 2, 3, and a plus sign. So can an LLM. My calculator uses some adder circuits in its ALU to get the 5. The LLM gets it from memorising the next likely token, the same way your brain works most of the time. Your brain’s a lot more advanced, though, and can find the 5 in many different ways. Likely tokens are just the most convenient. Cognitive scientists call that “System 1”, though you might know it as “fast brain”. LLMs only have system 1. They don’t have system 2, the slow brain. Your system 2 can slow down and logic out the answer. If I ask you to solve the problem in binary, like My calculator does, you probably have to use system 2.
The question you should be asking is: does system 1 experience qualia? And based on split brain studies in participants who have undergone corpus callosumectomy, I believe the answer is yes. Of course, the right brain isn’t the same thing as system 1, but what these studies demonstrate is that there are thinking parts of your brain that you can’t hear. So I’d errr on the side of caution with these system 1 machines.
I once built a thinking machine out of dominos. Mine added two bits together. Matt Parker’s was way bigger, and could do 8 bits. Children have made thinking machines in Minecraft out of redstone. Thinking machines aren’t very hard.
What do you consider thinking, and why do you consider LLMs to have this capability?
Extrapolating from information.
My calculator can extrapolate 5 when I give it 2, 3, and a plus sign. So can an LLM. My calculator uses some adder circuits in its ALU to get the 5. The LLM gets it from memorising the next likely token, the same way your brain works most of the time. Your brain’s a lot more advanced, though, and can find the 5 in many different ways. Likely tokens are just the most convenient. Cognitive scientists call that “System 1”, though you might know it as “fast brain”. LLMs only have system 1. They don’t have system 2, the slow brain. Your system 2 can slow down and logic out the answer. If I ask you to solve the problem in binary, like My calculator does, you probably have to use system 2.
The question you should be asking is: does system 1 experience qualia? And based on split brain studies in participants who have undergone corpus callosumectomy, I believe the answer is yes. Of course, the right brain isn’t the same thing as system 1, but what these studies demonstrate is that there are thinking parts of your brain that you can’t hear. So I’d errr on the side of caution with these system 1 machines.