Even amazing writers and musicians can barely scrape a living. Being an artist is mostly unpaid labor that enriches Irish culture. Why wouldn’t we incentivize those capable of being artists for some pitiful sum of money if they’re willing?
No matter how much capitalists pretend that artistic talent falls from the sky like mana for the rest of us to enjoy, it isn’t actually free. It takes thousands of hours of effort to become something resembling an artist. And this program is a cheap way to make that happen.
A link to the paper itself, if like me you have a math background, and are wondering WTF that means and how you measure creativity mathematically. Or for that matter what amateur-tier creativity is. Unfortunately, it’s probably too new to pirate, if you don’t have a subscription to the Journal of Creative Behaviour.
At least according to the article, he argues that novelty and correctness are opposite each other in an LLM, which tracks. The nice round numbers used to describe that feel like bullshit, though. If you’re metric boils down to a few bits don’t try and pad it by converting to reals.
That’s not even the real kicker, though; the two are anticorrelated in humans as well. Generations of people have remarked at how the most creative people tend to be odd or straight-up mentally ill, and contemporary psychology has captured that connection statistically in the form of “impulsive unconventionality”. If it’s asserted without evidence that it’s not so in “professional” creative humans, than that amounts to just making stuff up.
If we increase an LLM’s predictive utility it becomes less interesting, but if we make it more interesting it becomes nonsensical (since it can less accurately predict typical human outputs).
Humans, however, can be interesting without resorting to randomness, because they have subjectivity, which grants them a unique perspective that artists simply attempt (and often fail) to capture.
Anyways, however we eventually create an artificial mind, it will not be with a large language model; by now, that much is certain.
Ah, but if there’s no random element to a human cognition, it should produce the exact same output time and time again. What is not random is deterministic.
Biologically, there’s an element of randomness to neurons firing. If they fire too randomly, that’s a seizure. If they don’t ever fire spontaneously, you’re in a coma. How they produce ideas is nowhere close to being understood, but there’s going to be an element of an ordered pattern of firing spontaneously emerging. You can see a bit of that with imaging, even.
Anyways, however we eventually create an artificial mind, it will not be with a large language model; by now, that much is certain.
It does seem to be dead-ending as a technology, although the definition of “mind” is, as ever, very slippery.
The big AI/AGI research trend is “neuro-symbolic reasoning”, which is a fancy way of saying embedding a neural net deep in a normal algorithm that can be usefully controlled.
I didn’t say there’s no randomness in human cognition. I said that the originality of human ideas is not a matter of randomized thinking.
Randomness is everywhere. But it’s not the “randomness” of an artist’s thought process that accounts for the originality of their creative output (and is detrimental to it).
On actual mental illness specifically, as opposed to just “weirdness” in general, I have no hard data. If it’s caused at the physiological level, it makes sense that it wouldn’t follow the same pattern. You can of course name a bunch of mentally ill but prominent thinkers and artists from the past, but there’s almost certainly a lot of neglect of base rate going on there.
It’s worth noting production LLMs choose randomly from a significant range of tokens they deem fairly likely, as opposed to choosing the most likely one every time. If they were too conservative with it, they too would fall on the near side of that curve.
My point is that “weirdness” is rooted in subjectivity. Since LLMs have no subjectivity, they’re forced to rely on randomness, monkey-with-a-typewriter style, which is why their outputs are either banal or nonsensical.
Even amazing writers and musicians can barely scrape a living. Being an artist is mostly unpaid labor that enriches Irish culture. Why wouldn’t we incentivize those capable of being artists for some pitiful sum of money if they’re willing?
No matter how much capitalists pretend that artistic talent falls from the sky like mana for the rest of us to enjoy, it isn’t actually free. It takes thousands of hours of effort to become something resembling an artist. And this program is a cheap way to make that happen.
Or a few minutes and a neural net.
This is going to make people furious but it’s kind of true, and might actually be part of the argument for the policy.
LLMs are mathematically limited to amateur skill ceiling in creativity. Additionally, they’re fundamentally combinatorial and incapable of originality. This is why we are yet to see a single page of LLM fictional prose that doesn’t suck balls.
A link to the paper itself, if like me you have a math background, and are wondering WTF that means and how you measure creativity mathematically. Or for that matter what amateur-tier creativity is. Unfortunately, it’s probably too new to pirate, if you don’t have a subscription to the Journal of Creative Behaviour.
At least according to the article, he argues that novelty and correctness are opposite each other in an LLM, which tracks. The nice round numbers used to describe that feel like bullshit, though. If you’re metric boils down to a few bits don’t try and pad it by converting to reals.
That’s not even the real kicker, though; the two are anticorrelated in humans as well. Generations of people have remarked at how the most creative people tend to be odd or straight-up mentally ill, and contemporary psychology has captured that connection statistically in the form of “impulsive unconventionality”. If it’s asserted without evidence that it’s not so in “professional” creative humans, than that amounts to just making stuff up.
If we increase an LLM’s predictive utility it becomes less interesting, but if we make it more interesting it becomes nonsensical (since it can less accurately predict typical human outputs).
Humans, however, can be interesting without resorting to randomness, because they have subjectivity, which grants them a unique perspective that artists simply attempt (and often fail) to capture.
Anyways, however we eventually create an artificial mind, it will not be with a large language model; by now, that much is certain.
Ah, but if there’s no random element to a human cognition, it should produce the exact same output time and time again. What is not random is deterministic.
Biologically, there’s an element of randomness to neurons firing. If they fire too randomly, that’s a seizure. If they don’t ever fire spontaneously, you’re in a coma. How they produce ideas is nowhere close to being understood, but there’s going to be an element of an ordered pattern of firing spontaneously emerging. You can see a bit of that with imaging, even.
It does seem to be dead-ending as a technology, although the definition of “mind” is, as ever, very slippery.
The big AI/AGI research trend is “neuro-symbolic reasoning”, which is a fancy way of saying embedding a neural net deep in a normal algorithm that can be usefully controlled.
I didn’t say there’s no randomness in human cognition. I said that the originality of human ideas is not a matter of randomized thinking.
Randomness is everywhere. But it’s not the “randomness” of an artist’s thought process that accounts for the originality of their creative output (and is detrimental to it).
For LLMs, the opposite is true.
So, when it comes to mental illness and creativity, despite some empirical correlations, “There is now growing evidence for the opposite association.”
However, there are inverse-U-shaped relationships between several mental characteristics and creativity:
Although you’ll notice that disinhibition rapidly becomes detrimental.
On actual mental illness specifically, as opposed to just “weirdness” in general, I have no hard data. If it’s caused at the physiological level, it makes sense that it wouldn’t follow the same pattern. You can of course name a bunch of mentally ill but prominent thinkers and artists from the past, but there’s almost certainly a lot of neglect of base rate going on there.
It’s worth noting production LLMs choose randomly from a significant range of tokens they deem fairly likely, as opposed to choosing the most likely one every time. If they were too conservative with it, they too would fall on the near side of that curve.
My point is that “weirdness” is rooted in subjectivity. Since LLMs have no subjectivity, they’re forced to rely on randomness, monkey-with-a-typewriter style, which is why their outputs are either banal or nonsensical.