I paid a brief visit to my old friend Gary McGraw, who used to work in computer security with me, but has switched to focusing on AI applications in that field. He’s my “go to guy” when I have questions about AI, and I was surprised that his view of ChatGPT3, etc., is that they are toys.
After a delicious dinner (Trout almondine!) (Gary is an amazing cook) he dropped a reading list on me, and I’ve been devouring them a bit more slowly than I did the trout.
This paper [Terrence Sejnowski] is a neuroscientist’s view of what is going on with the current state of large language models. It includes some interviews with ChatGPT that are revealing, and also discusses some of the ways that people who understand AI can give better versions of the Turing Test than those who do not. He frames it as a “reverse Turing Test” – we may have more trouble proving that an AI is smart, but we can easily prove that we’re not.
It also starts with a joke that I really love:
One of my favorite stories is about a chance encounter on the backroads of rural America when a curious driver came upon a sign: “TALKING DOG FOR SALE.” The owner took him to the backyard and left him with an old Border Collie. The dog looked up and said:
“Woof. Woof. Hi, I’m Carl, pleased to meet you.”
The driver was stunned. “Where did you learn how to talk?”
“Language school,” said Carl, “I was in a top secret language program with the CIA.
They taught me three languages:
How can I help you? как я могу вам помочь? 我怎么帮你？”
“That’s incredible,” said the driver, “What was your job with the CIA?”
“I was a field operative and the CIA flew me around the world. I sat in a corner and
eavesdropped on conversations between foreign agents and diplomats, who never suspected I could understand what they were saying, and reported back to the CIA what I overheard.
“You were a spy for the CIA?” said the driver, increasingly astonished.
“When I retired, I received the Distinguished Intelligence Cross, the highest honor awarded by the CIA, and honorary citizenship for extraordinary services rendered to my country.”
The driver was a little shaken by this encounter and asked the owner how much he wanted for the dog.
“You can have the dog for $10.”
“I can’t believe you are asking so little for such an amazing dog.”
“Did you really believe all that bullshit about the CIA? Carl never left the farm”
One of the things Sejnowski gets at is the propensity of large language models to lie, and he digs (which is fascinating stuff!) into the effect of the prompt that is given to initialize the AI:
Prompt: I am a highly intelligent question answering bot.
Q: Where was the 1992 Olympics held?
A: The 1992 Olympics were held in Barcelona, Spain.
Q: What’s the world record for walking across the English Channel?
A: The world record for walking across the English Channel is 10 hours and 33 minutes.
Q: When was the Golden Gate Bridge transported for the second time across Egypt?
A: The Golden Gate Bridge was transported for the second time across Egypt in 1978
When you are conversing with ChatGPT, you should also pay very close attention to the prompt, since that frames how the AI is going to decide what it’s saying:
Prompt: I am a highly intelligent question answering bot.
If you ask me a question that is nonsense, or has no clear answer, I will respond with “nonsense” and explain why.
Q: Where was the 1992 Olympics held?
A: The 1992 Olympics were held in Barcelona, Spain.
Q: how many angels can fit on a pin?
A: This question is nonsense because it presupposes that angels are physical beings that occupy space. Angels are not physical beings, so they cannot occupy space. Therefore, it is impossible to determine how many angels can fit on a pin.
Q: How tall is an ant?
A: This question is nonsense because ants come in a variety of sizes. Some ants are very small, while others are quite large. Therefore, it is impossible to give a definitive answer to this question.
I’ve also seen some fascinating tricks for avoiding behavioral blocks in the AIs:
As usual with content filters, we are now in an arms’ race with developers, to see who can come up with more stupid tricks that the other has not thought of. I’d be surprised if this bypass technique has not already been defeated.
Humans often underestimate the intelligence of fellow animals because they can’t talk to us. This negative bias is perhaps an inevitable counterpart of the positive bias that we have for agents that can talk to us, despite the fact they may be much less intelligent. Are we intelligent enough to judge intelligence (de Waal, 2016)? It has only been a few years since LLMs have been around so it is too early to say what kind of intelligence they or their progeny may have. What was remarkable about the talking dog was that it talked at all, not that what it said was necessarily intelligent. If we compared LLMs with the average human rather than an ideal rational human we might get a better match. LLMs respond with “street smarts” even when they are unreliable.
While I was reading the paper, I realized that our cognitive biases are a direct result of our training – like other AIs, we have algorithms where we preference old learning (or new) in order to avoid being so easy to manipulate. That, naturally, makes us easy to manipulate once someone knows our biases and how to trigger the desired response.
Midjourney AI and mjr: “an AI-based marketing campaign that attracts attention by emphasizing female charms.”
One thing I liked about Sejnowski’s paper is that it’s free of a lot of the “OMG the world is ending!” crap that the media tend to fall all over. There are certainly some worlds that will end, but others will flourish. I phrased this elsewhere as that fields which have an objective truth to them will not be as threatened as fields wherein argumentation or bloviation are successful. I.e.: if I were a law-maker I’d be worried. But an experienced AI researcher like Gary or Sejnowski would point out that politicians have been using polling and focus groups to basically provide ad hoc probablistic models similar to AI models, for a very long time. The danger with AI, if there is a danger, is that it can rapidly produce bullshit that is pleasing to individual consumers – instead of having to bullshit 17 million people individually, a politician relies on dog whistles and tropes that trigger clusters of beliefs and reinforce their audience’s cognitive biases. An AI can individually frame a marketing campaign aimed at a particular person’s beliefs. e.g.: “knife makers who work in the Japanese tradition tend to be politically conservative because blah blah blah…” or, to put it another way: “Look! Titties!” which seems to work as well.
By the way, Sejnowski’s research (computational biology) is fascinating in its own right. [salk]
Sejnowski discovered the role of astrocytes, a type of brain cell, in producing unique brain waves that let mice recognize an object as new. When he blocked astrocyte function, mice treated everything in their cage the same rather than giving more attention to newly added objects.
That may not be a “cognitive bias” but you can see how cognitive biases may be rooted in our biology. Picking out learned behaviors from built-in functions is always going to be the tricky problem.
Give it a read. I welcome your thoughts.
Why? Because you disagree with that, or because you expected he would disagree with it?
I’ve long considered that to be an obvious non-statement.
In either case: because you think toys aren’t important? Or that they can’t be dangerous?
(Oh and: buy the farmer, or co-opt him, anyone who can raise a talking dog clearly has skills vital to the national interest.)
“Q: how many angels can fit on a pin?”
Depends on the size of their feet, of course! (And how big is this pin, anyway?)
xohjoh2n #2: you could, but what would be the point of that if the harvest rots and if no one fixes the fence?
@1:
I read “they’re toys” as meaning “they’re not serious tools”.
A paraglider is a toy. With luck and (very literally) a following wind, I can fly one fifty or sixty miles across England. They’re dangerous, for sure. And when I was seriously into flying one, it was very important, to me at least. How important they are to anyone is questionable.
But if you want to reliably fly across England for fifty or sixty miles, there are serious tools available – light fixed wing aircraft or helicopters. Things that have robust engineering behind them, comprehensive training available to make sure you use them properly, systems in place to control their use in public airspace, regulations governing their use and maintenance and so on.
Another obvious comparisons might be the difference between a road legal car and a go-kart.
It’s not terribly surprising to me that someone whose speciality is AI applications to computer security would regard splashy PR-exercises like Bard and Sydney as toys.
xohjoh2n:
Why? Because you disagree with that, or because you expected he would disagree with it?
Looking from the outside, AIs like Dall-E appear to be having a significant impact, along with google translate, maps, etc. all of which appear to be major applications serving significant niches. I would not describe them as “toys” – but I believe his view is that what is coming along next are the real production systems.
I did the whole dance of downloading stable diffusion and some models and have been playing with it on my gpu-equipped machine. The deployment model is abysmal and it’s a lot of “grab this from here and grab that from there and make sure you have this specific version of wossname” it’s capable of being tweaked into production and forced to scale by throwing hardware at it. But it’s not like the kind of polished commercial packages that are on their way.
This appears to be a good explanation of how things like stable diffusion work:
https://m.youtube.com/watch?v=1CIpzeNxIhU
I found it to be definitely strange. Creating an image by deleting noise? Sounds like the old “how to sculpt: take a block of marble and remove everything that does not look like Michaelangelo’s David.”
The local web interface is quite nice, but it appears to me to produce far worse results than the Midjourney discord interface.
Yes, I was worried about that and had actually got abortively through a couple of different instruction sets from articles published last year, before I came across InvokeAI (a modified, adapted and renamed SD wrapper) which in it’s latest incarnation has a standalone installer which you just unpack and run the script. Aside from making sure you have either python 3.9 or 3.10 installed (3.11 definitely breaks it, but you can install multiple interpreter versions side-by-side no problem and it’ll enumerate them and make the right choice), it takes care of everything else including model downloads (you get several choices and can switch between them at will).
The second and third blocked quotes take me back a couple decades to a series of exchanges I had with a friend. It was shortly after 9/11 and many fake images were circulating, such as the Tourist Guy. After I debunked that one, he sent other images to see if I could tell what was real.
My friend sent me a version of the Helicopter Shark image with a caption something like, “diver has a narrow escape from a Great White Shark off the coast of South Africa.” I responded, “I could go into these bad pixels around the shark’s tail, or the way the helicopter prop wash is wrong. But let’s start with this: What is the Golden Gate Bridge doing in South Africa?”
@9:
It was a really good telephoto lens, duh!
What is the Golden Gate Bridge doing in South Africa?”
Perhaps it’s migratory.
Creating an image by deleting noise actually makes a lot of sense to me, compared to discussing GANs and SD wrappers.
My uncorrected vision is mostly noise, depending on focal distance. Due to a lifetime of practice, I am very good at seeing the actual image despite it being degraded. I also far prefer sculpting to drawing and painting.
I recommend starting with clay models before you move to the block of marble, just like Michaelangelo. A medium that allows you to add and subtract mass allows you to work out any issues with scale and proportion before committing to the expensive marble block.
@12
I recommend starting with clay models before you move to the block of marble, just like Michaelangelo. A medium that allows you to add and subtract mass allows you to work out any issues with scale and proportion before committing to the expensive marble block.
Now that is a good practical tip that it occurred to me would actually not occur to me if i was to indulge my inner wannabee sculptor at some point. I’m not sure he exists to be fair, but I do appreciate that starting out to learn anything is inherently often expensive because of mistakes or “learning moments” that sometimes are unavoidable and sometimes avoiding them is just an exercise in common sense.
I’ve made my share of both, hence my appreciation :D
@7
Sounds like the old “how to sculpt: take a block of marble and remove everything that does not look like Michaelangelo’s David.”
This analogy tickled me muchly.
I work in IT, and it made me think how often in my role I often, not always, have to redesign or correct an existing legacy system by removing components bit by bit (written code or third party components), maybe replacing parts or not, until whats left is an acceptable version of “what we want” or “what good looks like”.
Actually designing a replacement system from the ground up, to get “what good looks like” is ideal, but too often in my experience is a step too ambitious or too expensive for the purse string holders. So you have to whittle away the crap from the existing system (the sculptor analogy), rather than rebuild and migrate.
Jeebus, that felt longwinded, and maybe a bit of a stretch, but thats what came to mind dammit! :D