Here’s another one of those internet things like the blue/gold dress photo. [stderr] And it appears that neuroscientists have a theory for how it works.
Before we go any further, let’s just do an unscientific test. Watch this, and note to yourself what you experience.
OK, now: some people report that they hear a sound at the final frame(s) when the pylon “lands” and there is the “vibration” from its impact. Do you?
My experience is that I feel like I expect a sound and it doesn’t happen. What I would normally say is something like that my past experiences have taught me that an impact that would create a shockwave like that is also going to make a sound, so my brain is waiting for the sound and – when there is none – it registers as unusual. What I am experiencing is disconfirmation of a learned behavior. I’ll go a step further and wonder out loud if one reason my brain is relatively OK with “shockwave + no sound” is from watching a lot of military footage – I’ve seen plenty of ground shockwaves without sound so maybe I’m not too surprised to see another one.
Scientists have another theory [medex] which appears to be a different way of saying what I think I said.
A synaesthesia-like effect in which people ‘hear’ silent flashes or movement, such as in popular ‘noisy GIFs’ and memes, could be due to a reduction of inhibition of signals that travel between visual and auditory areas of the brain, according to a new study led by researchers at City, University of London.
The study is the first to provide insight into the brain mechanisms underpinning such auditory sensations also known as a ‘visually-evoked auditory response’ (aka vEAR or ‘visual ear’).
Is “reduction of inhibition of signals” a fancy way of saying “learning”?
It was also found that musicians taking part in the study were significantly more likely to report experiencing visual ear than non-musician participants. This could be because musical training may promote joint attention to both the sound of music and the sight of the coordinated movements of the conductor or other musicians.
This makes me wonder whether a trained orchestral musician would experience a sort of shock, if they saw the conductor moving inappropriately for the sounds and reactions of the rest of the orchestra. Imagine if you had footage of a conductor who begins ‘conducting’ (whatever that is) and the musicians in the orchestra just sit there immobile, but there is music. I’d expect that some of us wouldn’t even notice whereas others might find the image to be freaky since it ran contrary to their lived experience. I had similar experiences with some computer games, where the in-game physics was subtly wrong and my brain would go into shrieking warnings because gravity was apparently broken.
“We found that people with ‘visual ears’ can use both senses together to see and also ‘hear’ silent motion, while for others hearing is inhibited when watching such visual sequences.”
Some neuroscientists believe visual-ear may be a type of synaesthesia, with other examples including music, letters or numbers that can evoke perceptions of colour. However, visual ear appears to be the most prevalent, with as many as 20% of people reporting some experiences of it compared to 4.4 per cent for other types.
This study seems to be pretty cool:
To shed light on what may be going on in the brain when people view such content, the researchers applied a weak alternating current to participants’ scalps, using a technique called transcranial Alternating Current Stimulation (tACS), to explore how the visual and auditory parts of the brain interact in those who experience visual ear and those who don’t.
The first experiment of the study included 36 healthy participants, including 16 classical musicians from the London Royal College of Music. All were shown auditory and visual ‘Morse code’ sequences, while electrical simulation (tACS) was applied to either the back of the head (visual areas of the brain) or the sides (auditory areas) using ‘alpha-frequency’ tACS stimulation. Participants were then classified as visual or non-visual ear depending on whether they reported ‘hearing’ the silent flashes.
The researchers found that in non-visual ear participants, alpha-frequency stimulation to auditory areas significantly reduced auditory performance but improved visual performance, while the opposite pattern was found for the same frequency of stimulation to visual areas (poorer vision, better audition).
Almost half of their study participants were classical musicians from the LRCM; I suspect there is a bunch of strongly learned behaviors, there.
“We were also interested to find that, on average, participants with visual ear performed better on both visual and auditory tasks than those without. Perhaps their audio-visual cooperation benefits performance because more of the brain is engaged in processing visual stimuli.
“Such cooperation might also benefit musical performance, explaining why so many of the musicians we tested reported experiencing visual ear.”
Next up: can a classically trained oboist be more easily trained to drool at the sound of a glockenspiel?
Note: I did not read the original study so we should assume standard journalism reporting errors. I.e.: the study was actually about nut-counting behavior in naked mole rats.
For me that dress looks blue and brown. The blue looks like Cerulean blue red shade pigment. Brown parts of the dress are trickier to describe—dirty, grayish, olive brown with a little bit of golden metallic shine to it. But I wouldn’t call this color “gold,” because it’s too dark and not shiny enough.
Nope, I neither hear nor expect a sound. I know that gif images don’t make sounds, so why should I expect to hear anything? I don’t feel a vibration either. When the pylon “lands,” I don’t perceive that as a shockwave either, I only see a misaligned frame in the gif file. I have made some animations myself, so I only see a poorly aligned frame, I don’t even perceive it as a shockwave. I assume that this might be a learned behavior—my experience making animations has resulted in me seeing misaligned frames where non artists see shockwaves.
I did not hear a sound, nor feel a vibration. But then I realized that the landing was synchronized to my heartbeat. (When I sit still, I’m usually pretty much aware of my heartbeat.)
Except . . . no it wasn’t. It was over-riding my normal sense of my own heartbeat. (I checked my pulse on my wrist.) I think one thing that led to this is that its frequency actually was quite close to my normal heart rate, and that allowed the effect to happen.
Does anyone else hear the whistling sound of the wires?
I pretty persistently think I feel a vibration. Unless of course I scroll down and can’t see it anymore. I asked the Partner who reports feeling nothing and hearing nothing, but nevertheless is a professional musician. (In fact, the band is playing a gig tonight.) I would not describe myself as a musician but I can play several instruments – just not very well.
And those naked mole rats sure are cute, aren’t they?
I wasn’t expecting to hear a sound because I always keep my devices on mute, but I did “feel” the vibration. Kinda weird, really.
Now, can this be applied to the tree in the forest that falls when no-one’s around?
Musician here:
I “hear” a clear sound (I can tell it’s in my mind, not my ears, but it sounds like a very clear “thunk, thunk, thunk”) and similarly a sense of a vibration in my chest from the shockwave. It’s not synchronized to my heartbeat.
I wouldn’t be surprised if it’s very related to synesthaesia, I visualize notes as colours as well.
If I remember correctly Oliver Sacks wrote some stuff about synesthaesia as well, end the benefits it provided. I seem to remember reading something about using the visual cortex to aid in auditory processing and vice-versa.
I can hear a little thump, with the landing/shaking in the image. I’ve been playing and writing music since early childhood. I’m “classically trained” and all that. (Not to be confused with only doing “classical music.”)
In real life, that pylon would be a large distance away (1/2 mile?), so there would be more of a delay before it reaches my ears (but maybe not much, I don’t know). Also, it’d probably produce a different set of frequencies, but presumably what I hear is just whatever corresponds to the shaking in the image. And it’d be much louder than the hallucination, which is so soft I might have assumed (if I didn’t know it was illusory) that it was some other noise from my apartment or outside.
Yes, definitely. It’s such a common experience that “shocking” isn’t how I would want to put it. But if a movie has a non-musician playing the role of a musician, there’s a very good chance of continuity errors like that. It’s usually not too unnerving, but it happens a lot…. sort of an uncanny valley effect, I guess.
Sometimes it’s very subtle, and I wouldn’t notice it, unless for example I repeated the same short clip a few times (which obviously interrupts the movie). On the other end of the spectrum, there are also scenes like the cantina on Tatooine in A New Hope. It’s just sort of puzzling, because I want to have a sense of how the alien instruments function, but I can’t map the sounds to the motion of the puppets.
But where are you going to find oboists who aren’t drooling all the time?
There’s also this, which might interest some folks:
Can You Recover Sound From Images? (11:22). Veritasium dude has a “gee whiz” attitude about everything, but I have to admit it is impressive that they can get any sound at all. (Not very well, and only with a lot of work, but it’s still something….)
I didn’t hear a sound, but I thought a “BOOM” with each landing. I could fairly easily mistake it for a sound. The misaligned frames made me slightly nauseous after I watched for longer than a couple of seconds.
I don’t know how it correlated, but I’m musically inept. I have zero natural aptitude and next to no training. If researchers were looking for the opposite of a classically trained musician, that would be me.
The wires are being spun in a counter-clockwise direction.
.
.
I neither heard nor felt anything.
I do not hear a sound, but my mind creates a false impact vibration. I know it’s not real, it’s a “mind’s eye”/imagination deal.
I… might be the odd one out. I definitely hear a thump watching that gif.
Of course it thumps, every kitty knows that.
Reminds me of something that’s been bothering me with movies and such recently.
If you are far enough away from an explosion to be safe, there is a very distinct and noticeable lag twixt the flash and the boom, much like lightning and thunder.
Anyways, I can sorta imagine a “thump”, but then that structure isn’t massive enough to make much of a thump at that distance, and then those other horizontal-looking lines should be entangling the jump rope lines.
I begin to sort-of hear a muted, dull thump after I watch the gif for about 20 loops. I don’t actually hear it, more like I can’t help but imagine the sound. I do not feel a vibration or a shockwave.
Hmm, how to put this…. For me, it isn’t the kind of phenomenon Marcus described here: “My experience is that I feel like I expect a sound and it doesn’t happen.” It’s an auditory experience. It does happen, for somebody like me and not for somebody like Marcus (at least in this particular case).
Suppose you look at a painting, and you read that its title is “Apple Orchard.” You see some trees and the sky and grass and whatnot. And maybe you expect to find a visible apple somewhere in the picture — maybe you think the painter would be kind of stupid to leave that out of such a picture. But what happens is that you don’t find one. No apples. You remember what apples look like, you can mentally visualize or imagine them, and based off this, you can tell that you see no apples in the picture. And so you shrug at the weirdly-titled painting with no apples in it and move on to the next one in the gallery.
Now consider a different type of situation. Suppose you hallucinated an apple. I don’t care how or why — you took some drugs or whatever. So it’s just like you saw an actual apple. But it isn’t an actual apple which exists in the external world. You didn’t have to convince yourself that you saw it, nor did you have to work your way into it somehow, by telling yourself a story about why you should’ve seen an apple like you’ve seen in other situations. It’s just something you did see, which isn’t actually there.
I’m now blinking in time with the gif.
~bats at the jumping tower~
I don’t hear anything either, my experience is much as you describe.
“Is “reduction of inhibition of signals” a fancy way of saying “learning”?”
The short answer to this question is “No”. The cortex is full of inhibitory neurons that, when they fire, reduce the chance of the neurons they are connected to firing. A “reduction in inhibition” could be due to lots of factors: these inhibitory neurons might be less active temporarily due to some kind of external input (like a trans-cranial magnetic stimulation pulse, “TMS”) or because their activity has itself been inhibited by other inhibitory neurons, or the presence/absence of a neurotransmitter of one kind of another, or because in some person’s brain there might simply be less of these neurons.
The longer answer is “No, there is no reason why reduced inhibition needs to be learned. BUT it could be.” There’s an ongoing debate in the synaesthesia literature about, essentially, the extent to which genetics, epigenetic factors, and childhood learning produce and influence these experiences. So whatever the cause of reduced inhibition, unless it’s due to TMS or some other experimenter-controlled factors, there’s the question of what its cause is, and that is a very thorny question.
The answer to a different question is that “reduced inhibition” is one theory among several in the synaesthesia literature about what the proximal cause of synaesthesia is. And each of the alternate possibilities is subject to similar complications.
I write all this with a high degree of confidence, because I spent several years working and publishing on (different aspects of) synaesthesia and learning.
@ 1 Ieva Skrebele
I think I getting the same reactions that you are.
This may also be related to police shootings. They are trained to expect weapons, and when they shoot unarmed people with phones or wallets their brains may actually be registering guns in their vision.
Callinectes@#21:
This may also be related to police shootings. They are trained to expect weapons, and when they shoot unarmed people with phones or wallets their brains may actually be registering guns in their vision.
That’s disturbingly plausible. Considering that many cops are taught to see aggression and to always be ready for it, are they priming their brains to see threat where it’s not?
Probably a good social science paper in there – show cops and non-cops, and people who have just joined the force but not been indoctrinated, videos of certain situations and give them a button to push when they think that they should draw a weapon…