AUDITORY OBJECTS

MAI AND IMAGINE SCIENCE FILMS VISIT
THE DAVID POEPPEL LAB AT NEW YORK UNIVERSITY

PART ONE: The MEG Experiment

As I sit in the small office facing the magnetically-shielded test chamber, lab manager Jess Rowland straps me into a neck brace and begins calibrating a laser to scan the shape of my head. Today, I’m participating in a neuroscience experiment as part of our six-hour trip to one of New York University's labs, our first collaborative lab visit with Imagine Science Films. So far, I have no idea what the experiment is about.

With each pass of the laser wand, my features start to take shape on the screen in front of me until I can see a rotating model of my head floating on a blue background. 

Photograph by Siena Oristaglio

“What we should do is have you practice the actual experiment so you know what you’ll be doing while you’re inside the machine.”

Jess leads me into room full of monitors and equipment. In front of me is a keyboard and a microphone. Resting on a nearby computer is a toy plasma globe.

“We’re studying sentences today. You’re going to hear three sentences sequentially. The first will sound very distorted and weird, the second will be very clear, and the third will again sound distorted and weird. What I’d like you to do is judge the intelligibility of the sentences on a scale of one to four. One indicates that you don’t understand at all and four means the sentence was completely intelligible.”

“Okay, that seems easy,” I say. “What are you looking for in this experiment?”

Jess pauses with a wry smile.

“It’s good to be a little naïve about the scope and goal of the experiment going in. I can tell you about it afterwards but, generally, the idea is to learn about the way the brain processes language."

Jess plays the first test sentence. It sounds like an electronic duck, squawking from inside of a washing machine. I press the "1" key on the keyboard to indicate my complete lack of comprehension. Another sentence plays.

“Help the weak to preserve their strength.”

The voice is clear now, if oddly emotionless. I press "4." Then, the sentence repeats in garbled electro-duck mode but now the words are familiar.

 

Jess plays the first test sentence. It sounds like an electronic duck, squawking from inside of a washing machine. I press the "1" key on the keyboard to indicate my complete lack of comprehension.

 

“Hzzz zz weak tzz prezzrve thzz stzzzgth.”

I hit the "3" key. Intelligible-ish. Jess runs practice sentences until I seem to get the hang of it.

“Let’s get you ready to go in the scanner.”

We enter through a thick door into a small, white room with a bed. On one end is what looks like a large refrigerator with a toilet bowl in its middle, lit by a projected computer image on a pane of ground glass. The structure has ports for refilling the liquid helium that cools the machine. This is the scanner. 

Photograph by Noah Blumenson-Cook

The magnetoencephalogram, or MEG, detects extremely faint magnetic fields produced by the brain. These fields are so weak and the equipment so sensitive that the walls of the chamber contain huge electromagnets that work like a gigantic set of noise canceling headphones, actively canceling out the surrounding interference. Lab manager Jeff Walker begins carefully taping electrodes to specific points on my head.

“We’re going to put some tape on you to keep the electrodes still. Let me know if this gets uncomfortable in any way. They frown on us using staples.”

I laugh. With the electrodes in place, I put in earphones that block out all sounds except for the sentences and instructions for the experiment. As I lie on the bed, the projected image in front of me tracks my eye movement and pupil dilation.

For the next two hours, I interpret sentences and press buttons with my head immobilized in the massive machine. The exercises are broken up into half-hour segments, and at first, I attack my simple task with determination. 

 

After 50 sentences, time ceases to exist. There is only the clear voice and the electric duck. I start to realize that I’ve fallen into a pattern.

 

“Keep our cat in the neighbor’s yard.” 

“Kzzz ozz caz iz thz neigbrzz yzz.” 

Around 25 sentences in, I begin to realize that I’m slipping halfway into a dream state. When Jess interrupts the experiment for a brief break, I feel relieved. She tells me to take as much time as I feel I need to recuperate. I resist the urge to close my eyes and drift away.

“Coax a young calf to drink from a bucket.”

“Czzz a yzzng czzz to drizz frzzz a bzzzz.”

Video still by Colin Stringer

After 50 sentences, time ceases to exist. There is only the clear voice and the electric duck. I start to realize that I’ve fallen into a pattern, that the buttons I’m pushing may not be connected to conscious decisions. I begin to worry if the data I’m generating is of any use. 

Eventually, Jess chimes in again to tell me the experiment is complete. I emerge from the machine disoriented and unsteady, but feeling incredibly well-rested.

 “How was that?” Jeff asks as he peels the electrodes from my head.

“Intense.”

“You stayed awake. That was impressive.”

After I take a moment to assemble myself, I follow Jeff and Jess into a conference room to discuss what just took place.



JEFF WALKER: So how was your experience? Was it all you thought it could be and more?

 

NOAH BLUMENSON-COOK: It was like being in a 70s dystopian science fiction movie. Did you ever see Logan’s Run? Do you remember the last scene where Logan is being interrogated about Sanctuary, and there are all these weird voices and spinning heads?

 

JEFF WALKER: Oh, I remember that, yes.

 

NOAH BLUMENSON-COOK: It was sort of like that. I’m curious as to what exactly was happening. From my perspective, I would get a good run of answers, maybe two or three I would feel really good about. Around question eight, my response would be a little bit blended with the sort of dream state I was in and then I would snap back into it. I invented a scale for myself: one was no comprehension at all, at two I could pick out a word or two, three meant I could mostly understand it and four meant I completely got it. I almost never hit four.

 

JEFF WALKER: The way Jess had that experiment set up, it was for you to determine your own scaling system. We’ve done another experiment where that’s a little more defined and you actually did it the way that experiment was run.

 

Video still by Colin Stringer

JESS ROWLAND: The experiment was designed so that some sentences would be more clear than others. You may have noticed that some of the garbled sentences were the same as the preceding sentence. Sometimes they were the same and sometimes they were different.

 

NOAH BLUMENSON-COOK: Every time the sentences differed, it totally threw me.

 

JESS ROWLAND: That’s been our experience with other people who have done the study. We want to see what kind of effect prior knowledge has on your ability to understand unclear sentence structures. We want to understand the ways that people make speech intelligible and what factors are important for making that happen. If you hear the clear sentence, you have a template, a kind of auditory object or memory. When you hear the unclear sentence you can apply that template to make sense of the information. We want to see what kind of effects there are in the brain that correspond to making a sentence intelligible.

 

We want to understand the ways that people make speech intelligible and what factors are important for making that happen. – Jess Rowland

 

NOAH BLUMENSON-COOK: Now it makes sense why I needed to be naïve to the experiment going in.

 

JESS ROWLAND: Exactly. If I had said “Oh, by the way, some of these sentences will be the same” then you would have all kinds of biases going into the experiment. It was important for you to figure that out on your own.

 

NOAH BLUMENSON-COOK: What are you looking for in the MEG data?

 

JEFF WALKER: We’re looking for brain function, neuronal activity. We look at it via the magnetic fields that your neurons generate from electrical synaptic firing in response to the stimuli. One of the reasons we have to repeat the stimuli over and over again is that each of those events are pretty small in terms of field strength, as compared to the outside environment. What Jess will do is go back and take each of those events and average them together to get a picture of what your brain is doing.

 

JESS ROWLAND: At that point we can compare what’s happening in your brain when you’re listening to the clear sentence and when you hear the garbled sentence, and see in what ways it’s similar.

 

NOAH BLUMENSON-COOK: What kind of variations have you seen so far?

 

JESS ROWLAND: Well, you’re subject number five. The experiment is still pretty new.

 

NOAH BLUMENSON-COOK: Are there any correlations you expect to find?

 

JESS ROWLAND: The speech signal has a certain rhythm to it. This rhythm happens in syllable chunks and those have a discrete frequency range in which they can be found. What I hope we’ll find is a signal in that range that happens in your brain while you’re listening to the sentences.

 

NOAH BLUMENSON-COOK: How far does the brain’s electromagnetic field extend?

 

The speech signal has a certain rhythm to it. This rhythm happens in syllable chunks and those have a discrete frequency range in which they can be found. – Jess Rowland

 

JESS ROWLAND: It’s an extremely faint field, and it drops off very quickly. The closer you can get to the source, the better.

 

NOAH BLUMENSON-COOK: Which must be why you have to put your head into such a tiny space.

 

JEFF WALKER: It’s like wearing a toilet bowl for a hat. 

 

NOAH BLUMENSON-COOK: With such a tiny signal, how do you know when you’re looking at brain activity?

 

JEFF WALKER: You’re looking at responses to stimuli, so you get very clear peaks to the signal. When you have a model of the person’s head, you can take that information and overlay it onto a three-dimensional space.

 

NOAH BLUMENSON-COOK: Thank you. This was fascinating to say the least.

 

JESS ROWLAND: Thank you for being a subject!

 

To read the article written by Imagine Science Films' Sarah Banker, click here.



Forthcoming from IMMATERIAL:

Part Two of The David Poeppel Lab Visit

Special thanks to Sarah Banker and Colin Stringer of Imagine Science Films

 

 

jessGlobe.jpg
Jess Rowland is a sound artist, musician, and composer currently represented by Edgetone Records. She has earned composition grants, commissions by theater and dance companies, and is affiliated with the Center for New Music and Audio Technology. In addition to recent performances at the Berkeley Art Museum, Luggage Store Gallery, Spectrum NYC, and Mobius Gallery, she has published a paper focusing on her current Sound Art concerns in the Leonardo Music Journal. She received her MFA in Sound Art Practice and New Media from UC Berkeley, currently serves as lab manager in the Auditory Neuroscience lab of David Poeppel, and will soon undertake an Artist in Residency at Kala Gallery, Berkeley.
jeff-300x274.jpg
Jeffrey Walker is an Associate Research Scientist and MEG Lab Guru at the The KIT/NYU MEG Joint Research Lab. Originally from Denver, Colorado where he received his BS in Zoology, he spent many years in California and Maryland. He is now enjoying the adventure that is living in New York.