psycho acoustics: the nightmare of listening tests

[pski]

I hope everybody will bear with me, this is going to be a bit long, I have a lot to say on this subject.

As the regulars should know by now I am one who can hear differences in equipment, and I'm also a technical person who builds his own stuff and tries hard to come up with correlations to measurable aspects of systems.

I've noticed many ways in which the brain influences what I hear, so I'm definitely in the what you perceive can change camp, BUT that does not lead me to believe that all listening is therefore worthless, just that I need to have some understanding of what is happening so I can make somewhat meaningful interpretations of what I perceive.

At this point I'm going to share some experiences in perception in another hobby of mine, hopefully this will illuminate the subject. I'm also heavily into amateur astronomy including building my own telescopes. Visual astronomy is another field in which human perception is key. We frequently are looking at objects which are right on the edge of perception, the brain also does all kinds of interesting things to modify what you perceive. The experienced observers have learned to manipulate the brain to get it to let them see whats really there. Astronomy has a big advantage over audio, you can look at the same spot in space with a big telescope and camera and determine if what the trained observers are seeing is really there or not.

The upshot is that the trained observers can see things that untrained people can't, its not because they have "better vision", or better telescopes, but they have learned to manipulate how the brain filters what they perceive. "objective" measurements (pictures from the Hubble etc) verify that what they observe really is there.

When I was new to the hobby I experienced this in a (at least to me) dramatic way. I had bought a fairly inexpensive yet quite decent telescope and was trying to see a famous nebula, which all the books said I should be able to see with this telescope. I spent a month and could not find it. From the star charts I knew just where it should be, but when I looked there, I couldn't see anything. I went to a star party with a bunch of experienced observers, and asked one of them if he could find the nebula with my telescope, he looked through it and said its right there, I looked and looked and could not see it. He chuckled and said "you are looking too hard, don't concentrate on seeing it, just relax and "look" at the whole field of view". I did that, and presto it popped into view, I concentrated hard at looking at the nebula and it vanished. I hadn't changed the scope, I hadn't changed where I was looking, the nebula was on the same spot on the retina, it wasn't the difference between rods and cones etc, it was what I was concentrating on!

My theory at this point is that the brain is performing a form of AGC (automatic gain control) on what I'm looking at it tries to "normalize" what I see. When I concentrate on one specific part of the field, it throws out the rest and normalizes what I'm concentrating on to dark gray. When I don't concentrate on any one specific thing, it can only normalize on the whole image so the subtle differences in brightness are preserved and the nebula pops into view.

My assertion is that the same thing happens in audio. When you are deliberately performing a test, and concentrating hard at "critical listening", you concentrate on a specific aspect of the sound, your brain throws out the rest and normalizes that aspect. When you do the same with a different piece of equipment, it normalizes that aspect, and low and behold they sound the same or or only slightly different. BUT if you forget about critical listening and just experience the whole totality of the music you start to be able to perceive subtle contrasts and how the different equipment renders them. But the moment you "focus in on them" they float away.

So are DBTs useless? No, you just have to learn how to do them. The "listen to the same few seconds and switch back and forth" is going to guarantee you are in the critical listening mode. You have to listen for at least several tracks so you can get into the "experience the totality of the music" mode and ignore the fact you are taking part in a test. This is not easy to do, but can be done if you just relax and don't try so hard. At least for me when I get into this mode I can start hearing all kinds of subtle nuances in the performance and the space the performance was performed in. The "contrasts" in the sound field are increased. Different gear does this "showing the contrasts" differently. The brain is still normalizing, but its normalizing the whole thing. You still don't get an "absolute" view into the sound, but it is possible to determine how different equipment shows the contrasts.

I know there will be some who will say, "but how do you know this "contrast" is real and not something the brain made up?" My take is this: we know the brain filters what we perceive, what is more likely, the lack of contrast is reality and the more contrast is made up, OR the lack of contrast is the brain filtering out information and the higher contrast is closer to reality? My experience has been that the brain does far more filtering than it does creating out of nothing. So I'm going to go for assuming the greater contrast I hear when listening to the totality of the music is closer to reality and the lack of contrast when critically listening is the brain filtering information, thus I'm going to make judgements about what makes a difference based on what I hear when in the "totality" mode rather than in the critical listening mode.

I hope all this makes some sort of sense.

John S.

Not really.

[evdplancke]

I hope everybody will bear with me, this is going to be a bit long, I have a lot to say on this subject.

As the regulars should know by now I am one who can hear differences in equipment, and I'm also a technical person who builds his own stuff and tries hard to come up with correlations to measurable aspects of systems.

I've noticed many ways in which the brain influences what I hear, so I'm definitely in the what you perceive can change camp, BUT that does not lead me to believe that all listening is therefore worthless, just that I need to have some understanding of what is happening so I can make somewhat meaningful interpretations of what I perceive.

At this point I'm going to share some experiences in perception in another hobby of mine, hopefully this will illuminate the subject. I'm also heavily into amateur astronomy including building my own telescopes. Visual astronomy is another field in which human perception is key. We frequently are looking at objects which are right on the edge of perception, the brain also does all kinds of interesting things to modify what you perceive. The experienced observers have learned to manipulate the brain to get it to let them see whats really there. Astronomy has a big advantage over audio, you can look at the same spot in space with a big telescope and camera and determine if what the trained observers are seeing is really there or not.

The upshot is that the trained observers can see things that untrained people can't, its not because they have "better vision", or better telescopes, but they have learned to manipulate how the brain filters what they perceive. "objective" measurements (pictures from the Hubble etc) verify that what they observe really is there.

When I was new to the hobby I experienced this in a (at least to me) dramatic way. I had bought a fairly inexpensive yet quite decent telescope and was trying to see a famous nebula, which all the books said I should be able to see with this telescope. I spent a month and could not find it. From the star charts I knew just where it should be, but when I looked there, I couldn't see anything. I went to a star party with a bunch of experienced observers, and asked one of them if he could find the nebula with my telescope, he looked through it and said its right there, I looked and looked and could not see it. He chuckled and said "you are looking too hard, don't concentrate on seeing it, just relax and "look" at the whole field of view". I did that, and presto it popped into view, I concentrated hard at looking at the nebula and it vanished. I hadn't changed the scope, I hadn't changed where I was looking, the nebula was on the same spot on the retina, it wasn't the difference between rods and cones etc, it was what I was concentrating on!

My theory at this point is that the brain is performing a form of AGC (automatic gain control) on what I'm looking at it tries to "normalize" what I see. When I concentrate on one specific part of the field, it throws out the rest and normalizes what I'm concentrating on to dark gray. When I don't concentrate on any one specific thing, it can only normalize on the whole image so the subtle differences in brightness are preserved and the nebula pops into view.

My assertion is that the same thing happens in audio. When you are deliberately performing a test, and concentrating hard at "critical listening", you concentrate on a specific aspect of the sound, your brain throws out the rest and normalizes that aspect. When you do the same with a different piece of equipment, it normalizes that aspect, and low and behold they sound the same or or only slightly different. BUT if you forget about critical listening and just experience the whole totality of the music you start to be able to perceive subtle contrasts and how the different equipment renders them. But the moment you "focus in on them" they float away.

So are DBTs useless? No, you just have to learn how to do them. The "listen to the same few seconds and switch back and forth" is going to guarantee you are in the critical listening mode. You have to listen for at least several tracks so you can get into the "experience the totality of the music" mode and ignore the fact you are taking part in a test. This is not easy to do, but can be done if you just relax and don't try so hard. At least for me when I get into this mode I can start hearing all kinds of subtle nuances in the performance and the space the performance was performed in. The "contrasts" in the sound field are increased. Different gear does this "showing the contrasts" differently. The brain is still normalizing, but its normalizing the whole thing. You still don't get an "absolute" view into the sound, but it is possible to determine how different equipment shows the contrasts.

I know there will be some who will say, "but how do you know this "contrast" is real and not something the brain made up?" My take is this: we know the brain filters what we perceive, what is more likely, the lack of contrast is reality and the more contrast is made up, OR the lack of contrast is the brain filtering out information and the higher contrast is closer to reality? My experience has been that the brain does far more filtering than it does creating out of nothing. So I'm going to go for assuming the greater contrast I hear when listening to the totality of the music is closer to reality and the lack of contrast when critically listening is the brain filtering information, thus I'm going to make judgements about what makes a difference based on what I hear when in the "totality" mode rather than in the critical listening mode.

I hope all this makes some sort of sense.

John S.

Thanks John for this very valuable post that puts this thread back on track!

[Ron Olsen]

If you want to know how the brain works to interpret auditory stimuli, please listen to James Johnston's remarks from 1:00 to 5:20 in the Audio Myths Workshop: http://www.youtube.com/watch?v=BYTlN6wjcvQ

FYI, JJ is a recognized expert in the field; see his credentials here: http://home.comcast.net/~retired_old_jj/

Bottom line: blind listening tests are necessary to determine if a perceived auditory difference is real or not.

[darrenyeats]

I hope everybody will bear with me, this is going to be a bit long, I have a lot to say on this subject.

As the regulars should know by now I am one who can hear differences in equipment, and I'm also a technical person who builds his own stuff and tries hard to come up with correlations to measurable aspects of systems.

I've noticed many ways in which the brain influences what I hear, so I'm definitely in the what you perceive can change camp, BUT that does not lead me to believe that all listening is therefore worthless, just that I need to have some understanding of what is happening so I can make somewhat meaningful interpretations of what I perceive.

At this point I'm going to share some experiences in perception in another hobby of mine, hopefully this will illuminate the subject. I'm also heavily into amateur astronomy including building my own telescopes. Visual astronomy is another field in which human perception is key. We frequently are looking at objects which are right on the edge of perception, the brain also does all kinds of interesting things to modify what you perceive. The experienced observers have learned to manipulate the brain to get it to let them see whats really there. Astronomy has a big advantage over audio, you can look at the same spot in space with a big telescope and camera and determine if what the trained observers are seeing is really there or not.

The upshot is that the trained observers can see things that untrained people can't, its not because they have "better vision", or better telescopes, but they have learned to manipulate how the brain filters what they perceive. "objective" measurements (pictures from the Hubble etc) verify that what they observe really is there.

When I was new to the hobby I experienced this in a (at least to me) dramatic way. I had bought a fairly inexpensive yet quite decent telescope and was trying to see a famous nebula, which all the books said I should be able to see with this telescope. I spent a month and could not find it. From the star charts I knew just where it should be, but when I looked there, I couldn't see anything. I went to a star party with a bunch of experienced observers, and asked one of them if he could find the nebula with my telescope, he looked through it and said its right there, I looked and looked and could not see it. He chuckled and said "you are looking too hard, don't concentrate on seeing it, just relax and "look" at the whole field of view". I did that, and presto it popped into view, I concentrated hard at looking at the nebula and it vanished. I hadn't changed the scope, I hadn't changed where I was looking, the nebula was on the same spot on the retina, it wasn't the difference between rods and cones etc, it was what I was concentrating on!

My theory at this point is that the brain is performing a form of AGC (automatic gain control) on what I'm looking at it tries to "normalize" what I see. When I concentrate on one specific part of the field, it throws out the rest and normalizes what I'm concentrating on to dark gray. When I don't concentrate on any one specific thing, it can only normalize on the whole image so the subtle differences in brightness are preserved and the nebula pops into view.

My assertion is that the same thing happens in audio. When you are deliberately performing a test, and concentrating hard at "critical listening", you concentrate on a specific aspect of the sound, your brain throws out the rest and normalizes that aspect. When you do the same with a different piece of equipment, it normalizes that aspect, and low and behold they sound the same or or only slightly different. BUT if you forget about critical listening and just experience the whole totality of the music you start to be able to perceive subtle contrasts and how the different equipment renders them. But the moment you "focus in on them" they float away.

So are DBTs useless? No, you just have to learn how to do them. The "listen to the same few seconds and switch back and forth" is going to guarantee you are in the critical listening mode. You have to listen for at least several tracks so you can get into the "experience the totality of the music" mode and ignore the fact you are taking part in a test. This is not easy to do, but can be done if you just relax and don't try so hard. At least for me when I get into this mode I can start hearing all kinds of subtle nuances in the performance and the space the performance was performed in. The "contrasts" in the sound field are increased. Different gear does this "showing the contrasts" differently. The brain is still normalizing, but its normalizing the whole thing. You still don't get an "absolute" view into the sound, but it is possible to determine how different equipment shows the contrasts.

I know there will be some who will say, "but how do you know this "contrast" is real and not something the brain made up?" My take is this: we know the brain filters what we perceive, what is more likely, the lack of contrast is reality and the more contrast is made up, OR the lack of contrast is the brain filtering out information and the higher contrast is closer to reality? My experience has been that the brain does far more filtering than it does creating out of nothing. So I'm going to go for assuming the greater contrast I hear when listening to the totality of the music is closer to reality and the lack of contrast when critically listening is the brain filtering information, thus I'm going to make judgements about what makes a difference based on what I hear when in the "totality" mode rather than in the critical listening mode.

I hope all this makes some sort of sense.

John S.

John, imagine we picked two pieces of sky, one with a nebula and one without and we repeatedly and randomly showed you each and asked if it was the nebula or non-nebula sky you were seeing. If you got it right 50% of the time, what should we conclude?

DBTs do not necessarily entail "snippets" - that is a misconception. You can take as long as you like. There is no excuse for rejecting blind listening as a concept.
Darren

[JohnSwenson]

John, imagine we picked two pieces of sky, one with a nebula and one without and we repeatedly and randomly showed you each and asked if it was the nebula or non-nebula sky you were seeing. If you got it right 50% of the time, what should we conclude?

DBTs do not necessarily entail "snippets" - that is a misconception. You can take as long as you like. There is no excuse for rejecting blind listening as a concept.
Darren

Isn't that exactly what I was saying? I thought I specifically said that DBTs are valuable as long as you go into them understanding that the very act of trying to concentrate on differences can alter your perceptions. I guess I didn't make myself clear enough since everybody is assuming I said DBTs are not valuable. Oh well, I'll try better next time.

On the "snippets" bit, I know that very well, its just that I have seen and been part of quite a few tests that do just that, I was trying to point out that this type of test fosters the type of testing I find less valuable and there is a different way to do it.

John S.

[Phil Leigh]

I hope everybody will bear with me, this is going to be a bit long, I have a lot to say on this subject.

As the regulars should know by now I am one who can hear differences in equipment, and I'm also a technical person who builds his own stuff and tries hard to come up with correlations to measurable aspects of systems.

I've noticed many ways in which the brain influences what I hear, so I'm definitely in the what you perceive can change camp, BUT that does not lead me to believe that all listening is therefore worthless, just that I need to have some understanding of what is happening so I can make somewhat meaningful interpretations of what I perceive.

At this point I'm going to share some experiences in perception in another hobby of mine, hopefully this will illuminate the subject. I'm also heavily into amateur astronomy including building my own telescopes. Visual astronomy is another field in which human perception is key. We frequently are looking at objects which are right on the edge of perception, the brain also does all kinds of interesting things to modify what you perceive. The experienced observers have learned to manipulate the brain to get it to let them see whats really there. Astronomy has a big advantage over audio, you can look at the same spot in space with a big telescope and camera and determine if what the trained observers are seeing is really there or not.

The upshot is that the trained observers can see things that untrained people can't, its not because they have "better vision", or better telescopes, but they have learned to manipulate how the brain filters what they perceive. "objective" measurements (pictures from the Hubble etc) verify that what they observe really is there.

When I was new to the hobby I experienced this in a (at least to me) dramatic way. I had bought a fairly inexpensive yet quite decent telescope and was trying to see a famous nebula, which all the books said I should be able to see with this telescope. I spent a month and could not find it. From the star charts I knew just where it should be, but when I looked there, I couldn't see anything. I went to a star party with a bunch of experienced observers, and asked one of them if he could find the nebula with my telescope, he looked through it and said its right there, I looked and looked and could not see it. He chuckled and said "you are looking too hard, don't concentrate on seeing it, just relax and "look" at the whole field of view". I did that, and presto it popped into view, I concentrated hard at looking at the nebula and it vanished. I hadn't changed the scope, I hadn't changed where I was looking, the nebula was on the same spot on the retina, it wasn't the difference between rods and cones etc, it was what I was concentrating on!

My theory at this point is that the brain is performing a form of AGC (automatic gain control) on what I'm looking at it tries to "normalize" what I see. When I concentrate on one specific part of the field, it throws out the rest and normalizes what I'm concentrating on to dark gray. When I don't concentrate on any one specific thing, it can only normalize on the whole image so the subtle differences in brightness are preserved and the nebula pops into view.

My assertion is that the same thing happens in audio. When you are deliberately performing a test, and concentrating hard at "critical listening", you concentrate on a specific aspect of the sound, your brain throws out the rest and normalizes that aspect. When you do the same with a different piece of equipment, it normalizes that aspect, and low and behold they sound the same or or only slightly different. BUT if you forget about critical listening and just experience the whole totality of the music you start to be able to perceive subtle contrasts and how the different equipment renders them. But the moment you "focus in on them" they float away.

So are DBTs useless? No, you just have to learn how to do them. The "listen to the same few seconds and switch back and forth" is going to guarantee you are in the critical listening mode. You have to listen for at least several tracks so you can get into the "experience the totality of the music" mode and ignore the fact you are taking part in a test. This is not easy to do, but can be done if you just relax and don't try so hard. At least for me when I get into this mode I can start hearing all kinds of subtle nuances in the performance and the space the performance was performed in. The "contrasts" in the sound field are increased. Different gear does this "showing the contrasts" differently. The brain is still normalizing, but its normalizing the whole thing. You still don't get an "absolute" view into the sound, but it is possible to determine how different equipment shows the contrasts.

I know there will be some who will say, "but how do you know this "contrast" is real and not something the brain made up?" My take is this: we know the brain filters what we perceive, what is more likely, the lack of contrast is reality and the more contrast is made up, OR the lack of contrast is the brain filtering out information and the higher contrast is closer to reality? My experience has been that the brain does far more filtering than it does creating out of nothing. So I'm going to go for assuming the greater contrast I hear when listening to the totality of the music is closer to reality and the lack of contrast when critically listening is the brain filtering information, thus I'm going to make judgements about what makes a difference based on what I hear when in the "totality" mode rather than in the critical listening mode.

I hope all this makes some sort of sense.

John S.

John - a couple of observations if I may?

1) most albums that you hear were mixed and mastered in "critical mode". this is one of the great benefits of total recall automation and non-destructive DAW's, You can make a change and instantly A/B it... Yes Ther will also be some more contemplative listening once a good mix or master has been achieved... But if changes are deemed necessary, they are invariable applied in "critical mode". Partially because of this, I find the critical mode useful in assessing changes - at least in assessing IF THERE IS AN AUDIBLE CHANGE... Better or worse may only be ultimately be agreed through non-critical listening. I am personally convinced that if a difference can only be heard after long periods of non-critical listening, there is no difference.

2) Your astronomy analogy... There is a world of difference between straining to resolve a "faint fuzzy" that you, I and the rest of science agree actually exists... And straining to hear something that nobody can agree exists or not!
Kind regards
Phil

[Soulkeeper]

most albums that you hear were mixed and mastered in "critical mode".

I thought most albums were mixed in "critical mode" and mastered in "relaxed mode" (or at least attempted mastered in "relaxed mode"), and that this was why it's considered a good idea to have a different person doing the mastering?

[Phil Leigh]

I thought most albums were mixed in "critical mode" and mastered in "relaxed mode" (or at least attempted mastered in "relaxed mode"), and that this was why it's considered a good idea to have a different person doing the mastering?

But what happens when the mastering engineer/ producer/artists decides that a change is required?... It's back to the (re)mix! Not everything can be fixed in mastering... If the vocals are too loud or have too much reverb etc...
You can do general EQ nd overall dynamic fixes, levels etc... But if a particular track is causing a problem...

[darrenyeats]

Isn't that exactly what I was saying? I thought I specifically said that DBTs are valuable as long as you go into them understanding that the very act of trying to concentrate on differences can alter your perceptions. I guess I didn't make myself clear enough since everybody is assuming I said DBTs are not valuable. Oh well, I'll try better next time.

On the "snippets" bit, I know that very well, its just that I have seen and been part of quite a few tests that do just that, I was trying to point out that this type of test fosters the type of testing I find less valuable and there is a different way to do it.

John S.

John,
What's to prevent you from listening to the whole in a DBT? Especially if you have in mind this could be the key to discernment?

Generally, questions of listening skill (the above being a specific example of listening skill) apply equally to sighted or blind listening, in my view.

The bit I am failing to understand is which of your comments apply necessarily to blind listening and not sighted listening.

I do agree blind testing can be impractical in many cases! With you there! Best regards,
Darren

[mlsstl]

...the very act of trying to concentrate on differences can alter your perceptions.

Have to say this is one aspect of the DBT criticisms that I find over-the-top. The thought that this process intrinsically corrupts the purity of the listening process by making one think too hard is laughable, especially when mountains of research have clearly demonstrated how easily humans are swayed by their myriad unconscious subjectivities under sighted conditions.

Narrowly focusing one's attention is not a blind vs sighted issue. It is also just as easy to concentrate on differences when listening under sighted conditions.

However, these same people who are worried about warped perceptions due to stress under blind conditions often blithely dismiss any notion of subconscious bias under sighted conditions. They fancy themselves in perfect and complete control of all possible influences.

But sometimes blind testing is the only thing that will knock the entrenched elite off their high horses. Saw a TV program the other day about "The Judgement of Paris". In 1976 a wine tasting was held in Paris, specifically to prove the superiority of French wines. The panel was composed 100% of top French wine experts. The results stunned the wine world when California wines came in first. No one doubted the results would have been dramatically different if the tastings had been done sighted.

In short, we do not cease to be limited by our human condition because we believe ourselves to be experts in something. Audio is not an exception.