Double Blind testing is like fishing without knowing what kind of fish you're after!

[russ69]

Guys/Gals,

Would you go fishing without knowing what your fishing for? I wouldn't.

Thanx, Russ

[JJkizak]

Fishing for distortion.

JJK

[Hifi jim]

Fishing for distortion.

Sounds more like trolling to me... []

[russ69]

I wanted to illustrate blind testing with an example. Let's say we have a photograph of a very well hidden tiger in the brush. Now if we ask people, what do they see in the photograph many might not see the tiger. On the other hand if we say; Can you see the tiger in the photograph, most people will know what to look for and find the tiger. Listening is the same way, if you help people to locate what they are listening for they can focus on the task and identify changes more accurately.

Read and Discuss............

Thanx, Russ

[oldenough]

This isn't leading to a discussion about cables.... is it?? Cos' as i'm sure you know...that would be pointless. [:|]

[Hifi jim]

I wanted to illustrate blind testing with an example. Let's say we have a photograph of a very well hidden tiger in the brush. Now if we ask people, what do they see in the photograph many might not see the tiger. On the other hand if we say; Can you see the tiger in the photograph, most people will know what to look for and find the tiger. Listening is the same way, if you help people to locate what they are listening for they can focus on the task and identify changes more accurately.

Read and Discuss............

Thanx, Russ 

 

I see your point Russ, but my problem with double blind testing of any component is that it's more of a test of human memory, which we know is very unreliable. The example you've made here hints at the point of view that serious listening is a learned skill.

[33RPM]

Let me twist this around a bit: what if I played two different audio files in succession, and told you that file A was uncompressed PCM ("listen to the richness, harmonic content, and "air" around the instrument" I might prompt), and then played back file B, which I claim to be data-reduced MP3 ("hear how the image is flattened, and the transients are smeared?"). Problem is, file A and B are exactly the same, 16-bit, PCM recording. If I tell you what they are before I switch back and forth between each, there's a good chance that you will "hear" things in file A that either aren't really there, or ARE there in both files, but since you are paying more attention and concentrating to hear the "improvements" when I switch to file A, only seem to be present in one recording. Now, if I then switch back and forth between the files and have YOU tell me which ones they are without prompting you, you'll attain chance at best, since they're the same - even though you're convinced that file A is better. That is the power of suggestibility - the placebo effect. Just because you think something sounds better doesn't mean that it IS measurably better or perceptibly different.

Problem is, perception is very hard to test validly and reliably, double-blind or not. It's the classic dillema of the outsider looking in - who am I to say what people can or cannot hear? I say just enjoy the music and enough with all the damn audio testing - and I'm a psychologist + statistician! []

[skonopa]

I though the whole purpose of a double-blind test is to remove any pre-conceptions or biases going into the test and to find out if any difference, regardless of what the specific difference may be, can be detected. I guess, using the fishing analogy, like approaching a body of water and seeing if it is even possible
to catch any fish at all, regardless of what type/species, ie, some
biological study to see if any life at all may be present in a
particular body of water, such as one of the hot springs in Yellowstone
National Park.

Using the "hidden tiger in the picture" example, suppose that we have two pictures. One containing the tiger and one not. Suppose even the person admistering the test does not know what the difference between the two photos are (one containing the tiger and one not), or perhaps not even know there is a difference (perhaps the pictures could be switched randomly - both containing the tiger, neither containing the tiger, or one with/one without).

The idea is that if the subject can detect any difference at all between the photos, without any pre-conceptions or biases (i.e., the subject loves tigers, or detests them, etc).

Now, if the person administrating the test knew there was a tiger missing in one of the photos, he may go and ask the subject to find which photo contains the tiger. There is a good chance a number of people taking the test may indeed find a tiger in the photo that actually does not contain the tiger and vice versa, with the preconception that there must be a tiger in there somewhere.

[TBrennan]

Note that DBTs are done to detect differences not preferences. You can either hear a difference or you can't, it's pretty simple. Not that the basic simplicity of this stops some audiophiles from rationalizing at great length how they supposedly hear things they can't show they hear.

I've taken part in DBTs and they're sobering. You can either deal with the fact that you may often bullshit yourself and learn from it or you can deny the validity of the process and continue to bullshit yourself. Your choice.

[fini]

It would be way easier to distinguish cuts A and B if they each featured different songs.

[nonobaddog]

That is a very weird double blind test design. Kind of pointless. Really it seems to be designed to test the subject's susceptibility to suggestion rather than their ability to detect differences in anything.

A double blind listening test means that the subject is blind to which version they are listening to and the test administrator is also blind to the version they are presenting. That way they can not impart any bias to the test.

In your test you are intentionally imparting lots of bias and observing the subject's reaction to it.

[Deang]

http://www.avguide.com/forums/blind-listening-tests-are-flawed-editorial

http://www.aes.org/e-lib/browse.cfm?elib=5540

[nonobaddog]

Here is one of my favorites too.

http://www.hometheaterhifi.com/volume_11_4/feature-article-blind-test-power-cords-12-2004.html

I just think it is funny how people throw around the term double-blind when they really mean blind and don't even know it.

[djk]

"This isn't leading to a discussion about cables.... is it?? Cos' as i'm sure you know...that would be pointless"

I agree.

In my years in a high-end audio salon I found that 90 out of 100 people can't hear (in the sense we are discussing), and that only 1 out of the 10 that could hear would pick the most musically accurate components.

For a while I found this depressing, until I realized that I could offer blow-out prices on equipment that was sonic garbage to the 90% and not feel bad. Warning: if you are an abusive customer and try and beat me up on price I'll sell you one of these pieces below my cost (after all, somebody has to buy it, and you earned it).

[Deang]

If you want to see something really depressing (for me), check out the beating I took on the AVS forum this week.

The thread is actually about the original poster being concerned about the low impedance dips of the RF-7, and was looking for recommendations for an amp that could handle them. He made the comment that he was sending his crossovers to me for upgrading/modification. When they started to give him some grief, he emailed me to let me know. I went over and was quickly pulled into a black hole. They wanted a lot of measurements and DBT under my belt to support the subjective improvement I was claiming -- and I couldn't provide either. I didn't do well.

Probably best just to start at the bottom of page 4.

http://www.avsforum.com/avs-vb/showthread.php?t=1147061&page=4

[Zen Traveler]

Hi Dean, Thanks for all of the advice you have given me on this Forum. :-)

I post over at AVS as Zen Traveler and thought what you had to say about the 3 to 4 khz ringing interesting.... Where would I find that material on a multichannel audio disk? I remember you said the mod improved the sound of cymbals, but could you point me in the direction of something else in the Rock genre....I have Queen's Bohemian Rhapsody DVD-A and it seems bright to me, but I wonder if it is the material. Btw, the title of the linked thread is " Amp Stable down to 2 Ohms," and it does brings up some good questions about why a 102 dB rated speaker needs a beefier power supply.

[TBrennan]

The "double blind test" will live forever as the ultimate rationalization used by people who want desperately to be relevantly involved in audio, but who obviously have no discernment for what they are hearing - thus "everything sounds the same". It's ironic that they attempt to negate the main reason for being in the hobby in the first place, but hey, that's why it's called rationalization.

There's another group that embraces the double blind myth. People whose egos insist they always must have the best, but whose wallets won't cooperate, will insist forever that their $29 CDP is exactly the same, and sounds no different than, a $2900 state of the art unit........

You make an argument ad hominem and build a straw man. You'd never get away with that in a Jesuit high school.

Many who value DBTs don't deny things can sound different but they want to know which things really do sound different. That's how progress is made for one thing. It's also how you avoid wasting money and effort. And if a cheaper device can sound the same as a more expensive one then isn't that a good thing?

I think many don't dare put themselves to the test. And I must add that having bullshitted myself I've no doubt of the ability of others to do the same.

[Deang]

"I post over at AVS as Zen Traveler and thought what you had to say about the 3 to 4 khz ringing interesting.... Where would I find that material on a multichannel audio disk? I remember you said the mod improved the sound of cymbals, but could you point me in the direction of something else in the Rock genre....I have Queen's Bohemian Rhapsody DVD-A and it seems bright to me, but I wonder if it is the material."

I only pointed to that thread to show that with some people you can't claim any kind of subjective improvement without a lot of charts, graphs (preferrably filled with circles and arrows), and of course -- DBT.

I never said anything about 3-4kHz ringing, read it again. I don't know what you mean by finding "that material on a multichannel audio disk". All I can really say is that the problem exists and certain material exacerbates the issue. What is the purpose of pointing you "in the direction of something else in the Rock genre". It would be better if we moved these posts to the AVS thread -- I don't want this thread hijacked.

[Deang]

"Many who value DBTs don't deny things can sound different but they want to know which things really do sound different."

I can't think of a single thing in audio subjected to DBT that demonstrated a real "difference". Things can sound different, and they usually do until you do DBT -- then they don't. As Simon would say, "it's horrific comedy".

[Mallette]

There are reasons for A/B testing, but I can't think of any I care about. Music reproduction is either accurate or not and I don't need a "B" to tell me.

In any event, I've seen this thread in it's various incarnations over the years make cable threads look like a joust of gentlemen compared to Cold Harbor. I'll drop back in on about page 6 to see who's still standing...

Dave