NO test is vallid unless it is Doubble Blind Test (DBT

[BigCliff]

That is corect in France probably where photo was tooken

If it was in France then the other sign would read Arrete!

[michaelhigh]

I don't need to know I got hit in the face with a brick to tell I was hit in the face with a brick. I don't need to know what color the brick was, I can look in the mirror and tell by the color of the brick rubbed off on my face what color it was. It's probably going to brick-colored anyway...

[gnatnoop]

I am reading like and diss-likes on here butt nobody is doing it sciencetifically . . . . .

Wrong. It takes a long time to evaluate a system change. It may require listening to a wide range of music, speaker placement changes, subwoofer integration or perhaps some cabling changes. An opinion is formed over time, then compared to the previous piece of equipment. A judgement is then made as to the value of the change. Small changes can be subtle and can be totally missed with blind testing. It's well documented that DBT does not distinguish small differences. Big differences don't even need DB testing, it's too obvious. You'll be much better off listening to a wide range of equipment and gaining experiencing than you will ever achieve wasting your time performing double blind test scenarios. Listening is not science, we use science to see if our perceptions can somehow be quantified. So far, we still have to trust our ears......

+1 Sometimes takes me days or weeks to form an opinion, listening to various selections that i've heard over the years on different gear. [Y]

[davis419b]

Very well said Mark ! Even a moron would understand the way you explained it.

[cornfedksboy]

I'm not a proponent (or opponent for that matter) of the DBT; however, just as one poster stated earlier, he prefers to buy equipment that is asthetically pleasing and popular so he can resell it. Others prefer to have speakers that generate the best sound regardless of appearance, and still others might want a speaker that is decidely not a "popular" speaker as this insinuates rarity or exclusivity. Some prefer heavy bass or a localization of sound while others like to have a mellow and soft sound field that is surrounding and relaxing.

Choosing a speaker based on a scientific study is equally as valid as any other example given. It allows one to enjoy art or entertainment expressed in their own unique way.

[JeffRay]

In my first life, my buddy and I would hit the music shops from time to time. We would scope out the trade ins and try out all the new stuff. We were not full time pros but we knew how to play. I would grab every pro grade Trumpet and Flugelhorn and head to the practice room (do they still have those?). My buddy would start checking out the best guitars. I couldn't play a chord on the guitar and he couldn't blow a note. But... when one of us found a great instrument we'd call the other one into the room and say "man, you gotta hear this." The point? We're talking music. No musician would ever consider double-blinding an instrument; he'd think that was silly. Now an instrument designer? Sure, he'd use every tool at his disposal; electronic or otherwise to assemble and tune an instrument. Then he will play it and voice it until it's right. I don't know nuthin about three hundred dollar AC power cords, I'm suspicious that those ought to be double blind tested by someone who knows nothing about AC current or Made in Mexico Romex and the $3 Circuit Breaker your'e plugging it into. But. . . but . . . a Loudspeaker is an analog transducer. Loudspeakers are closer to musical instruments than anything in the recording chain except perhaps microphones. There's a reason the people around this Forum love the sound of Heritage speakers. The same reason I'm acquiring the parts to build my CornScalas. The same reason I traded off a Mt. Vernon Bach Stradivarius for a Holton LeBlanc Al Hirt model. I could tell you about tapered lead pipes, beryllium copper bells, chamfered polished valve ports, and extra braces to control resonances, yada yada yada. My Bach and Shilke were considered SOTA by all my friends but the LeBlanc? It played bigger and louder, and for me, the TONE, ah, the TONE. Why am I coming back to Klipsch after all these years? Same reason. ----------- If I mod them I'll keep the mod or reverse it for the same reason too. TONE! When I get my Klipschs finished and dialed in I can't wait to call my buddies up and say "man, you gotta come hear this."

jeff

[NOSValves]

The difference in sound between two similar pieces of equipment is very difficult to discern, and most people wouldn't be able to tell the difference in a DBT. Being perfectly honest, I would tend to choose the piece of equipment that looks better and is popular. Why not? It's easy to sell if I don't like it. And anyway, I get more more satisfaction looking at my Hog in the garage rather than some a yama-harley - even though they both do the same job...

And this sums it all up!!

I do have a question how is this Snyder dude getting away with his horrible writing skills (not that I mind). If it was me I'd of been insulted at least twenty times in every thread I posted in....go figure. I'm kind of Jealous.

[pauln]

Nice thoughts JeffRay!

It is music we are talking about...

So, when components are compared using test instruments and physical measurments, is that *DDT?

* Double Deaf Testing

[edwinr]

Yeah. I wonder about Snyder. His 'horrible writing skills' are just too good to be true...

[Fjd]

From reading these posts, it seems that determining if DBT testing is warranted really depends on the situation at hand and a defined goal. Imagine a drug trial where we ask the subject if the drug made them feel better instead of measuring the physiological changes to determine the affects of the experimental variable.

If I wanted to mass-market an audio product, a statistically valid sample group of my target market selecting my product as “best” or “most desired” or “most pleasurable” (or the defined goal) in a true ABX test seems to be beneficial. I wouldn’t want to take the product to market, if my target market ABX test group kept picking it as “worst.” However, even to get to this point, I would have a lot of work to do in "defining the qualities" that characterize "good sound" in relation to my target market/audience.

As a consumer, I believe an individual DBT would take on less importance to most of the population. I make various “trade-offs” in my buying decisions and myself selecting an item based on a true ABX test on which sound is the "best" may not be the "most" important aspect of the decision (especially since I like to "match" certain audio equipment for a certain presentation of certain music I'm looking to explore). Besides, I doubt I could distinguish my own equipment in a “non-level” matched ABX comparison (where A and B are different sources that are level-matched and X is either A or B but not level matched).

[Fjd]

I do like pauln’s comments on “threshold of perception” and why it is difficult to rely on DBT exclusively.

Also, it wouldn’t make sense to try to build and target a low power SET tube amp to users of the old Infinity IRS Betas or Carver Amazing Loudspeaker with a sensitivity of 88dB/1W/1m, but it might be interesting to ABX a single ended solid state amp (i.e. First Watt) vs. a popular, similarly priced SET tube amp or one of the other similarly priced PP tube amps that match well with very high efficient speakers (i.e. Klipschorn or La Scala) while the remaining equipment stays the same. Will the test show that the differences could be too subtle to discern? How critical to the overall sound is a similarly priced amplifier in the chain of equipment? What are all the drivers in a purchase decision? I know that I’ve heard very distinct differences in pre-amplifiers of similar price without a DBT.

This is really oversimplified and I’m sure my “word selection” can be picked apart and I didn’t get into level-matching and other variables, but once you have equipment that you believe meets the “defined goal” in relation to your “target market” then the listening group that represents your target market can listen to A and B as long as they like and select A or B as the “best” or “most pleasing” or “most accurate” (or as possessing the “defined goal”), etc.

The listening group can also listen to X as long as they like and the only thing we know for sure is that X is either A or B. However, at some point the listening group must decide whether X is A or B. Repeat this test enough times to establish statistical validity with your listening group while the test operator will assign X randomly to A or B. If X is selected as representing A or B correctly enough times, have you met your “defined goal” in relation to your “target market” and now have a product that will compete for sales? Still maybe not, given the potential variables in the real world, but the odds should be better.

[Don Richard]

DBT is a technique for determining if differences can be detected per the testing protocol. The company I worked for developed a DBT in conjunction with some veterinarians for assessing the effect of painkillers on horses. We focused a sharp beam of light on the subject's leg so that it was burned and read the results on a polygraph to determined stress levels. The longer the horse could stand being burned before reacting determined the painkiller's effectiveness.

I'm not so sure that this method could be used to determine subtle differences such as audio reproduction might present.

[russ69]

If I wanted to mass-market an audio product, a statistically valid sample group of my target market selecting my product as best or most desired or most pleasurable (or the defined goal) in a true ABX test seems to be beneficial.

The bottom line in audio gear is simply, do I like what I hear. If the differences are great, it's a no-brainer and no DBT is required. If the differences are less dramatic, it's up to me to decide if the small difference is worth the price. I could care less what a panel of average people think or think they hear. It has absolutely no value to me whatsoever. If I gave value to the average Joe's opinion on the street, I guess I'd be driving a Toyota Corolla or listening to a Bose system. I'm the end user, so it's up to me if something has value to me, in my system, no one else can do this for me.

[SSnyder]

in for less then 2 thousand words

people who need have to see NAME of equipent to pontiffacate on its' merit of coarse are DBT deniers +++++++snake oil sailsmen got no darned use for DBT because who would by magic sound chrystals . . . . . ....... . .. . . .. . ....... see below

Large Brilliant Pebbles
On the floor in room corners, Large size Brilliant Pebbles reduces comb
filter effects caused by very high sound pressure levels that develop
in the corners when music is playing - as much as 3 or 4 times higher
than the average sound pressure level in the room!! The Large size
Brilliant Pebbles is also effective on tube amp Output Transformers; on
top of speaker cabinets; and on armboards of turntables. Other
effective locations include on top of Tube Traps; on side walls at the
first reflection points; on the wall behind the listener position at
points of high pressure; or anywhere in the 3 dimensional space of the
room where a sharp rise in sound pressure relative to the average sound
pressure in the room is measured using a test tone and sound pressure
meter. Price $129 each.

[bkrop]

Does a urine test have to be double blind to be valid?

[Fjd]

Mark - Great articulation of thoughts presented. I agree in that I (and probably most people due to the flaws in DBT you articulated) don’t want to make purchasing decisions based solely on DBT or ABX testing and the motivations you outline like aesthetics, status of the brand, physical attributes, personal emotional baggage, past experience with brand, expert opinions, peer pressure, utility, cost, etc are important and do come into play. Having the flu on listening day will most likely change perceptions.

However, since you have been on both sides as a consumer/listener and developer/seller of products, what are your thoughts on Canada’s National Research Council that I believe based most of their research for loudspeaker design on DBT testing? Doesn’t the testing at least eliminate some bias in the development of products?

Also, since a start-up company would not necessarily have the means to do DBT that would yield any meaningful results, what could you suggest a start-up do to develop a product and eliminate the bias associated with knowing the product’s make, model, price, appearance, etc.

russ69 - I'm not referring to the actual purchaser in the excerpt you quote and that was the distinction I was trying to make. I'm more curious from a developer standpoint (not the purchaser standpoint) how they would go about eliminating bias and where DBT may work. I do realize that a company designs and markets a product to those consumers with the greatest propensity to buy; therefore, I didn't name my product or the target group or what characterizes good sound to that target group (Bose and Toyota do sell a lot of product so they must be somewhat aware of their respective target groups). I've posed a question to Mark since he has been on both sides rather extensively.

bkrop - It seems to me that the most important thing in a double blind urine test (at least to the person administering the test) is that I don't miss the cup.

[Don Richard]

I have a sound in my head - a particular "kind" of sound, or nature of sound, or character of sound, which I enjoy more than other kinds, or natures of sound.

As a one-man design shop, I can choose and offer my own bias freely

This refreshing bit of honesty should be a required disclaimer for the audiophool press, who collectively deign certain audio products to be superior to all others by using sighted comparison tests and comparing to a "reference" system that they listened to at some point in the past. TAS champions this method of testing as accurate when it cannot be. It can only be biased, and in what direction?

[Hifi jim]

This refreshing bit of honesty should be a required disclaimer for the audiophool press, who collectively deign certain audio products to be superior to all others by using sighted comparison tests and comparing to a "reference" system that they listened to at some point in the past. TAS champions this method of testing as accurate when it cannot be. It can only be biased, and in what direction?

Limited to words, what other way could the audiophile press describe and compare the sound they hear? What else could reviewers possibly compare their findings with, if not a reference system of some sort? The reference they choose may not be to your liking, budget, or many other factors, but it is available and could be bought or demoed more readily than listening to a live performance which would sound different at any given moment. Without some sort of reference (e.g., Khorns sound bigger and livelier than Fortes), the point of reviewing and describing a components sound would be futile.

[NOSValves]

Now, what are magazines? Magazines are support structures for the audio industry. They are a kind of parasite living on the host of the industry. if you want to continue living on that host, you can't kill it - you need to get your nutrition but keep the host alive. So, they reinforce the basic propaganda mechanism used by the industry, which is "uncompromising design." Then they simply stage races or dog shows to demonstrate how these uncompromising designs compete. It's at its fundamentally dishonest root, but it doesn't mean the people involved like reviewers are bad people. It means they are doing a pre-determined job with RULES attached to that job. And, rule #1 is don't kill off the host!

The words......parasite, propaganda, fundamentally dishonest to descibe some folks but the people involved are not bad people.......Okay man I guess our ethical standards are very different.

[russ69]

My belief is that audio equipment comes in many flavors. I prefer some flavors over others and sometimes my favorite flavor isn't made to my liking or is poorly made. The designer has to make the choice and then the market decides if he made a design that is popular. In audio it isn't always item A is better than item B.