Can sound quality be measured?

[NOSValves]

http://news.cnet.com/8301-13645_3-20113895-47/can-sound-quality-be-measured/

I've met a lot of audio designers in my time, and all of the best ones have one thing in common, they have great "ears." They know what good sound sounds like. The opposite camp is populated with engineers that rely exclusively on measurements to "prove" their designs are better. To my way of thinking, the second group rarely makes great sounding products. Audio is too complex to be analyzed with just numbers alone.

Nowadays I'm meeting more digital audio engineers specializing in designing room and speaker correction software. They are usually very nice people, and their graphs and tests always look impressive on their laptops, but the presentations fall apart when I listen to their sound. Results vary from not bad to truly horrendous, but great sound is the least likely end product of their hard work.

Apparently, they were so focused on measuring sound they forgot to listen, or hire someone who actually knows what good sound sounds like. If the goal was to achieve better measurements I'd congratulate them for their accomplishments. But it's not, and discovering exactly what types of measurements indicate improved sound quality is an art. An art few of these engineers have mastered.

There are a couple of reasons why measurements fail to correlate with subjective sound quality assessments. First, electronics and speaker measurements have little to do with the sound of music. Test tones are too simple and predictable; music is far more complex and random. Reproducing the sound of a violin or a drum kit are exceedingly difficult tasks, and since the real goal of any hi-fi is to play music and not test tones, the designer's first priority should be making products that sound "good" for the intended market. For example, if you're designing DJ headphones, you aren't trying to deliver the most accurate bass. Far from it, you want to pump up the bass. Amplifier designers shouldn't waste their time trying to create an amp to drive simple test loads, they need to make an amp that handles the complexities associated with real speakers playing music. And amplifier designers don't know which speakers are going to be used with their amps. Every speaker presents a different type of "load" to the amp.

In the 1970s there was a big push to lower the "total harmonic distortion" specifications of amplifiers to ever lower levels, based on the belief that if they measured better they would sound better. They were measuring the wrong things. The narrow focus of designing for measurements rarely produces bona-fide sonic improvements. That's not to say that in the hands of truly talented engineers measurements aren't useful, they absolutely are, but measurements are no substitute for listening.

I've heard all of the latest auto setup and calibration systems featured in Denon, Onkyo, Pioneer, Sony, and Yamaha receivers, and the results are hit or miss. Granted, these systems can improve the sound, but more often than not, the processing merely changes the sound. In those cases, I can achieve better results by listening and making adjustments with the manual speaker setup. The processed sound might measure better, but again, that doesn't necessarily correlate to sound quality enhancement.

I recently discussed the measurement quandary with my friend, writer Brent Butterworth, who believes measurements are useful tools, but we never came to a meeting of minds on this matter. I'm paraphrasing here, but he said that measurements that reveal flaws in the sound of a speaker might go unnoticed by the ear, and that some speakers that don't measure well, can still sound subjectively good. So there you have it.

So if trained engineers struggle to derive useful information from measurements, I can't imagine how consumers looking at a wiggly line on a chart helps them decide which amp or speaker to buy. I'm not referring to specifications or numbers like watts per channel or driver sizes; I'm talking about charts and graphs that plot "spectrum of 1 kHz sinewave, D.C. to 1 kHz," or "anechoic response of tweeter on axis." If you have ever picked up useful information from peering at charts in audio reviews, and you're not an engineer, please share your insights in the Comments section.

[moray james]

Amen brother! Lots of folks like to refer to the experts a prime example of this group and one of the best is Floyd Toole formerly of the NRC in Canada a later at Harman. As degreed a fellow as you are likely to ever find in this industry. So tell me just one speaker that Floyd ever designed over his career that is a good one a special one? I agree that measurements can be useful in design they can help you to fast zoom into specifics but they dont tell near as much as we sometimes think that they do. It is very easy to fall into the trap of trusting the measurements and not your ears. Your ears are the only thing to trust if you want to design anything which sounds good or sounds right. Thanks NOSValves good post. Best regards Moray James.

[derrickdj1]

So if trained engineers struggle to derive useful information from measurements, I can't imagine how consumers looking at a wiggly line on a chart helps them decide which amp or speaker to buy. I'm not referring to specifications or numbers like watts per channel or driver sizes; I'm talking about charts and graphs that plot "spectrum of 1 kHz sinewave, D.C. to 1 kHz," or "anechoic response of tweeter on axis." If you have ever picked up useful information from peering at charts in audio reviews, and you're not an engineer, please share your insights in the Comments section.

As with all art, it is hard to quantify unless you are at Christie's and hearing is the ultimate test for speakers, amp, sub and other gear. I put little weight on speaker/sub charts from an anechoic chamber or open field.

[Rudy81]

Outstanding post and one worth objective consideration. Over the years I have learned to take measurements of the systems I have put together as well as the use of Audyssey Pro. Frankly, I don't know if my ears are poor, mediocre, or golden...I just don't know. So, I go with what sounds good to me and then use the available technology to verify what I hear, or to note any major problems that I cannot audibly detect.

Like many of you, I have listened to many systems, Klipsch and many others. My musical taste has shaped what I like and that led me to put together a DIY active system. I totally agree that you should pursue what sounds good to you and it may not measure well with available tools for the enthusiast.

I do agree that designing a product to purely 'test' good on the bench doesn't tell you anything about how it will sound. On the other hand, I have listened to many systems that others have lauded as the best of the best and I have left wondering why I can't hear that system as they do! I suspect a lot has to do with our listening experience and the physical limitations of our hearing. Bottom line in my mind is, go with what sounds good to you and makes you happy when listening to music. It is just that simple.

[tube fanatic]

This debate has been raging ever since the hi-fi industry began and the answer is still not to be found. Ultimately, if something sounds good to someone, then it is good as stated above. Why do some listeners find the sound of an SET with 5% distortion to be totally captivating and the sound of a push-pull pentode amp with .05% distortion to be awful? No answer to this can be found on the test bench. A friend, and fellow amp designer, specializes in exotic designs with very low measurable distortion. Yet, he often scraps many of them in favor of others with much higher distortion because the latter just sound "right." Unfortunately, much of the public (meaning the very tiny percentage of folks who even care about good sound) still has the Stereo Review mentality when it comes to judging equipment. And these days, with the "bricks and mortar" audio dealers being essentially nonexistent, there's no way for people to even hear alternatives to mass marketed audio gear.

[oldtimer]

Art can eventually become science. In sound reproduction it has not to this point. The key is to find what should be measured and how, because the current methods fall far short. We are at the point of the invention of the telescope to look outside our atmosphere, look how far and what different measurements we use to look into space now.

Or maybe it is a fools errand to reproduce acoustical sounds using electronic methods. Mallette has done experimental work in recording using his "will it fool the cat" criterion. Some things will, but I suspect not everything does. Once a sound wave enters space there is an interaction between the two. Reproducing this is at present a herculean task. And amplifiers are not necessarily the hardest part.

So can sound quality be measured? Maybe. But not right now.

[Boxx]

Sure it can.... but only subjectively....

(We measure what we treasure)

[PrestonTom]

Amen brother! Lots of folks like to refer to the experts a prime example of this group and one of the best is Floyd Toole formerly of the NRC in Canada a later at Harman. As degreed a fellow as you are likely to ever find in this industry. So tell me just one speaker that Floyd ever designed over his career that is a good one a special one? I agree that measurements can be useful in design they can help you to fast zoom into specifics but they dont tell near as much as we sometimes think that they do. It is very easy to fall into the trap of trusting the measurements and not your ears. Your ears are the only thing to trust if you want to design anything which sounds good or sounds right. Thanks NOSValves good post. Best regards Moray James

You are way off the mark on this one.

You should actually read his work and not rely on what others say about it. After that, if it does not make sense, then read it again and try and figure it out. Few of us have the knowledge and experience that he has. Dismissing it out of hand is putting you at a real disadvantage.

Wait, I forgot "IMHO"

[moray james]

Tom I wish I knew what Floyd knows but that does not make Floyd a great speaker designer. As far as I know he has however not designed one single significant loudspeaker. If I am off the mark then tell me just one speaker which is considered a true classic a ground breaker designed by Floyd? I am in no way dismissing what Floyd does so don't put your words in my mouth thank you. Listen Floyd must know what he is talking about I expect that he has figured out all of his published papers better than I will ever do so explain to me how this guy who you seem to think has rays of sunshine all around his head has not pulled even one little rabbit out of his hat? It (measurements) is not the holy grail it is imporant and it will I expect some day become very good but not yet and nor near enough you still need a mind attached to a pair of ears with a big dose of imagination. Since Kellog came up the idea of a direct radiator in what 1928 what's the best stuff we have, 50's to 60's still considered by many to be the best , changes since then? Improvement since then? Computers digital science and tada what a disapointment. Sorry Tom. I do think science will figure it all out some day but it is still a long way away and you can get very lost in measurements if you do not understand very well exactly what those measurement are telling you and what they are not telling you. I have been there and done that and like I said the ear has to have the final say still today.Best regards Moray James.

[Ski Bum]

Moray, I can't help but think of the JBL lsr-6332 as a perfect example of Toole's concepts in practice. They are super fine and dandy speakers. Did he design them himself? No. He just wrote the engineering rule book that guided those who did.

[Fjd]

Moray, I can't help but think of the JBL lsr-6332 as a perfect example of Toole's concepts in practice. They are super fine and dandy speakers. Did he design them himself? No. He just wrote the engineering rule book that guided those who did.

Excellent point. I believe that with a PhD in Electrical Engineering, Dr. Toole, rather than working on specific design, spent most of his career researching higher levels of acoustical and psycho-acoustical areas of sound. It seems that much of his research has focused on small-room acoustics and evaluating the relationship between technical measurements of audio equipment and listeners’ perceptions. While designers have applied his findings to loudspeaker design and manufacturing, this application has, in turn, also influenced Dr. Toole’s work.

[Deang]

I agree with Ski Bum. Toole is brilliant. If you haven't at least read this you are missing out on important stuff. The last half dozen pages are very good.

http://www.aes.org/tmpFiles/elib/20130224/5270.pdf

"From the point of view of the sound-quality assessments the evidence suggests that listeners simply preferred the loudspeakers with the fewest technical defects of those that were measured and are known to be audible. In this sense the subjective measurements appear to be neutral. From the point of spatial quality, there seemed to be evidence of a factor relating of early reflected sound to sensations of spaciousness...Listeners with reduced hearing sensitivity tend to be less consistent in their opinions and to exhibit a preference for loudspeakers that may or may not be shared by listeners with normal hearing."

"Conventional listening evaluations are essentially uncalibrated measurements; and they are very often to be in error. A review of published opinion on loudspeaker measurements indicated that not all workers are employing the most sensitive tests. Manufacturers are responsible only for their own products; however, product reviewers have a much wider responsibility. Theirs is a particularly difficult position. Purely subjective reviews are easily challenged unless extraordinary care has been taken in the preparation and conduct of the listening tests. The measurements seen in some publications appear to be a form of ornamentation for review, implying but not confirming analytical ability. Others show fairly comprehensive data, but the interpretations have tended to be descriptive than conclusive."

Good stuff.

So, wide bandwidth with a smooth amplitude response and an even power response.

His work is largely in the area of acoustics, and it has resulted in the release of a lot of really good sounding loudspeakers. The Jubilee would be a good example of applying Toole's ideas to a design.

Now, you can achieve very good audible results with inexpensive drive units and cheap crossover parts. No question about it. However, you will have a better loudspeaker if you use higher quality drive units and other parts. There is something to be said for a higher level of resolving power and refinement in the sound.

Many here love the Cornwall, Heresy, and many of the other loudspeakers that Klipsch makes. Indeed, they all sound very good. However, did you know that you can't match the dispersion characteristics of a round woofer and a rectangular or square horn? It sure doesn't stop them from sounding great though, does it? PWK had it right from the beginning: high efficiency, low distortion, controlled directivity, and flat frequency response - in that order. He might have put an even power response after controlled directivity.

[sunnysal]

There is no doubt in my mind that fanatical membership in either the golden ear or the engineered and measured camps is foolish. the objectivists and subjectivists seemingly will never ascribe any validity to the middle ground. as the OP pointed out music recording and reproduction are highly complex endeavors. The religious fervor always displayed when these topics are discussed never ceases to amaze and amuse me. Toole and many others (including our beloved PWK) recognized the need and validity of engineering and measurement to improve the reproduction chain, they have all IMHO contributed to improving musical reproduction. However I still view reproduction as more art than science at this point, with science helping us get a partial grasp on what is going on but not yet telling the whole story. Add to this the fact that personal taste inevitably plays a huge role in the equation and I am afraid no one will ever agree about this topic. I live as a subjectivist, what sounds good to me sounds good to me, I go for a system that makes music that pleases me with my partucular taste. I in no way reject the objectivist´s search for wasy to measure and improve performance. The "art" of music has, without a doubt, been advanced by the "science" of sound. to argue against that IMHO is foolhardy. as always YMMV. warm regards, Tony

[Deang]

+1 [Y]

Nice to see you in here Tony. How have you been?

[Don Richard]

Measurements bring speed and consistency to product design. Anyone who designs a piece, any piece, of audio equipment without measuring it is not using all of the resources available to him and his designs will likely take longer to develop and perfect. You see frequency response plots on loudspeakers all of the time, but frequency vs amplitude is only half of the story. To be complete a frequency response curve needs to have a frequency vs phase curve to go along with it in order to make sense out of what is going on. When I was triamping my Khorns it took nearly a week to get it set up and sounding right. I used very simple test equipment to align the system and after the basics were done in a day it took several days of adjusting by ear to finish the job. A few months after that a friend who owns a sound reinforcement company came by with his SMAART equipment and ran some tests. The frequency response was really flat but there were phase anomalies, which he corrected with some delay and EQ adjustments. That took about 20 minutes and when he was done the system really sounded great. If I had gotten him to come over at the beginning of the project he could have done the alignment in way less than a day and gotten better results than I got in a week.

After all of this was done I began to notice something. CDs that I thought were bad recordings all of a sudden sounded good. Good recordings sounded more like a live performance than a recording. But a lousy performance that was well recorded became unlistenable.

[Rhetor]

I have enjoyed the read on this thread . . . I gain so much from overhearing conversations of the learned in fields in which my knowledge base nowhere approaches modest . . . I have my own field of expertise, neither sound engineering nor speaker fabrication. But, NOSvalves, what your original post and the subsequent posts have given me is a little bit of permission to do what I have been doing . . . Tweaking with automated settings. You all may be so far down the road to acoustic nuance and fabrication that, perhaps, you do not understand the importance of its influence on the practical, nor your sage conversation on those of us coming late to the sound sojourn. I have a Denon 4311 in my HT, and have collected a cast of Heritage speakers in the same. Much to the chagrin of numerous Audyssey audiophiles, resulting in a modicum of castigation and and forum shame upon myself, I have never found the results of the Audyssey runs to my liking. So, at all times in secret and without public revelation, I now confess to being an Audyssey tweaker, a non-purist who tweaks the results of Audyssey runs in my HT because it sounds better to me. I find the Audyssey results a good starting point to get to a more likable solution. So, I am a beneficiary of the results of the most rudimentary science in av--automated speaker equalization (and all of the underlying math for eq'ing 11 speakers, and a 3 woofer sub). I could never figure that out on my own . . . While my language skills and knowledge of efficacious human communication theory abound, my proficiency in rudimentary math remains suspect. In all, thank you (by means of your post) for giving me permission to employ both the fruit of science (Audyssey) and the fruit of preference (my ears), to, in a very practical real world way, employ what I possess of both (which in either case are woefully unsophisticated) without feeling I am committing some audio cardinal sin in the end, by not only trusting the numbers for permission to figure out what I like to hear. Thanks. Now, I am going back to lurking on this thread.

[NOSValves]

Don,

Where in the original post or anyones comments do you see a statement saying don't measure anything? In the end your friends job was eased by your by the ear adjustments... give yourself a bit more credit. You also have to realize the test equipment could have put a preconceived notion in your human brain that after his minor adjustments the system actually sounds better... the brain has a funny way of getting in the way with some folks...

[Chris A]

Much to the chagrin of numerous Audyssey audiophiles, resulting in a modicum of castigation and and forum shame upon myself, I have never found the results of the Audyssey runs to my liking.

That's been my experience using the basic version of Audyssey that came with my Onkyo PR-SC886 pre/pro, which doesn't allow tweaking of PEQs and dynamic PEQs after the fact. So I used other means: namely a Behringer DEQ2496 and REW with a Behringer calibrated microphone, the resultant GEQ filter settings of which I transformed into my active crossovers (EV Dx38s) via PEQ filters in order to later remove the DEQ2496 from my system entirely. The DEQ2496 allows you to input a desired "house curve" and a pink noise/calibrated microphone auto-EQ setting function. Then I tweaked these settings to my ear's delight and checked the results using REW.

I think that you might have some interest in the following thread based on your above comments:

http://forums.klipsch.com/forums/p/112125/1127557.aspx

When it comes to EQ, I don't trust any machine to do it "automagically". However, these automated devices can get you closer to a good starting point in terms of EQing -- iff your room is heavily damped to start with (at least in my humble experience) and the software allows for tweaking after it runs, allowing the updating EQ, gain, and phase/delay filters by hand.

Chris

[Don Richard]

Where in the original post or anyones comments do you see a statement saying don't measure anything? In the end your friends job was eased by your by the ear adjustments... give yourself a bit more credit. You also have to realize the test equipment could have put a preconceived notion in your human brain that after his minor adjustments the system actually sounds better... the brain has a funny way of getting in the way with some folks...

I was answering the title of the post, "Can sound quality be measured?" Certainly with today's technology what we measure correlates more closely than ever with what we hear. As far as loudspeakers go, a Bode plot of frequency and phase coupled with spectrum analysis of distortion correlates quite well with what we hear. That was the point I was trying to make.

[Ski Bum]

I was answering the title of the post, "Can sound quality be measured?"

Guttenberg may be well meaning in what he writes, and he seems to be enjoying himself, but he's clearly an unsophisticated doofus at best, and an enabler of the woo-hucksters at worst. Good work, if you can get it, I suppose.[8-)]