Header

Objective Reviews & Commentary - An Engineer's Perspective

April 7, 2012

What We Hear

mcgurk ear bbc twoFIRST SOME UPDATES: This is a promising sign. I’ve managed to publish another article in less than a week and it’s a big one! Since last weekend’s April Fool’s article we’ve managed to finalize the ODAC design and it’s off to production. The best case estimate for availability is mid-May assuming all goes well. I plan to publish an article with the full measurements in the next few weeks. For now, I need a temporary break from the ODAC. (photo: BBC Two)

WHAT WE HEAR: The recent review of the O2 Amplifier at Headfonia has generated questions about why the reviewer heard what they heard. For example, a few have asked if the O2 is genuinely a poor match for the Sennheiser HD650 as the reviewer claims. How we do we determine if that’s true and, if the review is wrong, what happened? Others have asked if “colored” audio gear can sound better. This article tries to explore what’s often wrong with typical listening comparisons, subjective reviews and “colored” amplifiers.

THE SPECIFIC COMMENTS IN QUESTION: I’m not trying to place unique blame on Headfonia, or the authors, it’s just useful to have a concrete example. The Headfonia review is a timely example. The authors (Lieven and Mike) wrote:

I really don’t find the Senns [HD650] to pair well with the O2”. He also said the O2 (presumably with the HD650) was: “ …missing some punch and impact. But the bass isn’t the only thing bothering me actually, there also is something bothering me with the mids . They sound a bit thin to me to be honest, I just feel like I am missing out on something using the O2. It isn’t bad sounding, and I’m sure lot’s of people love it, but I’m not one of them, I’m missing something to enjoy what I’m hearing.”

HUMAN PERCEPTION & SCIENCE COLLIDE: Just because the earth seems flat from where you’re standing doesn’t make it true. Our senses don’t always provide us with the real truth. This article is mainly about how all of us, even audio reviewers, perceive things and why.

wine heatheronhertravelsTWO BUCK CHUCK (part 2): Robin Goldstein conducted over 6000 wine tastings with over 500 people including many wine experts who work in the industry. He discovered when people know the prices of the wines they clearly prefer more expensive wines. But when the the same wines are served “blind” (anonymously), the votes reverse and they express a strong preference for the least expensive wines.

FAKE RESTAURANTS & BUG SPRAY: Goldstein exposed one of the most prestigious sources of wine ratings in the world, Wine Spectator, as arguably having their awards up for sale. He created a fictitious restaurant with a wine list featuring poorly reviewed wines with descriptions like “bug spray” and “barnyard”. Then, after paying the appropriate fee, his fake restaurant managed to score a Wine Spectator Award of Excellence. Goldstein argues organizations like Wine Spectator do not provide the sort of unbiased advice many anticipate. He also concluded their ratings have a heavy commercial and financial component the public is largely unaware of. It’s alarmingly analogous to high-end audio including the questionable awards. For those into podcasts check out the NPR story on Goldstein and wine myths . He’s also written a book on the subject called The Wine Trials. (photo: heatherontravels)

TASTE WRONG: How can the same people, taste the same wines in two different tastings, and rate them completely differently? The is our senses are not nearly as reliable as we think. In this case what’s known as expectation bias caused the wine tasters to perceive the wines differently. They expected the more expensive wines to taste better so their brains filtered their perception of taste to help deliver the expectation. But when they didn’t know anything about the wines, they couldn’t have any expectations, hence the bias was removed. I’ll explain more shortly but this is also critical to what we hear.

SEE WRONG: Studies show our vision is easily fooled. A video linked in the Tech Section contains a classroom experiment where students witness what they believe is a real a crime. They’re shown pictures and asked to identify the suspect and many are so sure they even would testify in court. But, alarmingly, there’s no consensus in the photos they choose. In the end, none of them are correct! Their brains filtered out too many details of the thief. When shown pictures, each of their brains helpfully invented different missing details to match one of the photos. We do the same thing with audio (see the Tech Section).

justinstolle baby with headphonesHEAR WRONG: Just like with taste and vision, our hearing is heavily and involuntarily filtered. Only around 0.001% of what we hear makes it to our conscious awareness. This is hardwired into us for survival. We would go crazy from sensory overload if this filtering didn’t take place 24/7. Studies show you can play the identical track, on identical equipment, two different times, and the odds are good people will hear significant differences if they’re expecting a difference. This is called expectation bias and it’s the brain “helpfully” filtering your hearing to match your expectations—just like wine tasting. I reference the auditory work of James Johnston and more in the Tech Section. (photo: JustinStolle CC)

THINK YOU’RE EXEMPT FROM HEARING BIAS? People who have made a career of studying the human auditory system have yet to encounter anyone who can overcome their sensory bias. If you have any doubt I challenge you to watch this video and try to overcome your listening bias. If you can’t reliably do it here with a single syllable, even when you know the exact problem, it’s safe to say you don’t have a chance with the much greater complexity of music and a wide range of expectations:

  • McGurk Effect Video – A 3 minute BBC non-technical BBC video that allows anyone to experience how our brains involuntarily filter what we hear based on other information. (courtesy BBC TWO).

SKILLED LISTENING: You can learn what to listen for and hear more subtle flaws. Studies have shown show recording engineers are especially good at it—even better than hardcore audiophiles and musicians. Recording engineers make their living listening for that one clipped note. But they’re still not exempt from expectation bias.

studio mixer by rachelledravenRECORDING ENGINEERS FAIL: Ethan Winer, something of a recording studio expert, points out even recording engineers get caught making adjustments to a mix, and hearing the desired result, even when the controls were not even active. Nothing changed with the sound, but their brains deceived them into hearing the expected changes. And in blind listening tests, just like everyone else, they hear differences that don’t exist if they’re expecting differences. (photo by; RachelleDraven CC)

THE TIME FACTOR: Our brains are like leaky sieves when it comes to auditory details. We can’t possibly remember the massive amount of data contained in even one song (roughly equivalent to one million words!). So, here again, our brains out of necessity heavily filter what we remember. The students in the sight example above thought they had accurate memories of the thief but they turned out to be very wrong. The science shows our audio memory starts to degrade after just 0.2 seconds. So if switching from Gear A to Gear B requires more than a fraction of a second you’re losing more information. The longer the gap, the more you forget and the more bias distorts what you do remember.

WHAT WE HEAR: Neurologists, brain experts, hearing experts, and audio experts, all agree the human hearing system, by necessity, discards around 99.99% of what arrives at our ears. Our “reptile brain” is actively involved determining what gets discarded. For example when you’re listening to someone at a noisy restaurant, the brain does its best to deliver just their voice. And, multiple studies demonstrate, when you’re listening to audio gear the brain also does its best to filter your hearing in the way it thinks you most want. If you’re expecting Gear A to sound different from Gear B the brain filters each differently so you indeed hear a difference even when there isn’t one. This auditory issue has many names. I generally call it “Subjective Bias” but it’s what’s behind “Expectation Bias”, “The Placebo Effect”, and “Confirmation Bias”. I highly recommend the book Brain Rules by John Medina and there are more resources in the Tech Section.

hd650 closeupBACK TO HEADFONIA’S REVIEW: Knowing the O2 measures well creates an unavoidable bias that it might be too “sterile” which implies, as Lieven said, it could be “missing something” or sound “thin”. His brain involuntarily, when he knows he’s listening to the O2, would filter his hearing to deliver the expected “sterile” sound. Someone could set up a blind test between the O2 and a measurably transparent amp Lieven really likes, and he would no longer be able to tell the amps apart because his brain wouldn’t know what to expect. Conversely, you could set up a test where Lieven compares the O2 in a sighted test to a more favored amp but in reality he’s always listening to the O2. Because he’s expecting to hear differences he would--even though he’s really listening to the same amp. He would hear what he expects from each amp. This isn’t unique to Lieven or Headfonia. We all suffer from this bias.

MEASUREMENTS PROVIDE REAL ANSWERS: The O2 performs, as demonstrated by measurements, very similarly with just about any headphone load and is audibly transparent with the HD650. There’s nothing about the HD650 that alters the signal delivered by the O2 in an audible way. It would be useful to measure the other amplifiers used for comparison in the review. It’s possible one or more are less accurate than the O2 which might create an audible difference. See the Tech Section for more.

A BETTER METHOD: Instead of pretending we’re somehow exempt from inevitable filtering and expectation bias, a much better approach is to simply remove the expectations. Odds are extremely high if the authors had used level matched blind testing they would have reached very different conclusions. This can even be done solo with an ABX switching setup allowing instant switching to stay within the critical 0.2 second window. The switching can be done frequently, hours apart, or over weeks. You can listen to 10 seconds of a revealing excerpt or 10 entire CDs. In the end, the ABX set up reliably indicates there’s an audible difference. If a difference is detected, an assistant can be used to help blind audition both and determine which is preferred.

SUBJECTIVE REALITY: I’ve done the above blind testing using my own HD650 headphones and Benchmark DAC1 Pre. The O2 sounded so similar to the $1600 Benchmark I, and another listener, could not detect any difference. Simple logic tells us if what Lieven wrote about the O2/HD650 is true, it would also likely hold true for the Benchmark DAC1 Pre/HD650 or indeed any sufficiently transparent headphone amp. Considering the DAC1 is a Stereophile Class A (their highest rating) headphone component and rave reviewed by numerous other high-end audiophile reviewers, there seems to be a serious disconnect somewhere. At least one of the reviewers must be wrong and the Headfonia authors are far outnumbered.

ponder by striaticA REVIEWER’S DIFFICULT CHOICE: Subjective reviewers, like Lieven, have three basic choices in how they want to view a piece of gear like the O2 that measures very well: (photo by: striatic CC)

  1. Try to wrongly claim all transparent gear sounds different than other transparent gear and make up weak excuses to avoid blind testing demonstrating what differences they can really hear.
  2. Try to claim non-transparent (i.e. “colored” lower fidelity) gear with added distortion is somehow better than transparent gear. This would condemn all transparent gear, even some really expensive gear, to being equally “inferior” as the O2.
  3. Concede all audibly transparent gear is accurate and generally will sound the same. They don’t have to like the sound of transparent gear, but they would accept giving the O2 a negative review means also giving a Violectric, DAC1, and lots more high quality headphone amps an equally negative review.

seed pods by ticoWHEN THERE REALLY ARE DIFFERENCES: Sometimes gear really does sound different for some good reasons. I provide examples in the Tech Section. The Headfonia review, and related comments, have argued lower fidelity “colored” amps are somehow better than accurate high fidelity amps like the O2. This is often called “synergy” or “euphonic distortion” and it’s a matter of personal taste but it does have some very significant downsides. (photo by: tico cc)

TWO WRONGS DON’T… The synergy theory goes something like: “Because headphones are imperfect it’s OK to pair them with imperfect amps that help compensate for the headphone’s flaws.” Someone might pair bright headphones with an amp having poor high frequency response to help tame the nasty highs. But what happens when you upgrade to say the HD650 with relatively smooth natural highs? Now your headphone amp is going to sound dull and lifeless and there’s no way to fix it. The generally accepted goal, for many solid reasons, is the entire audio reproduction signal chain should be as accurate and transparent as possible right up to the headphones or speakers. That’s what recording engineers who mix our music, and manufacturers of our headphones and speakers, are counting on. If you want different sound see the next paragraph.

foobar eqAMPS & DACS MAKE LOUSY EQUALIZERS: If you want to alter the sound a much better method than using a colored amp or DAC is using EQ. That’s what it’s for. There are even decent EQ apps for iOS and Android. For those purists who somehow think DSP or EQ is wrong, odds are your favorite music has already had significant DSP and/or EQ applied. There’s an example in the Tech Section showing even two passes of digital EQ are entirely transparent. Using a headphone amp to compensate for a flaw with your headphones is like buying a music player that can only play one song.

NOT SO EUPHONIC: Amps or DACs that contribute what’s often called “euphonic distortion” are a lot like using them as equalizers above. An amp with “euphonic distortion” might help mask other problems by adding audible crud to your music but that’s putting a band aid on the problem instead of fixing what’s really wrong. You’re stuck with the added crud from euphonic gear no matter what. Even with the most pristine audiophile recording with low distortion headphones, your gear is still going to dumb down your music with its ever present contributions. If you’re really craving say the sound of a vintage tube amp, it can be done rather convincingly in software via DSP. That way you can at least turn the crud on and off as desired.

IT’S A PACKAGE DEAL: Sadly most “colored” and “euphonic” audiophile gear also has other problems. To get that soft high end you’re after you might end up with a tube amp and also get a very audible 1% THD. Or, like some Schiit Audio products have done, your “esoteric” amp may destroy your expensive headphones. Gear with lower fidelity in one area is much more likely to have problems in other areas.

FIDELITY DEFINED: The “Fi” in “Head-Fi” is supposed to be fidelity, as in “high fidelity”. If you look up the words, you’ll find it means “being faithful”, “accuracy in details” and “the degree to which an electronic device accurately reproduces sound”. Note the words “accuracy” and “electronic”. Accuracy is something that can be measured in most electronic audio gear (DACs, amps, etc.). So if “high fidelity” is really the goal why would anyone be opposed to measurements that support that goal? Ask yourself that question next time you encounter someone dismissing or downplaying audio measurements.

magic by cayusa 275MAGIC: Some compare the supposedly better sound heard from high-end gear to “magic”. Lieven quotes Nelson Pass claiming “It is nevertheless possible to have a product that measures well but doesn’t sound so good. It is still a mystery as to how this could be.” In effect Pass appears to be saying amplifier sound is mystical. But it’s not. Lots of credible published research demonstrates measurements predict sound quality. If Pass is so confident measurements don’t tell the whole story, where’s his published blind listening test demonstrating his view? Such a result would certainly help sell more of his five figure products so he has a strong incentive to publish such a test. Still, magic is an interesting analogy. Magicians know how to exploit weaknesses in our senses to deceive us. Convincing someone a $2000 power cord really sounds better is a similar act of sensory deception. (photo by: Cayusa CC)

BUT IT SOUNDED BETTER EVEN IN THE NEXT ROOM! Anyone who’s read many of the blind vs sighted debates has probably come across someone claiming their wife even commented how it sounded better and she was three rooms away in the kitchen. That phenomena was explained by Stuart Yaniger in his recent article Testing One Two Three. It turns out many audiophiles have a few favorite test tracks they cue up after making changes to their system. The test tracks are an obvious clue to their spouse they probably changed something in their system. And, spouses being spouses, sometimes respond with the audio equivalent of “that’s nice dear”. Sit down and do a blind comparison if you really want to know the truth—preferably in the same room as the speakers.

IF THIS IS ALL TRUE WHY DON’T MORE PEOPLE KNOW OR TALK ABOUT IT? The biggest single reason is there’s an enormous amount of money behind high-end audio. It’s largely built on the belief that spending significantly more money on high-end gear yields better sound. Pick up nearly any magazine, or visit nearly any website, and you’ll read how a $3000 DAC sounds better than a $300 DAC. But is that really true or is it just a self-serving perception that exploits the weaknesses in our auditory abilities? The industry has carefully cultured and crafted a lot of perceptions over the last few decades and some of it has been organic—especially given how audio forums work.

SOME IMPORTANT HISTORY: High-end audio was nearly non-existent in the 1970s short of a few companies like McIntosh and even those marketed their gear around objective criteria. If anyone tried to sell a $2000 power cord in 1980 they would have been laughed out of business but now there are dozens of companies selling them. The wine industry has followed a very similar path. In 1980 people tended to buy wine they thought tasted good. But, courtesy of decades of clever marketing, wine has been made out to be much more complex and important. An entire industry in the USA has grown up around wineries, wine tourism, wine bars, tastings, pairings, flights, expert reviews, etc. Consumers have been taught to doubt their own ability to judge wine and instead trust expert wine ratings with their point scores and elaborate tasting notes. And, not surprisingly, the experts tend to rate more expensive wines much higher. It’s scary how similar it is to high-end audio. Compare “the Warped Vine Cabernet was leathery with hints of green pollen” to “the UberDAC had a congested sense of pace and tarnished rhythms”. Check out the books by Robin Goldstein, Mike Veseth, Jonathan Nossiter, and the documentary Mondovino.

CONDESCENDING LINGO: Like the wine industry, the audio industry has invented their own complex secret language. This is exposed as a clever marketing technique in the books referenced above. The vague complex language makes the average person feel they’re not qualified know what to buy and should defer to the “experts” (who in turn promote more expensive products). So next time you read about slam, pace, congestion, rhythm, blackness, sheen, timing, or other words that could have many meanings, consider it marketing mumbo jumbo and think twice about trusting anything else that person has to say. Misappropriating words to mean something that’s ill-defined and vague isn’t doing anyone any favors except those hawking overpriced gear.

question doubt by mister khaCREATING DOUBT:  Someone might have a favorite inexpensive wine but the wine industry often succeeds in making them wonder if it’s “good enough” to serve to guests. They’re tempted to ignore their own perceptions and instead buy something that’s well rated and more expensive. This is good for the wine industry but bad for consumers. (photo by: Mister Kha CC)

AUDIO DOUBT: The high-end audio industry mirrors the wine industry. Websites like Head-Fi create massive amounts of doubt among those who spend much time there. I’ve received hundreds of messages like:

“I have the $200 UltraCan Five headphones and I thought they sounded really good with my iPod. But then I found Head-Fi and read the new $500 UberCan Golds are a lot better but they apparently need at least the $500 UberAmp Titanium to really perform. Should I spend $1000 to upgrade?”

SAD STORIES: I read messages like the one above and it makes me sad. Here’s this college student, probably eating macaroni and cheese for dinner, contemplating spending $1000 more than they’ve already spent to listen to music on their $200 iPod? And this is from a person who, until they found Head-Fi, thought they already had really good sound! In my opinion it’s just wrong—especially when the doubt is largely based on flawed listening perceptions and often hidden commercial influences.

BEHIND THE SCENES AT HEAD-FI: The above is the marketing genius of sites like Head-Fi and Wine Spectator. This all plays into the Head-Fi recipe:

  • Censorship – Head-Fi bans discussion of blind testing on 98% of the site and they have censored or banned members who speak out against Head-Fi or their sponsors. They take many actions to heavily tilt perceptions in favor of visitors spending money they probably don’t need to spend.
  • Questionable Gear - Some of the most hyped gear on Head-Fi, in my opinion, is snake oil—Audio-GD, NuForce and Schiit Audio are three examples arguably based on hype rather than solid engineering. Some of it is sold through very limited sales channels and Head-Fi is probably responsible for the majority of the sales. 
  • Massive Advertising - Head-Fi is a banner farm of advertising that includes $1000 iPod cables. The advertisers want to capitalize on the doubt created among Head-Fi visitors and having the moderators and admins acting on their behalf.
  • Infomercials – Jude, and others at Head-Fi, produce what many consider “infomercials” promoting products for Head-Fi sponsors. When Wine Spectator granted the Award of Excellence to Robin Goldstein’s fake restaurant they suggested he also buy an ad on their website. Those sorts of cozy behind the scenes relationships are present in the audio industry as well. See Stereophile Exposed below.
  • Biased Evangelists – A disproportionate number of Head-Fi posts are made by what some consider shills who “evangelize” and fiercely defend products sold by Head-Fi sponsors. Some might be just rabid fanboys, but if you look at the products that receive the most positive, and greatest number of, comments they’re usually from Head-Fi sponsors and are often somewhat questionable products that probably wouldn’t do nearly as well left to their own (lack of) merits.
  • Deception – To the casual visitor, Head-Fi just seems to be an open forum where average people can talk about headphone stuff. But it’s really owned by a commercial entity that’s using it as a marketing machine. And it’s not “open”—see Censorship above. It’s a private forum run for profit. From what I understand the moderators and admins receive a steady stream of free gear while Jude receives all that plus serious income. It’s no longer just a hobby to these guys.

SAFE MYTH BUSTING: There’s a great show on the Discovery Network called MythBusters. They pick a few myths for each episode and set out to see if they’re true. To my knowledge they nearly always avoid busting myths that might cause serious financial harm to anyone and instead focus on things like toilets exploding when used by cigarette smokers. This same “courtesy” is almost universally extended by the media covering high-end audio. The past has shown crossing the line is dangerous.

the audio criticTHE AUDIO CRITIC: Peter Aczel set out many years ago to expose myths in audio with his magazine and website The Audio Critic. He had a relatively small but significant and loyal following. But, progressively and sadly, he was marginalized in a variety of ways. Today nearly all audio journalists are inexorably in bed with the manufacturers in one way or another. And if you’re openly pro-Obama you don’t get to work on Mitt Romney’s political campaign. (photo: The Audio Critic)

STEREOPHILE QUESTIONED: For those who think I have an axe to grind, it’s nothing compared to Arthur Salvator’s axe. He strongly questions the credibility of John Atkinson and Michael Fremer at Stereophile. Whatever his personal bias, he presents what appears to be factual data implying Stereophile has heavily favored advertisers and selling magazines over being objective audio journalists. Sadly, as with Wine Spectator’s Excellence Awards, I believe this is likely to happen whenever you can follow the money from their paychecks up the chain to the manufactures of the products they’re supposed to be reviewing.

DECEPTION RUNS DEEP: I used to work in high-end audio. I can attest many believe the “party line” as they’re more or less immersed in it. It’s much easier to dismiss the rare blind listening test than consider their own industry is largely based on erroneous myths. Believing the earth is flat allows them to sleep better at night. So keep in mind even those selling $2000 power cords might genuinely believe their product sounds better. They likely listened with the same expectation bias described above and hence are believers. But that doesn’t mean lots of expensive gear really does sound better.

THE GOLD STANDARD: Across many branches of science blind testing is the gold standard for validating if things work in the real world. It’s well accepted for use where human life is at stake in medicine, forensics and physics. So it’s stunning when audiophiles claim it’s an invalid technique with mountains of evidence to the contrary dating back to the 1800’s. The Tech Section addresses all the common complaints about blind testing. But the short version is there are no practical objections when you do it right.

BLIND AVOIDANCE: Perhaps the most interesting disconnect in the industry is the almost universal avoidance of blind testing. If someone really believes their expensive gear sounds better why not try a blind comparison? This is where things get fuzzy about what many in the industry really believe. I would think many would be genuinely confident and/or curious enough to at least try some blind testing. But, almost universally, they refuse to participate in blind tests and often actively attack blind testing in general. Why? Deep down is it because they’re worried the expensive gear, cables, etc. really don’t sound better? Did they secretly try a blind test and failed to hear a difference? Or have others convinced them blind testing is a total sham and the earth is really flat?

BLIND DONE WRONG: When blind testing is used by someone in the industry they arguably misuse it and retain the very expectation bias blind testing is supposed to remove. Some examples of “blind abuse” are in the Tech Section.

dream by xtream_iFANTASIES: Another way to explain the avoidance of factual information is some simply enjoy the fantasy. High-end audio is a serious hobby for many. For some constantly swapping out gear is a big part of the fun. Blind testing could, obviously, jeopardize some of their enjoyment. One person’s hobby is their own. But if I imagine wine tastes better when consumed from vintage Waterford crystal should I really be trying to convince others to get rid of their pedestrian wine glasses and spend a lot more on vintage crystal because the wine tastes better? This is exactly what most audiophiles are doing at sites like Head-Fi, Headfonia, and many others. They’re convincing others to spend more money with the promise of better sound when they’re often not getting better sound.

FANTASIES ARE NOT ALWAYS HARMLESS: Some have told me I’m pulling the beard off Santa Claus and I should leave deluded audiophiles to their harmless fantasies. But the fantasies are not harmless when, collectively, the often repeated myths are costing people tens and probably hundreds of millions of dollars. Not everyone buying high-end gear has money to burn and reads the Robb Report. Many just want better sound. They don’t need $1000 DACs, power conditioners, expensive cables, etc. to get the sound they’re after yet they’re told over and over they do need those things.

crystal by foolfillment 275WATERFORD CRYSTAL: I think most would agree the difference between regular glass and crystal doesn’t change the actual taste of wine but it’s fair to say it might change the overall experience. There’s nothing wrong with buying and using expensive crystal wine glasses. But I believe it’s crossing an important line when you try to convince others the crystal itself makes the wine actually taste better. (photo by: foolfillment CC)

MONEY WELL SPENT: I’m not suggesting there’s no reason to buy expensive audio gear. I own a $1600 DAC. There can be satisfying reasons to spend more including higher build quality, more peace of mind, the heritage and reputation behind certain manufacturers, better aesthetics, and other less tangible benefits. But, for the sake of those interested in getting the best sound for their money, it’s best not to confuse sound quality with those other benefits. Just as drinking from crystal doesn’t improve the actual taste of the wine itself. Buying a pre amp with a massive front panel and $2000 power cord doesn’t improve the sound even if it might improve the experience for some. I’ll be covering this in much more detail soon in my upcoming Timex vs Rolex article.

ONE MAN’S BOOM IS ANOTHER MAN’S BASS: Perhaps the most accurate headphones I own are my Etymotic ER4s but they’re also among the least used. Why? They’re just not very exciting to my ears and are ruthlessly revealing of poor recordings. I want to be clear nobody can dictate what’s best for someone else when it comes to headphones, speakers, phono cartridges, and signal processing (including EQ). All of these things alter the sound to varying degrees so it comes down to individual preferences as to which is best. But, to be clear, I’ve also presented good reasons why it’s worthwhile for everything else in the signal chain to be as accurate and transparent as possible—that is to have the highest fidelity. Just because I like say a bit of bass boost doesn’t mean I should go buy a headphone amp with a high output impedance that makes my HD650’s sound more boomy. That same amp would be a disaster with my Denon D2000s, Etymotics or Ultimate Ears. It’s much better to choose a headphone with more bass, like my DT770, or use some EQ to punch up the bass to my liking. Then I can use my headphone amp with any headphone and I can have my cake and eat it too.

WHERE’S THE LINE? I can understand wanting to own high quality equipment. But where do you draw the line between subjective enjoyment and blatant snake oil and fraud? Is a $3000 “Tranquility Base” with “Enigma Tuning Bullets” from Synergistic Research more like Waterford crystal or a magnet to clamp on the fuel line in your car claiming to double your gas mileage? What about Brilliant Pebbles--a $129 (large size) jar of rocks that are supposed to improve the sound of your audio system? These are example companies presented in the myths video (Tech Section). Should such products be able to stand up to blind testing? Does it matter if the claimed benefits are real? To me it’s a slippery slope once you start agreeing with the “blind tests don’t work” crowd as the possibilities for unverifiable claims are endless.

TRUTH IN ADVERTISING: Many countries have laws against false claims and sometimes it’s enforced via legal action. For example, companies claiming unproven health benefits for products answer to the FDA. And Apple is currently being sued for misrepresenting Siri’s capabilities in their TV ads. Should audio companies be held to similar standards or at least be required to use a disclaimer when claims are unproven? When do claims enticing people to spend lots of money become fraudulent?

THINGS TO CONSIDER: If this article has you wondering just what’s real and what isn’t I would suggest considering some of the following:

  • Focus On What Matters To You – If you only want the best sound for your money don’t let others make you doubt your own enjoyment or convince you to spend more than necessary for things you can’t hear or don’t need. Likewise if you want a headphone amp that you can hand down to your kids in 20 years don’t let someone like me keep you from buying a Violectric V200 instead of an O2 even if they sound the same. The Violectric is better made. Of course your kids may have audio brain implants in 20 years.
  • Your Tastes Are Your Own – Subjective reviews are just that: Subjective. Don’t assume you will hear what a reviewer claims to have heard. We all hear differently, have different preferences, etc.
  • Music is Art but Audio is Science – Don’t confuse the art of being a musician with the science of reproducing recorded music. What an amplifier looks like is art, but what it does with an audio signal is entirely explained by science.
  • Use Common Sense – Be aware the controversy over blind testing is almost like a religious war. But try to step back and use common sense, consider the source, references, credible studies, etc. Follow the links in this article to additional resources. Consider the huge wealth of scientific evidence in favor of blind testing, measurements, what matters, etc. vs the subjective claims with little or no objective evidence of any kind to back their claims. If differences between cables, power cords, preamps, etc. were really so obvious don’t you think someone would have set up a credible blind test demonstrating the benefits by now?
  • Consider The Source – When you’re reading about gear, testing, reviews, etc, always consider the source. What gear does the author own? Who do they work for? Where does their (and/or their employer’s) income mostly come from? Are they selling audio gear? Do they receive free gear? Do they otherwise have a vested interest in promoting the status quo? I’m not saying everyone is hopelessly biased by those things but they often contribute bias to varying degrees (see Stereophile Exposed above and the wine references).
  • Try Free ABX Software – This will allow you to conduct your own solo blind tests comparing say a CD quality 16/44 track with a 24/96 track. See the Tech Section for more details.
  • Enlist A Friend (with a good poker face) – If you’re convinced that new gear sounds better find someone else to randomly connect the new gear or old gear in a way you don’t know which you’re listening to. Keep everything else as consistent as possible including the volume, music used, etc. See if you can identify the near gear. If you do, repeat the experiment several times keeping score of your preferences while your friend keeps score of which one was really connected. Compare notes when you’re done.
  • Eliminate Other Variables & Clues – If you’re trying to compare two pieces of gear do everything you can to eliminate anything that might cause you to suffer expectation bias or provide even subtle clues as to which is which. Let’s say someone is plugging your headphones, behind your back, randomly into source A or source B. You need to make sure the sound the plug makes going into the jack isn’t different enough to be recognizable. Also make sure the music is the same, not time shifted, one player might click when starting or has audible hiss and the other doesn’t, etc. Make sure the person helping either have a poker face or stays out of sight and doesn’t say anything as they may provide clues otherwise. And if you suspect someone else is hearing differences that likely don’t exist look for these kinds of clues that might be tipping them off—even subconsciously.

OTHER RESOURCES AND REFERENCES: In addition to all the links sprinkled around this article, check these out for more information:

  • Subjective vs Objective Debate – My first attempt at tackling some of this last spring
  • Music vs Sine Waves – An article I wrote debunking some myths about audio measurements.
  • Headphone Amp Measurements – An article I wrote for InnerFidelity on amp measurements
  • 24/192 Music Downloads – High resolution 24 bit music has to sound better, right? Or does it?
  • Matrix Blind Audio Test – A relatively simple and revealing blind test that’s easy to understand
  • Audio Myths Workshop Video – This is a somewhat technical presentation but it has lots of simple common sense examples. See the Tech Section for more details and how to obtain the high quality example files used in the second half to listen for yourself.
  • Why We hear What We Hear – A technical article on the human auditory system by James Johnston that goes into more detail about what we hear.
  • Dishonesty of Sighted Listening – An excellent technical paper by Sean Olive on evaluating speakers with blind vs sighted listening.
  • Hydrogenaudio – A geekfest of mostly objective audio folks. Lots of blind tests, analysis, and more.
  • Audio Differencing – Consider trying out the free software from Liberty Instruments and their test files. Do capacitors really change the sound?
  • Science & Subjectivism in Audio – An information packed page by Doug Self covering this topic.
  • Meyer & Moran Study – Over 600+ trials even audiophiles couldn’t hear a difference between CD and SACD high resolution audio.

THE FUTURE: Look for my “Timex vs Rolex” article where I’ll cover what you get as you spend more, what to look for based your priorities, etc. I’ll also be comparing some typical gear at widely different price points and exposing some high-end gear with low-end performance.

BOTTOM LINE: We can’t “trust our ears” the way we’re usually led to believe we can. And we can’t trust the ears of reviewers for the same unavoidable reasons. I’m not pretending I can write a few articles and change the minds of thousands of audiophiles—others before me have tried. But even if I only help a few save some serious money by focusing on what’s real, I’ve succeeded. Next time you encounter someone you think might be wrongly arguing for spending more money in the interest of better sound, consider providing a link to this article. If they read it and decide the earth is really still flat, or they want to spend more anyway, it’s their choice. But at least the information helps anyone who’s interested make a more informed decision.

 


TECH SECTION


INTRO: Below are further details on some of the references I used for this article and some added information for the more technically inclined.

BLIND OBJECTIONS: There are several typical objections to blind testing. But nearly every one has been well addressed. They include:

  • Switchgear Masking – Some try to argue the relays or switch contacts, extra cabling/connectors, etc. associated with some blind testing somehow masks the differences between gear. But there have been many blind tests without any extra cables or gear. See the Matrix Audio Test as one example where humans simply swapped cables. What’s more, the music we listen to has already been “masked” by all the unseen cabling and switchgear used in producing music. A lot of high end and well respected gear is full of switches and relays. Objection removed.
  • Listening Stress – Some blind tests are conducted by others administrating the test and listeners may feel some pressure to “perform”. But an ABX box can be used solo, in their own home, under exactly the same conditions they normally listen. Objection removed.
  • Listening Duration – It’s often said you can’t appreciate the differences between gear by listening to just one track or for just a short period of time before switching. An ABX box can be used at home for whatever duration desired. You can listen to Gear A for a week, then Gear B for a week, and finally Gear X for a week before casting your vote. That could go on for as long as you want. Objection removed.
  • Unfamiliar Environment – Blind tests are often criticized for being conducted in an unfamiliar environment. Never mind audiophiles can pop into a booth or room at a noisy trade show and claim to write valid reviews of how the gear sounded. But it’s entirely possible to do blind testing in your own home, with your own gear, your own music, at your own leisure, etc. See above. The Wired Wisdom blind test, for example, was done in the audiophile’s homes. Objection removed.
  • The Test System Wasn’t High Enough Resolution – See above. Blind tests have been done with audiophile’s personal $50,000 systems and the results are consistently the same. Objection removed.
  • Statistically Invalid – There have been some tests with marginal results that leave room for the glass half empty or glass half full approach as to their statistical significance. But there have been many more that have had a valid statistical outcome over many trials. The math is clear and well proven. Objection removed (for well run tests with clear outcomes).
  • Listening Skill – Some claim blind tests fail because the listeners involved were not sufficiently skilled. But it’s well established the filtering and expectation bias of our hearing is involuntary—no amount of skill can overcome it. It even trips up recording engineers. Ultimately, if say a dozen people cannot detect a difference, what are the odds you can? Objection largely removed.
  • Flawed Method – It’s always possible for someone to find flaws with the method. It’s like a candidate losing a political election, demanding a ballet recount, and finding a dead person cast one vote—never mind they need another 7032 votes to win. They want to try and invalidate the election on an insignificant technicality for entirely biased reasons. The same audiophile claiming to listen to a piece of gear for 5 minutes at a trade show and have a valid opinion about how it sounds somehow claims noise on the AC power line masked all differences in a blind test. Really?
  • It’s Not Real Proof – This is the ultimate weapon for the blind critics. Anyone can always claim something like “it’s possible when rolling dice to get double sixes many times in a row so your test doesn’t prove anything.” And, technically, they’re correct. But, in practice, the world doesn’t work that way. If they really believe in beating impossible statistical odds, they should be spending their time in a Las Vegas casino not writing about blind testing. Drug manufactures are not allowed to tell the FDA it’s possible their drug is really better than the placebo even if the statistical outcome of their carefully run trials indicates otherwise. There are well established standards that are accurate enough to use where human life is at stake but some audiophiles still, amazingly, try to claim these methods are somehow invalid.

WHEN BLIND IS FLAWED: Some in the industry, and the odd individual, have arguably misappropriated blind listening. For example, the UK magazine Hi-Fi Choice conducts what they call “blind listening group tests”. But instead of following proven and established guidelines for blind testing to determine if the listeners can even tell two pieces of gear apart, they seem to let their judges listen to gear as they choose over multiple listening sessions and take notes on their impressions without presumably knowing which gear is playing. This method suffers from much of the same filtering and expectation bias as sighted listening as they’re expecting differences so they hear differences. As far as I can tell (the published details are vague), they’re not directly comparing two pieces of gear looking for statistically significant differences over several controlled trials. There doesn’t seem to be any voting or scoring to determining what differences are real versus simply a result of expectation bias. Studies show one might blow their entire method out of the water by simply running their usual test with say 5 different amplifiers but, instead of playing all 5, repeat one of them twice and leave one out. I’ll bet you serious money they would hear all sorts of differences between the two sessions with the same amplifier playing. This has been done in blind wine tasting and some of the greatest perceived differences were between identical wines poured from the same bottle (check out the wine tasting podcast interview).

UNLISTENABLE: This guy claims to have done a blind test with 5 people and 10 trials each between two pieces of gear. He further claims “Every listener described the Halcro as being 'unlistenable'”. I’m sorry but if something is so bad as to be “unlistenable” do you really think you need a blind test to figure that out? That’s like comparing a good Cabernet to drinking vinegar. Also, for what it’s worth, he doesn’t seem to understand the difference between THD and THD+N where the “N” stands for noise. What he’s really measuring isn’t distortion, as he implies, but the noise floor. I encourage everyone to apply common sense to what they read and consider the source.

Sennheiser HD650 Impedance New Green=Open Gold=Simulated HeadOUTPUT IMPEDANCE: I’ve talked a lot about the importance of output impedance. Since I wrote my Output Impedance article in February 2011, Benchmark Media published a paper of their own that explores it from some other angles. Both are well worth a read. The graph to the right shows the impedance of the HD650 both in free air and clamped on head. there’s a big broad impedance peak centered at about 90 hz. If someone uses an amp with a sufficiently high output impedance the frequency response will also have a broad audible peak centered around 90 hz. The higher impedance can also reduce the damping and increase distortion. Combined, this could cause what many might might describe as more “boomy” or “bloated” bass. Someone else, however, might think it sounds more “warm” or “full”. Regardless, the change would generally be considered the opposite of “thin” as used by Lieven. It’s possible his typical amp for the HD650 has relatively high output impedance so that’s what he’s used to and considers “normal” for the HD650.

LEVELS MATTER: If the level were just a few dB lower when Lieven listened to the O2 that would, relatively, make it sound more “thin” because of the Fletcher-Munson curves and psychoacoustics. Studies show even a 1 dB difference creates a preference for the louder source. But most publishing gear reviewers don’t even have an accurate way to match levels. That’s a serious problem.

TESTING ONE TWO THREE: As mentioned in the previous section, Stuart Yaniger, an engineer who designs tube audio gear, recently wrote an excellent print article titled Testing One Two Three (it’s not on the web that I know of). The article has excellent advice for how to properly conduct blind testing and he outlines some of his blind listening experiments:

  • The Bastard Box – The author made an audiophile grade box comparing a $300 audiophile capacitor with a $1 capacitor from Radio Shack with the flip of a switch. He correctly identified the audiophile capacitor in blind listening only 7 of 12 times. Pure random guessing would be 6 of 12. By the statistical guidelines for blind testing, he needed to be correct 10 of 12 times for a meaningful (95% confidence) result. He failed so Radio Shack wins. 
  • Audibility Of Op Amp Buffers – He strung together 10 cheap op amp buffers in series to effectively multiply their flaws and did a blind ABX comparison of no buffers vs ten buffers. He found he could reliably hear the difference, so a buffer was removed (down to 9) and the test repeated. In the end he found with 6 or fewer buffers strung together he could not reliably detect any difference. This is an excellent example of blind testing both demonstrating a clear difference and then demonstrating the opposite-no audible difference. Blind testing let the author easily find the threshold of audible transparency (6 buffers). It’s also further proof that op amps are not the evil devices many audiophiles think they are when even six cheap ones strung in series can be transparent enough to sound just like a piece of wire. See: Op Amp Myths.
  • Listener Fatigue – The author devised a method to compare two pre amps for detectable “listening fatigue” by devising a means to blindly and randomly listen to one or the other with two hour meters recording the total listening time of each. If one was less fatiguing, in theory, it should accumulate more total listening hours. After two months of listening to both there wasn’t what he considered a significant difference in the total times implying both had similar levels of fatigue or no fatigue.

WHY WE HEAR WHAT WE HEAR: James “JJ” Johnston has lots of experience in audio ranging from Bell Labs to DTS. He’s made many presentations on audio and has spoken at Audio Engineering Society events. He’s recently re-formatted one of his presentations into an online article entitled Why We hear What We Hear. It’s an interested read with some valuable technical information on how our auditory system works. His article is a primary reference for my earlier comments about how we filter sound. His information correlates well with information from neurologists and brain experts as well as multiple AES papers on listening bias. Here are some highlights:

  • Expecting Differences - Perhaps the most important thing to take away from Johnston’s work is on the last (fourth) page.  He explains how when we listen for different things we will always hear different things depending on what we expect to hear. In other words I can play the same track for a person twice, with everything exactly the same, but if they’re expecting a difference they will in fact hear a difference. It’s not an illusion, or anything they can control, it’s just how our auditory system and brain work. This was also demonstrated in the AES paper Can You Trust Your Ears where the listeners consistently heard differences when presented with identical audio.
  • Auditory Filtering – Johnston talks about our hearing system involuntarily filtering what we hear. There are two layers of “reduction” resulting in somewhere around 1/100,000 of the data picked up by our ears ultimately reaching our conscious awareness. This was discussed in the first section with examples provided.
  • Auditory Memory – Johnston points out our ability to compare two sound sources starts to degrade after only 0.2 seconds. The longer we go between hearing Gear A and Gear B the less accurately we can draw conclusions about any differences. And the longer the gap the more expectation bias creeps into what we remember. Our memory, like our real-time hearing, is by necessity highly selective. Our brains are constantly discarding large amounts of “data”. Other studies back this up including the AES paper Ten Years of ABX Testing which found near immediate comparisons were far more likely to detect differences than extended listening with at least a few minutes between changes. This is the opposite of the common erroneous belief that long term listening is more sensitive.

AUDIO MYTHS WORKSHOP: The Audio Engineering Society (AES) is made up of mostly audio professionals especially those involved with production work including CDs, movie scores, etc. Their conventions are heavily attended by recording engineers, professionals who make their living hearing subtle audio details, and the people behind the gear they use. At the 2009 Audio Myths Workshop a panel of four experts presented a lot of good information on what we think we hear and why. If you don’t have an hour to watch the video, or just want to know what to keep an eye out for, here are some highlights I found especially worthwhile:

  • The Need For Blind Testing –  JJ points out it doesn’t matter how skilled of listener you are, if you know something about the two things you’re trying compare, your brain is unavoidably going to incorporate that information into what what you hear and what you remember later. 
  • A Dirty Trick – JJ recounts a story about a fake switch being set up to supposedly switch between an expensive tube McIntosh amplifier and, in his words, a “crappy little solid state amplifier”. In reality the switch did nothing and, in either position, everyone was listening to the solid state amp. But the audiophiles listening unanimously preferred the supposed “sound” of the tube amp. They heard what they were expecting to hear. His engineer friends tended to prefer the solid state amp with only one claiming they sounded the same. This is further confirmation of expectation bias.
  • Stairway To Heaven – Poppy Crum demonstrated how it’s easy to hear most anything when you’re told what you should be hearing. She played an excerpt from Stairway to Heaven backwards and there’s not much in the way of discernable lyrics. Then she put a slide up with suggested lyrics and most can easily hear exactly what’s on the slide with a fairly high degree of certainty. This is somewhat analogous to reading someone’s gear review, buying the same gear yourself, and hearing what you’ve been told you should hear.
  • State Legilature – No that’s not a typo. Poppy Crum plays a news announcer clip multiple times with everyone listening carefully. The “S” in the middle of the word legiSlature was edited out and it goes largely unnoticed by the entire audience. The brain helpfully fills in the missing letter for you automatically. Despite listening carefully, the audience of audio professionals mostly fails to hear the obvious problem because their brains automatically “invent” the missing audio information.
  • Crime Suspect – They play a TV clip of a theft occurring in a classroom. The students are asked to identify the thief. Many are so certain they can pick the right guy from a photo line up they would testify in court. But, in the end, they not only pick different photos, but they’re all wrong despite their certainty. It’s further proof of how the brain filters and how unreliable our senses can be even when we think we’re certain.
  • Argue To The Death – They talk about experiences on various audio forums where people argue subjective claims to the extreme using the argument “you can’t prove it doesn’t sound better”. No, but we can prove if typical people under realistic conditions can detect any difference with blind listening. Here’s a humorous parody of a typically “passionate” forum discussion (about hammers) that’s, sadly, analogous to some audio threads I’ve seen.
  • Identical Results – Measurements can produce consistent, reliable, and repeatable results. I can test Gear A on Monday and re-test it a month later and get very close to the same result. That’s far from true with sighted listening tests where you can get very different results 10 minutes later if someone is expecting a difference. Further John Atkinson and I can measure a DAC and we should both get very similar measurements as long as we use accepted techniques and publish enough details about the test conditions. He can verify my measurements, and I can verify his, with a high degree of certainty. Without carefully controlled blind testing, that’s impossible to do by ear.
  • ethan winer audio myths audibilityTransparency – Ethan Winer, an acoustics expert, discusses how audio electronics can be defined as audibly transparent by four broad categories of measurements and he provides   criteria for complete transparency. He states that gear passing all these criteria will not contribute any audible sound of its own and in fact sound the same as any other gear passing the same criteria:
    • Frequency Response: 20 hz to 20 Khz +/- 0.1 dB
    • Distortion: At least 100 dB (0.001%) below the music while others consider 80 dB (0.01%) to be sufficient and Ethan’s own tests confirm that (see below).
    • Noise: At least 100 dB below the music
    • Time Based Errors – In the digital world this is jitter and the 100 dB rule applies for jitter components.  (photo: Ethan Winer)
  • Comb Filtering – This is a problem with speakers and it’s likely the source of some audiophile myths. Ethan demonstrates how even changing your listening position (where your ears are in the 3D space of the room) by a few inches can create very audible changes in the high frequency response. Let’s say you sit down to listen, then get up and swap out a cable and sit down again, but this time you’re leaning further back. You may well hear differences but it’s not the cable. It’s the nature of the room acoustics and comb filtering. This is in addition to all the other reasons you might hear differences unrelated to the cable.
  • Noise & Distortion Thresholds – Ethan sets up a noise track and mixes it with music at various levels to simulate similar levels of noise and distortion. I can still plainly hear the noise (on the WAV files) at –60 dB (0.1%) and barely hear it at –70 dB (0.03%) on the cello excerpt. It’s gone, to my ears, at –75 dB (0.018%). So my usual rule of thumb of –80 dB (0.01%) is further validated with some conservative margin for error.  When, for example, I point out dissonant inter-channel IMD in the AMB Mini3 amplifier reached –55 dB, DAC alias components in the range of –50 dB to –25 dB, or 1% THD (-40 dB) in a singled-ended or tube amp, Ethan has helped confirm it’s entirely plausible such distortion could be audible. Here’s a link to the audio files Ethan uses in the video. The cello example, (Example2 in the WAV files), starts at about 33:30 into the video. The noise starts out for two cycles at –30 dBFS, then two cycles at –40 dBFS, and so on. Listen for yourself! When you see a spec like 1% THD you can think about the plainly audible noise at –40 dB in Ethan’s examples. This is somewhat oversimplifying the entire audibility threshold issue but it’s a reasonable worst case approximation. In this case it’s best to error on the side of being too conservative (likely the reason for Ethan’s own –100 dB transparency criteria).
  • Phase Shift – Ethan demonstrates how even large amounts of phase shift applied at various frequencies is inaudible as long as it’s applied equally to both channels. What can matter is interchannel phase shift—any significant shift between the right and left channels. Relative phase is also critical in multi-driver or multi-speaker set ups (i.e. between a woofer and tweeter or front and rear speakers in a surround set up).
  • Null (Differencing) Test – Towards the end Ethan talks about how null testing (which includes audio differencing) not only reveals every kind of known distortion (THD, IMD, phase, noise, frequency response errors, etc.) but also even unknown kinds of distortion. You’re literally subtracting two signals from each other and analyzing/listening to whatever is left over. With test signals this is something the dScope audio analyzer can do in real time with analog, digital, or mixed domains—analog and digital. The dScope, in effect, subtracts the test signal from the output of the device and adds the required amount of gain to boost what’s left over up to a usable level. It’s a test that can bust a lot of myths about various things that are claimed to degrade an audio signal. It can be done, using music, driving real headphones or speakers, in the analog domain (i.e. Hafler and Baxandall method scroll down) or in the digital domain (i.e. Bill Waslo’s software from Liberty Instruments). Ethan presents a few examples including the next item.
  • Reverse EQ – Many people seem to think even high quality digital (DSP) EQ does irreparable harm to music. Ethan has an example where he applies EQ, then applies the exact opposite of the EQ, and then nulls (see above) the result with the original version (no EQ). He gets a near perfect null indicating even with two passes through the EQ DSP no audible damage was done.
  • Other Examples – Ethan has further demonstrations aimed more at recording work rather than playback. He does an interesting “bit truncation” demo but it’s done in such a way as to make comparing the bit depths difficult.

BLIND LISTENING GUIDELINES: There’s a lot of info on how to conduct reasonably valid blind testing and some of it gets very elaborate. But don’t let that scare you from trying it. If you’ve never tried blind listening before you might be surprised how quickly it opens your eyes—even a very simple test. As many wine critics have found out comparisons immediately become much more difficult when you don’t know which is which. You can get more meaningful results from just following some simple guidelines:

  • Compare only two things at a time until you reach a conclusion 
  • Always use identical source material with no detectable time shift between A and B 
  • Match levels to ideally 0.1 dB using a suitable meter and test signal
  • Watch out for unintentional clues that might hint at which source is which
  • Ideally perform enough trials to get a statistically valid result (typically 10 or more but even 5 can be informative)

TRYING ABX YOURSELF: Foobar 2000 has an ABX component and there are other free ABX options.  This is an excellent way to say compare 16/44 FLAC files with a high resolution FLAC file. There are files available at Hydrogen audio or you can make your own if you have any good 24/96 or 24/88 demo tracks. But always do a high quality sample conversion of the high resolution track to produce the 16/44 track. Or compare a 256K AAC file with a 256K MP3 file ripped from the same CD track. Etc. What’s important is you eliminate all other differences. The levels have to be the same, etc. Audacity is free and can be used for editing the files. If the tracks are not in time sync you can configure the ABX options to restart the track each time you switch.

FOOBAR ABX: There’s a YouTube Video on ABX testing with Foobar. The software and ABX component can be downloaded here. The listener can, at any time, choose to listen to amp A, amp B, or “amp X” which will randomly be either A or B and they won’t know which. The software keeps track of “X” and lets the listener vote when they feel they know if “X” is really A or B. A normally hidden running score is kept of the votes. After ten or more votes, the final results can be revealed and checked for a statistically valid result.  If a difference is detected the system can be set up by someone else so the listener doesn’t know which is A and which is B. They can then pick a favorite, again taking as much time as they like, without the unavoidable expectation bias that occurs when they know which is which.

A BLIND ANECDOTE: I received some cranky emails from someone who had read my Op Amp Myths article and basically said I must be crazy or deaf for not hearing the obvious differences between op amps. He was a serious DIYer who claimed to have designed and built a lot of his own gear. I suggested he rig up two identical pieces of gear so he could level match them and switch between them with both having socketed op amps allowing any two op amps to be directly compared. I figured he’d never bother. Much to my surprise the next email a few days later was an apology and he explained it was really eye opening and thanked me. I’m fairly sure he’s going to be approaching his future designs somewhat differently.

REAL ENGINEERING OR SNAKE OIL? I’m curious what others think of the $3995.00 BSG Technologies qøl™ Signal Completion Stage? It’s an all analog device (line-in/line-out) that, according to BSGT, “Reveals information that, until now, has remained hidden and buried in all audio signals”. How, I wonder, does it do that? The Nyquist Sampling Theorem dictates how much information can be reconstructed from a 16 bit CD. And a decent DAC, as far as I know, reconstructs all of it. In fact, a good 24 bit DAC can reconstruct data well below the 16 bit noise floor of a CD—i.e. the DAC is better than the CD itself. So, from what I know, every sample on the CD can be reliably and completely converted into an analog signal. Hence there should not be any “hidden information” on the CD. One can argue information got lost further upstream in the recording chain—i.e. at the microphones, due to the 22 Khz Nyquist bandwidth limit, etc. But that information isn’t even on the CD hence it would be quite a miracle if it could be reconstructed out of nothing. Anyone else have any theories on this nearly $4K device and their main marketing premise?

HAVE FUN! The above is just one example. I know several audio fans who have had fun discovering just what they can really hear and what they can’t using blind testing. It opens up an entirely new perspective and provides a new way to experiment with audio. It’s especially useful, and accessible, for DIYers.

227 comments:

  1. Awesome article!

    So many people completely discount the psychological and neurological influences on perception. They either don't know about them, don't understand them, or don't believe they apply to them. I think that's the real issue.

    Some people will take any suggestion that their senses are not infallible as a grave insult. Other people will argue every last bit of electrical detail or measurement as if any tiny difference should be audible if you've got good ears and know what to listen for. They get hung up on tiny things and don't see the big picture, arguing over words and not the concepts that the words represent.

    To understand this sort of thing I think it really helps to have broad knowledge base to draw on. I'm not an expert in electrical engineering of analog circuits, psychology, or neurology. I'm not an expert at anything. I'm a jack of all trades but master of none and I think that equips me to understand some of these things better than people you only focus on a single discipline.

    You're taking an interdisciplinary approach as well. Some people accuse you of just listening with your 'scope but they don't realize that none of those numbers mean anything without reference to other fields. If lower distortion, noise, etc was all that mattered how would you know when to stop. Would you have designed the O2 differently? Would you have recommended or reviewed different products? What tradeoffs would be made differently? Where do we set a threshold and why?

    None of those questions can be answered in the isolation of a single field. Pure engineering without reference to psychology or neurology won't tell you how to properly gauge subjective perceptions. Pure psychology or neurology won't establish thresholds or let you design to them. We have to do listening tests. We have know how to do them. We have to know why and how they should be blinded. Then they have to be correlated to the measurements.

    You need to know EE to know how the gear works, you need to neurology to know how the brain works, you need to know psychology to know how they interact. You even need to know some sociology and economics as well if you want to understand why many people discount everything you've just outlined.

    That's a lot more than just looking at your dScope and saying that lower is always better.

    ReplyDelete
  2. Another great article as always. This blog is the site I point the misinformed to.

    Helping people understand on head-fi is a lost cause. I don't know whether to be angry or sad when I read people suggesting recabling for better sound. Some of the things I read there is on the border of insanity. I guess that's where obsession on a hobby can take someone to the point they don't ever second guess the things they read or use common sense; they don't step back and do a reality check. There is too much money to made out of fools when it comes to audio. I have never been more content with listening to music since I left head-fi instead of wondering what gear I need next.

    ReplyDelete
  3. Great article as always (and coming from an engineer with some research experience, this feels like reading a scientific article!)

    I just have a general question: You've sufficiently proved that subjective reviews of audio gears are highly unreliable. However, it is also true that objective reviews of most gears are also not available and even if they are, the majority of us (who are not sound/electrical engineers) will not understand the graphs and charts enough to determine if a particular gear is accurate or not. For instance, I've in the market to buy a portable amp that can fit in my jeans' pocket (sorry O2 amp), and there are alot of options from the cheap, like the FiiO E3, to the expensive, like the Pico Slim and RSA Shadow. As i'm not able to listen to most of these amps for myself, nor are measurements readily available, what will you recommend to be the best course of action for me to take to buy the right amp?

    A side question: The review at Headfonia mentioned that there are discernible floor noise that can be heard at high gain. Any possible explanation what could have caused that?

    Lastly, an observation: There is a local audio store where you are allowed to listen to any of their gears that they have before you made your decision. One habit that the salespeople all have is that they insist on you trying out the different gears before telling you the respective prices. (For example, when a new gear arrives, I always impatiently ask for the price before trying it out but my request will not be granted until I've taken a listen.) After reading your article, it's clear to me that they are trying to remove any bias that is present when you know about the prices beforehand, even though it is potentially more profitable for them to reveal the prices first.

    ReplyDelete
  4. Really great article, you are very good at explaining these concepts in a very simple and easy to understand way. It still confuses me how people can really think they are exempt from unconscious biases. Hopefully one day this whole audio industry comes back to reality, I've noticed how detached it is just from reading Head-Fi and getting recommended a 500 dollar Amp/DAC combo for 200 dollar headphones.

    ReplyDelete
  5. Thanks to everyone for all the encouraging feedback so far.

    @Kite, I'm going to try at least somewhat address your question about how to determine what to buy in my next article: Timex vs Rolex There are no simple answers but that could change if enough people become unhappy with the current status quo.

    There's a book on wine that talks about how Trader Joes almost single-handedly set the wine industry on its ear with their $2 wine ("Two Buck Chuck"). One of their $2 wines won a high profile blind tasting. The mainstream media picked up the story and the wine industry was forever changed.

    People started conducting their own blind wine tastings, etc. It put a lot of pressure on the "high priced is better" aspect of the rest of the wine industry. Two Buck Chuck was a big thorn in their side.

    So, in response, the big buck wineries have finally been forced, especially post 2008, to come up with lower priced offerings that actually taste good. Consumers are at least somewhat less likely to buy the usual marketing pitch for high priced wines. And Trader Joes continues to sell Two Buck Chuck in massive quantities.

    That's a long way of saying its really the consumers who need to put pressure on the audio industry to give them more of what's needed. That includes more complete specs, more and better independent measurements, more accountability, more blind testing, etc.

    The audio industry isn't going to give in without a fight, but just like Two Buck Chuck changed the playing field, so can the above and solid reasonably priced products like those from Emotiva.

    In the meantime, I would suggest trying to listen for yourself whenever possible even if that means buying mail order from a vendor with a generous return policy. And ideally apply some of the blind listening techniques in this article--even if it's just a friend swapping things around without your knowledge.

    ReplyDelete
  6. incredible as always

    one of the best statistics you published (that i have always believed to be true) is right here

    -Auditory Memory – Johnston points out our ability to compare two sound sources starts to degrade after only 0.2 seconds

    remember that next time you see someone comparing gear that clearly took them more than .2 seconds of switching to compare...

    ReplyDelete
  7. Hello NwAvGuy,

    I haven't read this entire article, I got about halfway through. I am planning on building the O2 regardless of headfonia's article. I had a thought regarding that, and regarding the general tone of your article.

    1) Isnt everyone's ear itself different? Wouldnt that change the minute particle physics of air inside the "chamber" and produce different results for everyone anyways? I felt that headfonia guys had been listening to their "ideal" setups for long enough that when faced with a setup that is MEASURED as "subjectively ideal", they simply werent used to it and thus were unimpressed? I tried the HD650 vs the HD600 out of a Fiio E11. hardly a good matchup but I noticed a significant amount of "lifelessness" in the HD600 vs the 650. Headfonia's article seemed to make it sound like the O2 made such a difference (wtf its transparent, so how can it make a difference) that the HD650 sounded closer to the HD600. I'd love some comments on that if you would be so kind.

    2) At my current position in the article, i feel as though you are spending a LOT of time finger pointing and appointing blame. Why not simply accept that although objective measurements is something the O2 accels at, some people simply WANT a colored sound? I get that headfonia was wrongly appointing the O2 as the source of blame for supposed lackluster HD650 performance, but everyone knows (and has measured) that the O2 is pretty much transparent. Thus headfonia's findings were flawed or caused by their own source equipment. In any case, though, isnt the entire reason for multiple headphones, dacs, and amps to exist entirely because different people prefer different types of sound?

    The other day a friend of mine complimented me after listening to a demo of my portable rig. When i got it back from him, I found that he was listening to stairway to heaven on my ATH-m50 out of ipod classic LOD with Fiio E11 set to BASS BOOST 2. I was horrified, but I realized that some people simply PREFER their music to sound like Beats by Dre.

    I just find this finger pointing kind of tedious. You are an absolutely brilliant engineer and the O2 has consistently proved it, as does the fact that not a single to my knowledge has offered to publish measurements as detailed as yours in response to your challenge. Why not leave things at that smug fact and choose instead to stoop to fingerpointing levels?

    ReplyDelete
  8. An absolutely fascinating article - thank you again, NwAvGuy.

    As somebody with an obsessive love of music, but with budget limitations (and a 3 month old son as my main priority), the information you provide is fantastically useful.

    I would absolutely love if you tackled the world of full hifi amplifiers someday, since the same principles absolutely apply. Is that something which has crossed your mind?

    One last question (sorry) - do you have any experience with any of the cheap DACs available from China (primarily sold via ebay)? If so, how do these stack up against recognised brands like FIIO?

    ReplyDelete
  9. HOW DARE YOU dis mythbusters! :P

    "If you’re openly pro-Obama you don’t get to work on Mitt Romney’s political campaign."

    But you can endorce him while also supporting obama if you are John McCain.

    Stairway To Heaven - it's so much fun to do this on youtube with all sorts of different lyrics for the same songs played backwards - very good/common example to test yourself.

    Sorry for my mostly not serious comments. . . a great article as always and I enjoyed reading it.

    I'll only post once this time! I think I've figured out this weird posting system.

    ReplyDelete
  10. As usual, great read.
    As a matter of fact, reads even better since I started using my Quantum Effect Cryogenically treated monitor cable.
    Really.

    ReplyDelete
  11. Ok, more questions:

    @Kite, I forget to answer your question about hiss. The O2 has no audible hiss with even the most sensitive headphones I know of. If someone hears hiss from the O2 it's either from the source feeding it, or the source is powered off or disconnected which creates some noise due to the floating input. That obviously can't happen under operating conditions.

    @Will, I do someday hope to move onto stuff that drives speakers. Like headphone amps, there are real audible differences in power amps, receivers, etc.

    When I publish the ODAC measurements I'll be, among other things, comparing it to a cheap eBay headphone DAC from China. Every such product I've tested has had one or more serious flaws. I don't recommend them. FiiO is a better option.

    @Shrimants, I'm really not trying to point fingers. I tried to be reasonably clear that most of the problems are nearly industry wide. It's just good to use a real world example, and a timely one is even better. So I chose the Headfonia review. And I chose Head-Fi and Wine Spectator because they're arguably the biggest and most well known of their respective genres.

    Theory is hard for a lot of people to grasp, or seems too abstract, without real world examples. So I need to use real products, websites, people, magazines, etc. And it's hard to write about this stuff and use real examples without someone being cast in a less than positive light. I don't think I've said anything that's not factual. And if I have I'll gladly correct it.

    Also, perhaps I need to make it more clear, but I respect many don't like "flat". But I still maintain buying a colored amp is a poor way to get the sound they want. EQ is a far better option as it's far more flexible, you can turn it on and off, you don't have to worry about future headphone/speaker purchases, etc.

    And "colored amps" often have more problems than just odd frequency response. Tube and single-ended amps, for example, often have plainly audible distortion. Others try to destroy expensive headphones. Etc.

    ReplyDelete
  12. Another excellent read. Well done NwAvGuy! I totally agree with your sentiments regarding electrical EQ. If used as a "luxury tone control" you can always defeat it.

    I am not a fan of EQ for correcting the response of a (loudspeaker) for anything other than electro-mechanical resonances. Acoustic problems have to be solved acoustically. But, since we are generally discussing headphones, 'tone shaping' is fine by me :)

    I know EQ is used in recording studios, such as to accentuate the 'pluck' of a string or reduce a tooth sibilance. Most good engineers do try to use as little EQ as possible, though, unless using EQ as an 'effect' like some use compressor pumping. If a lot of EQ is required to fix something, a different mic technique is in order.

    In multi-track recording, most instruments are mic'd at least once (say a body mic on a guitar plus a neck mic) and the channel strips are blended. Each channel might like differing EQ and once combined you get a very rich sound. Likewise on drums; you may mic the bottom and top of a drum and combine them to create a sound. Once all the channels are mixed down, the combined phase shifts and amplitude gain/cuts (EQ effects the frequency, gain, Q and phase) are combined into left/right stereo signal.

    All of that EQ is tone shaping, not headphone or speaker correction, and all of it goes through many op-amps and/or DSP.

    Very nice include on the Ethan Winer stuff.

    Cheers and best regards. Very fine job you do here.

    ReplyDelete
  13. I remember once I was mixing a song, and I opened up an EQ on a guitar track. I sat there for 15-20 minutes messing with different frequencies, adjusting gain and bandwidth, until I finally got a sound I was more or less pleased with. Then I looked over and realized that the EQ had been bypassed the entire time! The changes that I had absolutely no doubts I was hearing were nonexistent. It's too bad more people haven't had experiences like this. I'd like to see if they "trusted their ears" then.

    ReplyDelete
  14. I only got to the EQ part, but just that compelled me enough to comment. In short: yes, yes, yes, yes!

    I loved my HD650 immediately upon first listen. Then I read a well-written, well-reasoned little ditty on EQ to correct for basic differences in human physiology. I followed the instructions and made a relatively subtle parametric EQ adjustment with a free DSP called EasyQ, and now I love my HD650 even more. In fact, I can't imagine how I listened to music for so long without EQ.

    It turns out that my ears are just very, very sensitive to highs, and even the HD650, notoriously "dark" headphones, caused undue fatigue that I didn't even recognize until it was gone.

    I wholeheartedly recommend parametric EQ to anyone, regardless of the quality of the gear you own. Done right, it does not necessarily "color" what you hear; like wearing properly-sized shoes is good for your feet, EQ makes sound right for your ears.

    Just give it a shot and fight through those first few days when the music does and will sound "off." That's just you getting used to it. I'd be willing to bet that once you do, you'll never go back.

    ReplyDelete
  15. Great article.

    Just wanted to point out that color isn't necessarily a function of frequency response alone, and many audio engineers still use tube gear/tube modeling plugins and record music into a tape machine before converting to digital. These are people who are very skilled with EQ and conduct blind tests regularly.

    Personally Ive never even heard tubes directly compared to solid state gear and don't care for them, but still think that its reasonable that some people prefer them and use them regularly.

    ReplyDelete
  16. Yet another great article.

    In my opinion, if someone choose to buy "flawed" equipments simply because they like the sound signature it's fine by me. Because it's their preference and I definitely won't force them to change it. As I also don't like to be forced to change my preference either.

    That said, most subjective reviews did are somewhat "forcing" their preference to the reader. And this happens quite a lot of in the comments section of Headfonia. They always suggested gears that match their musical preference.

    I like your review very much, and I hope that you can focus on doing that more. Debunking myths are good and all, but really, I'd like to hear more of your reviews of audio gear.

    Being a computer hardware geek, I'd always compare something by the numbers. In computer hardware, it's easy to say what product are better than others, just put them on a benchmark suite and compare the resulting numbers. Wouldn't it'd be very nice if we can do the same for audio gears? And your blog is the forefront of that venture.

    For example, I'd like to read a review on the Asus Xonar Essense STX with a proper equipment rather than just plugging it on whatever DAC/ADC and measures via RMAA. The Xonar Essence STX in particular is pretty interesting, because they include a unique individual AP test result for each of their products. And indeed, the results are very interesting (ruler flat FR, ultra low THD. etc). Too bad they did not says what's their output impedance are, and that's one things that puts me in doubt.

    All and all, I do believe that Asus products will be well engineered, since they do have a lot of engineering and manufacturing resources at their fingertips compared to most (if not all) esoteric audio equipment manufacturers. They make great motherboards and other "ultra high frequency (MHz & GHz) stuffs" after all. Doing something that only process 20Hz-20KHz should be easy for them.

    But really, it would be reassuring if a third party can properly reviews and measures such products.

    I'd also be glad if someone can point out some "proper" audio gear reviewers that doesn't reviews "$1000 budget component" (ala Stereophile (whelp, <$1000 is low end, oh god)) to me.

    tl;dr Keep up the good work!

    ReplyDelete
  17. I think that's rounded up the majority of the things to point at when someone insists they have magic ears!

    Was interested to read about the noise track test: I suppose this is a worst-case test as the noise would not be related to the musical content in any way.

    Nice to see you also covered phase, as I remember Nelson Pass talking about the inferiority of opamps on Head-Fi versus discrete designs with reference to group delay, which I thought seemed rather odd.

    Whilst looking for it I stumbled across a very depressing Head-Fi thread about "opamp sound". Apparently AudioGd engineers are of the habit of telling people they tune sound signatures by mini differences in crosstalk...It also contains a wonderful example of admins being constructive, in that dear kwarth descends into the thread every so often to spout sophistries about how this isn't REAL science, joined occasionally by Currawong, who threw a paddy when I suggested that -70db vs -80db of crosstalk does not create a tubelike signature. I'm so closed-minded...

    ReplyDelete
  18. Great article as usual. It's funny I was reading that article on Headphonia just yesterday and was wondering how long it'd take you to follow-up here. Evidently Lieven started a war he shouldn't have...but hopefully your article gets more views than his does...

    Just curious what are your personal favorite headphone brands?

    ReplyDelete
  19. Hi NwAvGuy,

    Firstly may I thank you for what is an amazing blog on the realities of the hifi scene over the past forty years or so. Your insight into audio design and the subjectiveness of audio perception is amazing and worthy of a much wider audience. I'm doing my best to make as many people as possible aware of your site.

    I've recently purchased an O2 and it is all you said it would be. It just works and sounds pure with my DT770Pros and my HD650s. Looking forward to the ODAC and the desktop combination of it with the O2.

    In your most recent article you mention the music sounding better in the next room. Your suggestion of 'That's nice dear' from the other half is quite possible, but when I was into sound mixing and stuff in the 90s one of the tests of whether a mix was good or not was to play it and go into the next room and see what it sounded like. I don't know where I picked this up but it does seem to be true - if a mix sounds a bit off (and after several hours of mixing your ears can get a bit fatigued) then listening from the next room will tell you whether it is a good mix or not. Perhaps our brains have a perception of what sounds good from an adjacent room, but I can't think of a good evolutionary reason for that at the moment!

    Like others, I tested the various improvements to my hifi in the 70s and 80s, and I can summarise the valid (ie audible) improvements on one hand: thick speaker cables (QED 79 strand) make the bass sound tighter and deeper when compared to bell wire. Big low ESR capacitors in the power amplifier give tighter and deeper bass. A blob of adhesive (Blutac) between the cartridge and headshell on your record player makes the sound better. Gold-plated switches and relays give a more reliable signal path over tin-plated crackly switches. That's about it!

    The rest (apart from good engineering design) is snake oil, and like you I despair at friends who buy gold-plated IEC leads, wire their hifi directly into the consumer unit (yes really!) and pay fortunes for specialist leads that offer little apart from better construction and perceived enhancements.

    Knowing as I do how many brass and copper clamps (some probably not particularly tight either) the mains comes through before it hits the hifi, having gold-plated pins on an IEC lead isn't going to have any effect (especially as it's probably going into a phosphor-bronze socket in the wall and tin-plated pins on the equipment). Then there's the gazillions of op amps that the music has been through on its way to the CD...

    Keep up your excellent work and long may you continue to bring clarity and objectivity to all of us who read your words of wisdom.

    ReplyDelete
  20. NwAvGuy, as usual your article is very well written and a gorgeous read. And it´s interesting that headfonia.com did a review of your amplifier - the outcome is exactly what I would have expected it to be. But you forgot one thing (if I didn´t overread it): the reviewer 1. was biased (which you mentionend) 2. was disappointed because your amplifier doesn´t fulfill the expected sound.

    An example: when I got my ASUS Xonar Essence ST I was appalled at first because my Sennheiser HD-600 was so unexpectedly thin sounding (not biased here, it really was). In reality however it doesn´t sound thin, because this "thin" sound is just how it sounds in reality. The amp I´ve used before wasn´t as neutral as the ASUS. But a few months passed and I´ve found time and time again that the ASUS is an extremely transparent and possibly most objective listening device, it´s as neutral as it can be for its price. Of course, I got used to this "new" sound - but I came to admire its reliability too. As you probably know, the amp used for the Xonar is indeed capable for such a high-ohm headphone like the HD-600 and this combination doesn´t have the usual throwbacks others might have. But I had to learn what the "reality" was.

    To me it always seems that high-end stuff is engineered so that it appears to "interpret" music; it cannot of course, it´s just a machine and you can see the obvious flaw those things represent: they might sound well with one genre and bad with another.



    Anyway, I have to remind you that you should be careful to take hydrogenaudio as a prime example of objectivity - it is far away from being objective. There are numerous articles online there that easily demonstrate that many people over there are equally biased as subjectivists (the recent article about mastering engineers is just one example). Furthermore, your insistence on DBT is good but sometimes you seem to forget that a DBT indeed is just ONE statistical method of comparing one thing to something else: the outcome is strongly influenced by the statistical model used and of course also by our (psychoacoustic) perception because in case of audio we still use our ears. Which means that our brain still interprets what our ear hears. DBTs are good but they too tend to be unreliable and we haven´t found a truly objective method using our ears yet. Which brings me to the next: you seem to be creating fact out of a DBT result: you cannot ever do so. A DBT doesn´t create fact, it just produces a deviation from the assumed null hypothesis. If this deviation (the outcome) is marginal is another question.

    The McGurk effect is not really fit for exemplifying bias because it is a combination of sight AND sound that´s built into our brain (integration abilities for audio and visual information), it doesn´t have anything to do with our expectations. It´s more interesting for linguists than for engineers. So it appears that you (consciously or uncounsciously) misuse this effect for your own agenda when it´s not really fit for that. I did too a few weeks ago - but I erased that article.

    And then you constantly quote the Meyer/Moran article: don´t do it. There are more problems with this article than people admit. The test setup for comparison was well done but their conclusions are fishy, they didn´t seem to understand the null hypothesis. Even worse: 80% of the audio material they used wasn´t high resolution at all (no frequencies beyond 22.5 kHz) - so there wasn´t anything that could be removed by the loop they used. They must have peer-reviewed it before publishing so I often wonder if they went for the route "Publish it for the sake of discussion of an important thing".

    As you might have already guessed I´m a mixture of objective/subjective listening, I want to combine both as good as possible (on my own blog for example). It´s difficult and I don´t know if I´ll succeed - but I´ll try anyway.

    ReplyDelete
  21. Another very informative well written article.
    I enjoyed the read, hopefully many
    more will too.

    Just on the notice of filtering, it is really interesting how sometimes on some tracks I hear a wide soundstage, then it is gone, then I hear it again - on the same MP3 player (Sony X1060) with the same headphones (Sennheiser IE8). And the actual track hasn't changed either...

    ReplyDelete
  22. "This guy claims to have done a blind test with 5 people and 10 trials each using two DACs."

    I believe it was amplifiers:
    "The two amplifiers compared were a Wavac SH833 and Halcro DM58." - http://www.high-endaudio.com/RECENT.html#Mar

    He's a design engineer with Luminance Audio, so bias is inherent, however looking past that I hope we can gain something beneficial from improving ways of measuring audio data.

    I'm wondering if the term 'unlistenable' is accurate or if it's simply a hyperbole based on the comparison (similar to 'night and day'). However, there's much more research and tests that need to be done before he can claim that low THD at low levels point to better sound quality.

    Anyway, I'm glad you added it in NwAvGuy. And thank you for the stimulating article! All that info was in desperate need of sharing.

    ReplyDelete
  23. itional positive comments. This is a controversial topic and an uncomfortable one for many into high-end audio. To address some comments...

    @Marlene D, Certainly there's plenty of bias at Hydrogenaudio. But it's still, by far, the most objective popular audio forum I know of. If you know of other less biased objective similar forums, please share them?

    I'm not promoting any one method of blind testing. I like ABX because it have many advantages. How many trials you run, and what score you want to consider a valid result, is up to whoever is doing the testing. In the objections to DBT I tried to make it clear it's not "fact" just like it's possible to go to Las Vegas with $100 and come home a millionaire. But I think it's important to be realistic about how fine you want to split the hairs when challenging DBT. When "obviously audible differences" disappear by throwing a bed sheet over the equipment, that says a lot without using any statistics.

    The McGurk Effect IS an example of expectation bias and it does fit. What you see happening with the speakers mouth and face creates the expectation. Expectation can be created many ways.

    Have you read Meyer & Moran's AES rebuttal paper to all the alleged "problems" with their paper? It's funny how it rarely gets mentioned in all the M&M bashing. You're certainly entitled to your opinion, but if you haven't read that paper you should. From a scientific perspective I think they ran a relatively solid study. It's just their conclusion is hard for many to stomach so many try to discredit them--often falsely.

    As for objective/subjective I tried to explain if drinking your wine from Waterford crystal improves the subjective experience, that's great. I have no objections. But please don't try to tell the rest of the world the crystal itself changes the taste of the actual wine. That's what audiophiles, and the makers of audio gear, are doing everyday and it causes lots of people to part with more money under a false pretense.

    @Yang Zhou, it's hard to pick a favorite headphone brand. Someone asked in the Early April article what my favorite headphones are and I tried to answer there in more detail. If I have to pick just one headphone I own, it would be the HD650.

    @Anon, Stereophile did a full set of measurements on the Xonar STX. Indeed they found the output impedance to be uncomfortably high which seems to be a problem with many Xonar products (see my U3 review). I'm not sure a computer company, despite all their resources, will always do well as a high-end audio company. The engineering skills and criteria are very different. For example, with digital signals on a motherboard, a signal-to-noise ratio of only about 10 dB is required. They're also either unaware, or don't care about, all the Sonic Advantages of Low Impedance Headphone Amps.

    ReplyDelete
  24. @Doug, there are several iOS apps and I haven't tried some of the newer ones. The one that impressed was EQu but there may better options now. There's a lot of competition among apps.

    @Anon, With anything but speaker cables you would really need an awful (or intentionally distorting) cable to hear any difference regardless of the materials used. In fact, it's usually not even possible to measure any difference in audio performance between interconnects even with the finest equipment available like the $50,000 Audio Precision SYS-2722. Such measurements extend way past the thresholds of what's audible.

    To create a 0.1 dB change (well under the threshold of audibility) with a line level level interconnect cable into a worst case 10K input impedance, the difference would have to 116 ohms. That's pretty much impossible.

    With a 32 ohm headphone the cable resistance would have to differ by more than 0.37 ohms between two cables. That's unlikely even using different materials and gauges for the conductors.

    With speaker cables and 8 ohm speakers, however, a 0.09 ohm difference would create a 0.1 dB change. Considering speaker cables are often relatively long that's entirely possible.

    20 feet of 12 gauge wire is 0.064 ohms while 16 gauge wire is 0.160 ohms--a difference of 0.096 ohms.

    But that doesn't mean you should run out and buy $100+ speaker cables. Plain old 12 gauge copper zip cord from Home Depot will sound every bit as good. See the Wired Wisdom link in this article for a blind test reaching the same conclusion.

    ReplyDelete
  25. Silver does have better conductivity than copper, but it is a relatively minor difference (less than 10 percents). A slightly higher gauge copper wire can have as low or lower resistance than silver, while being cheaper and even lighter. And if the cable impedance is already very low compared to the load impedance (like it normally should be), reducing it by a few percents makes a very small difference. The most likely reason silver cables are assumed to sound brighter is simply that silver looks brighter than copper.

    ReplyDelete
  26. have you heard any tube amps that would get even a partial seal of approval or are they all either a gimmick or add too much coloration?

    I have a Beyer T1 but always find myself wanting to have the hd650 too. if i didnt find the beyer much more comfortable, i might have already switched back. any thoughts on the T1?

    ReplyDelete
  27. @David:

    It is entirely possible for a tube amp to manage something approaching transparency. In that case, you've now got what you could have accomplished with a cheaper SS amp, but with delicate hot glass things that have to be replaced every so often.

    ReplyDelete
  28. @David. I can offer slightly bending the metal sides (don't crack the plastic portion) of the headband made my HD650s vastly more comfortable and they still stay on my head. I do, however, agree Beyer makes even more comfortable headphones.

    The tube amp question is hard to answer. I've heard, with nothing handy to compare them to, a few tube headphone amps. It's hard to judge them without a reference. With the best experience I didn't hear anything obviously wrong besides some hum and noise when I cranked the volume up between tracks.

    But the above was also using high impedance relatively low sensitivity headphones which are generally better suited to tube amps (partly depending on if, and what type of, output transformers are used). A headphone like the AKG K/Q 701 is a much more challenging load for many headphone amps. And highly sensitive headphones, like the Denon D2000 or Shure IEMs, are far more likely to reveal noise problems.

    One thing a lot of people don't know is most tube amp components these days are cheap copies made in China--this is true even of amps made in the USA and Europe. And, Chinese tubes, output transformers, high voltage capacitors, etc. often don't perform anything like the "good stuff" from the 1950's. It's especially challenging to make output transformers that are highly linear and don't add a lot of crud and/or frequency response problems.

    I think it's amusing how so many people are obsessing over the transient response of DAC filters--i.e. pre and post ringing as well as PCM NOS non-oversampling DACs. Many of these same people claim to love the sound of tube amps that use output transformers. I wonder what they would say if they saw the horrible transient response of their tube amp with their actual headphones as the load? The transformers in tube amps creates far bigger problems than any DAC. And the OTL (transformerless) tube amps have horribly high output impedance.

    As for the Beyer T1 I've never heard them. The early ones supposedly are way too bright for many tastes and the latest ones are allegedly somewhat better. I'd like to have a listen someday.

    ReplyDelete
  29. Hail to thee, NwAvGuy!

    I sit here in enormous frustration after reading this article.

    I am frustrated at my own inadequacies as regards the English language.

    Whereas Bill Shakepeare would have at his disposal the skill to properly describe the value to humanity this article represents as well as his acknowledgment and gratitude for your eloquence and courage, all I can muster is a weak "thank you."

    I love my O2 and regard it as a piece of reference equipment.

    I believe it was a writer by the name of Theodore Sturgeon who said in response to a question about the quality of modern science fiction writing, "95% of it is absolute crap. Then again, 95% of EVERYTHING is absolute crap!"

    Your work is clearly part of the remaining 5%.

    Carry on. Please.

    Jonathan

    ReplyDelete
  30. What do you think about the Cypher Labs AlgoRhythm Solo? Does such a device really output a "better" analog line signal than from an iPod itself?

    Thank you.

    ReplyDelete
  31. @njung40, The only thing really wrong with the iPod's line output (LOD) is the level is low at only 0.5 Vrms. That's not a problem at all for a headphone amp, like the O2, with adjustable gain. But it could be with some other gear.

    The DAC in the iPod Touch is already impressive and very likely audibly transparent. One of these days I'm going to try to some blind tests to confirm that. Unfortunately it's tricky requiring either two iPods with identical settings in very close time sync, or somehow getting both the native analog output simultaneous with an external DAC which will add some latency and want to monopolize the dock connector.

    If you're using an audio system that already converts everything to digital anyway, such as a car audio system or A/V receiver, using the iPod's digital output is the best way to go as it avoids an extra A/D and D/A conversion. But if you're using the iPod as source for something that's entirely analog, like a headphone amp, I don't believe there's any audible benefit to an external DAC mostly because the DAC in better iPods is already very good. And you may introduce more jitter using an external interface where the clock has to be reconstructed.

    ReplyDelete
  32. Gosh nwavguy you like prophet in audio world, who will bring us from non-sense mythical audio world into era that everything clear and can be proven, as mesengger i can tell that you did really good job :)

    There's no doubt that your method is right and can proven, but is that necessary if we just heard with our ears?

    or you want to say there is no diferencies using your o2 with really expensive head-amp like, graham slee line up, burson, or schiit? in tearm of sound quality

    sory for my crap english

    ReplyDelete
  33. "Stereophile did a full set of measurements on the Xonar STX. Indeed they found the output impedance to be uncomfortably high which seems to be a problem with many Xonar products (see my U3 review). I'm not sure a computer company, despite all their resources, will always do well as a high-end audio company."

    Well, other than the high output impedance - which, in the case of the STX, is more of a limitation of the TPA6120A2 - the Xonar internal sound cards do seem to perform quite decently, even the lower end ones. That suggests that either the engineering is good in the non-headphone aspects, or it has become easy to produce good DACs by simply following the recommendations of the chip manufacturers and not trying to apply snake oil, or there are unknown flaws that cam only be revealed by more tests with high end analyzers.

    ReplyDelete
  34. @kura-kura, I'm hardly a prophet I just am trying to counterbalance some of the hype with credible science and expose some audio myths for that they really are and what's behind them.

    @Cyäegha is correct about the skin effect. In fact it won't have audible effect even under 100 Khz.

    ReplyDelete
  35. The point that hydrogenaudio strives to be the most objective forum on the net... I accept. But I cannot accept that they are less biased than others. I´ve said it before: subjectivists and objectivists are two sides of just one coin. One could say that hydrogenaudio is the objectivists' lair and the stevehoffman forum the subjectivists'.

    If I´ve overread your comments about a DBT not creating fact then I´m sorry. The thing is that I don´t want to challenge DBT, I use them myself, just like any other tool for avoiding perceptual results they can be extremely useful. But if one wants to be scientific one has to split hairs. 'Cause if one doesn´t results get misintepreted - and that´s sadly common. Furthermore, science never says "something is always this", with science things are always in flux: one result can be challenged tomorrow.

    The McGurk Effect: It has nothing to do with auditory expectation. The McGurk effect can only work as a combination of audio and visual. Expectations we have when using audio stuff seem to be similar but they aren´t because the McGurk effect has nothing to do with music but with speech recognition (which is something else entirely and also uses different parts of the brain). You cannot combine both into one just because both use the ear.

    I´ve read the additional paper Meyer & Moran published (that´s why I knew what sources they were using). It´s great that they did that (because of comments at AES) - but their conclusions are still fishy. They turn the null hypothesis around and twist it so that in the end they didn´t proove anything (if one would read their conclusion only). The study itself was of course good, I don´t want to discredit their work - far from it! But they still don´t seem to understand statistics. That´s also why I´ve written wondering about their article being peer reviewed. The usual AES articles are much more valid and less "racy".

    You have to understand: we both have the same goal. Just like you I hate that audiophiles are parting with money under false pretenses. But you are so wonderfully detailed in your articles that I felt that some things were underreprsented. I´m sorry if I have insulted you, that wasn´t my intention.

    ReplyDelete
  36. @ kura-kura: Good testing methodologies are needed because we don't just hear with our ears - our hearing is processed by our brain, and we end up hearing what we expect to hear (depending on what we see, what we have been told by others... and sometimes, we just imagine things). It's one thing to say that we just want to buy gear that "sounds good to us"; it's another to realize that some very expensive gear may only sound better because our brain is trying to make sense of the cost figure (or because the casing looks shiny).

    Of course, that's no reason not to buy crazily expensive gear if you have more money than you know what to do with - but otherwise, it's good to have some sense of perspective on what we can actually hear and what is down to psychological effects.

    @ NwAvGuy: actually, it won't have audible effects above 100 KHz either - since those frequencies aren't audible. ;D

    ReplyDelete
  37. Thank you NwAvGuy your great work both on design and education !
    I hope the positive feedback of most folks here will lend you power
    to carry on this blog in the same spirit !

    Regarding cable talk I propose to consider RF isses as well!

    Cables can collect RF signals from the environment and couple to
    the audio box. If the box is not designed well it can demodulate
    these RF signals into the audio band due to non linarities.
    Or add some jitter to the SPDIF or master clock.

    This is the case when you hear the incoming GSM call prior the
    handset starts ringing.

    RF pickup highly depends on cable strcture an arrangement.

    However I believe it is the audio box that shall be protected
    against RF, not the cable shall be over complicated.

    ReplyDelete
  38. @anon, I agree about RF. I designed the O2 input with RF filtering for exactly the reasons you mention.

    Some audiophile cables are likely worse in that regard (i.e. Kimber Kable's completely unshielded designs, etc.). I suspect RF shielding among reasonably conventional cables probably is more similar than different regardless of price.

    But I want to stress RF problems are nearly always obvious. You'll hear, at the least, some clicks, pops, thumps, chirps, a local radio station, voices, etc. that don't belong. I cannot imagine a way in which RF would cause things like loss of soundstage depth, loss of detail, loss of bass impact, or all the other things audiophiles claim differ between cables without there being some grossly audible clues it's an RF problem.

    If you really are having audible RF issues due to an unusual environment that's one those rare cases where balanced interconnects likely have a significant advantage. But just replacing the gear, rather than the cables, might solve the problem even with unbalanced cables. Properly designed gear should reject reasonable amounts of external RF.

    @Anon, regarding the Xonar STX I agree it performs well, at least in John Atkinson's PC (the internal noise environment and quality of the power supply lines in PC's differs widely). But, honestly, if you understand the importance of a low output impedance, why would any competent designer choose an IC like the TPA6120 that has, at best, a 10 ohm output impedance? There are far better choices they could have made, but didn't.

    @Marlene, you're entitled to your opinions but I still mostly disagree. For example, when a recording engineer changes a setting that, inadvertently isn't active, yet he hears the expected change anyway, that's also expectation bias. He sees (vision) the new settings, feels his fingers making the change (touch), etc.. It's really all the same phenomena from the perspective of the entire auditory system. Perhaps Audioskeptic might like to comment? He's one of my references and has extensively researched the human auditory system. I will defer to him if I have it wrong.

    ReplyDelete
  39. Just read the STX review at stereophile. Indeed 10ohm is pretty high for my current headphone (M50).

    Well, I'm currently looking for a good (preferably <$200) sound card/USB DAC for my PC because the current onboard audio is very noisy (buzzing when there's I/O operation) and I believe it's also have a quite high output impedance since the bass is a little bit lacking compared when I plugged my M50 to the Clip+.

    Does anyone have any recommendation as a stopgap solution before the ODAC is available? Currently thinking about the E10.

    ReplyDelete
  40. I've added a paragraph to the end of the Tech Section of this article outlining a nearly $4000 device being marketed as reconstructing hidden audio information. They have full page ads in the big magazines and an obviously substantial marketing budget (unlike the guys linked above selling jars of rocks). I'm curious if anyone has any theories on what the device actually does? I've also cleaned up several typos and made other tweaks to the entire article. Sorry for all the errors.

    @Anon, the E10 is a good choice--especially if you're looking to keep the price down. It has enough power for the M50 and it should solve your PC noise problem. Especially if you're willing to use the volume control on the E10 (leaving the PC volume at 100%) it's a decent DAC for the price.

    ReplyDelete
  41. I think kura-kura might have been spot-on with the first paragraph of his comment. His choice of words - "prophet" - was maybe a bit poor, but the sentiment was certainly right. You NwAvGuy are the one leading us, the gullible masses, from the dark ages into the age of reason, at least as far as the world of Audio is concerned. That does deserve our gratitude.


    About the O2 vs. Vioelectric, is there anything that would actually degrade in the O2 over time or with usage? Or was that just a more general note on build quality?

    My only issue with the O2 was the tracking at very low volume levels (at about <10% of the volume-knob the left channel is mostly silent, while the right one is still quite audible). That's the setting I usually use when wearing earbud-headphones and not listening to music, but still want to hear notifications about incoming mail and im-messages. I guess I just shouldn't have been that lazy and probably plugged the earbuds into the PC directly in this case .. but yeah, it's not like the tracking performance matters a great deal when listening for notification sounds.

    ReplyDelete
  42. @Grunty, to me a prophet is someone representing a higher power. In my view it's really the ones selling $2000 cables that are asking people to believe in some intangible "higher power". They have no credible proof their cables sound better. None.

    I see myself as being the opposite. I'm not asking anyone to believe in voodoo, magic, things that "have no explanation", or any higher powers. Instead I'm suggesting you might want to put your trust in well proven science that's readily verifiable--no blind faith required. The science of human hearing, audio measurements, and blind testing are well understood and have stood the test of time.

    As for your O2, It sounds like it has way too much gain for your earbuds. You might want to make one of the two gain modes 1X by clipping the resistors (see the O2 details article). That will let you use much more of the volume control's range. With proper gain you shouldn't notice any tracking problems until the volume is so low you can barely hear the music. If that's not true, you have a defective (or improper) volume pot in your O2.

    The first thing prone to wear out in the O2 or a Violectric are the jacks and the volume control. And the higher end Violectric amps have a big expensive Alps RK27 volume pot instead of the smaller Alps RK09 in the O2 (which is also used in the Benchmark DAC1). The Violectric V200 should last longer. They probably use better jacks as well. It also has a built-in power supply, more gain options, and other features. In this case you do get something for spending nearly $1000 for the V200 vs $150 for an O2--but better sound isn't one of those things.

    ReplyDelete
  43. The Violectric options and some others out there also can power particularly insensitive headphones like HiFiMAN HE-6 and long-discontinued AKG K240 models, to definitely higher volumes than the O2 can. Not many headphones realistically need more than 7V rms, but they exist. This has been covered before.

    That qøl™ Signal Completion Stage has really got me stumped. I thought the Creative X-Fi Crystalizer marketing was funny enough already, but that actually does something. In the very least, it does a smiley-EQ with increased loudness, maybe some others. They advertise it as supposedly counteracting what is lost with lossy audio compression. But the obvious question is about how they can tell for sure which information has already been thrown away, which they clearly can't. qøl is taking it to a whole new level, inventing a new type of audio component. I wonder what's going on inside that box. Whatever it is, it's clearly beyond the realm of human understanding...until just now. Maybe just some wires? Maybe an indestructible black box with unknown alien components inside? ;)


    On a side note, in some circles, they refer to a cult of NwAvGuy in somewhat derogatory terms. Wherever it is, sign me up! j/k. I just like having question the body of audiophile mythology that's become so pervasive. We need more real experts like you (and real data, which takes plenty of effort and time that isn't going unnoticed), and more types in the industry like Ethan Winer, Doug Self, etc., talking about these things.

    I must say that most of these general articles are not really news to me, but I still pick up a lot of interesting tidbits. At least for Yaniger's setup, 6 buffers okay but not 7, is quite an amusing and informative result.

    ReplyDelete
  44. The post glanced on MythBusters avoiding stirring the pot. Here's it straight from the horses mouth: https://www.youtube.com/watch?v=hq7kBhts9a8. In short, the actual MB guys wanted to bash Visa, but a room full of lawyers convinced the producers to think again.

    Also here's a thought about wine: http://www.futurity.org/science-technology/consumers-can%E2%80%99t-taste-what-wine-buffs-sense/. It's not all just a sham, but the people who can make it as a wine reviewer just might not taste like you and I do. But of course, that just makes the wine reviewers even more useless for the common people.

    ReplyDelete
  45. Great and VERY LONG article! That said, even though there are a lot of subjective terms such as 'tightness', a lot of them can be explained in an objective way e.g. in that example, how accurately the driver recreates the analog signal. I have no problem with subjective terms like that as long as the objective measurement back it up. That';s the same with anything really. You hear a difference? Okay. Is it backed up by measurements? Yes. Then what you are hearing is real. The scientific method. Same with anything. That's what it comes down to really. That said, I've never fallen for the McGurk effect videos. Really. lol.

    ReplyDelete
  46. @Marlene I must point out that expectation created by sight during an auditory test is an expectation effect. It is difficult to suggest that anyone of normal (i.e. non-pathological) cognition can separate out the various cues from vision, hearing, and other senses, given that each of them can create expectation on behalf of the others.

    Likewise, I must disagree with your rather strong dismissal of DBT's of various sorts. DBT's certainly must always speak in probabilities, but in fact when audible effects appear, the test results lead rapidly into the "once per universe" catagory, making the chance pretty clear.

    Things like signal detection are a DBT of another color, no more or less, and combine self-training (such as learning to focus attention in the blog post that NW has pointed)and are very valuable, but are still DBT's.

    For determining if something actually sounds different, a DBT or a cogate like signal detection are the only presently reliable methods that do not require surgery or intrusive medical sensors (which themselves tend to affect the subject's performance quite a bit). We can quite reliably show the response of the auditory periphery, for instance, by doing a subjective test.

    And, of course, DBT's two sole purposes are to eliminate inadvertant bias and provide a potentially falsifiable nature to a test.

    If we are to talk about the aesthetic side (meaning more than sound) of something, in order to determine one's PREFERENCE (as opposed to determining if such a preference arises from audio differences or not), one uses what one PREFERS to use, no more, no less. Preference is pretty much inviolate, but is also strictly personal in nature.

    ReplyDelete
  47. Hi,

    I had a chance to use Violectric V200/V800 combo.
    They sound brilliant on my T1 !

    I also took them apart and, hmm must say, besides the
    nice boxes, connectors and differential stuff, the
    electronics could be built from the same DIY budget
    like ODAC/ODA.

    Both designs are very simple and straightforward, no
    sign of snake oil. Both PCBs really look more DIY than
    professional products, no SMD is used at least in V200.

    V200 has pretty much the same structure like O2, the
    only major difference is that they use a customized
    discrete diamond buffer with feedback instead of high
    current opa. Probably it is cheaper than BUF634/LME496x0
    and can put out more current too.

    The capacitor bank is also bigger in support of the
    diamond buffers and the double HP connectors.

    All together I think both V200 and O2 shall sound equally
    excellent. V200 may have the edge that it can probably drive
    extreme cans like Hifiman better, or when you regularly
    drive 2x low impedance cans.

    However what I prefer about O2 design is the simplicity
    and probably NJM4556 represents today the optimum know
    solution in terms of linearity / quiescent current consumption.
    Which is good for battery powered HPAs.

    ReplyDelete
  48. Great read.

    I'm a PC geek, so I tend toward the objective, however I got in Audio years ago because I wanted my games to sound as immersive as possible.
    For the longest time I was curious about Valves. I tried to go in to it without deluding myself in to some kind of 'magical experience'.
    I've even picked up a set of NOS valves as 'upgrades'.

    ABXing valves doesn't seem practical, and with 3 types per channel, a complicated mess. Perhaps I can obscure the amp from my sight and ask a friend to choose from a few sets and swap them in and not tell me when or what.

    I'm not convinced that tubes within the same family and are electrically compatible sound different at all. Good Resources on valves are few and far between, the best you can hope for is a datasheet, everything else is what some guy heard from an old engineer he spoke to years ago. I can't even get a straight answer on whether they last longer if cooled (USB fan drops the temps by a third), or how long they can last. Every man and his dog as a different view.

    But when it comes to PC related Audio, there is some claims to make you say "Bullshit!". Claims that .wav is better than flac due to less processor overhead to the notion that audio files are better preserved within uncompressed zip archives less they suffer from 'data rot' (which of course only affects audio data)
    DACs in general are great for PCs, but when people start trying to infect the PC scene with their borderline audio religious rubbish like USB cables etc. I want to scream.

    ReplyDelete
  49. Thanks to all for the latest batch of comments!

    @Audioskeptic, thanks for the added info on expectation bias and DBT (relative to Marlene's comments). It's great to have your contributions on how our ears and brains really work.

    @Anon, while I haven't dissected all the Violectric amps I mostly agree with your comments based on what I know from others, photos, etc. You are correct their amps, especially the V200, can drive extreme cans--like some from HiFiMan--better than the O2. And you're correct they otherwise are both transparent amps and the O2 wins on simplicity and is also battery powered.

    @Zeruel, in my experience there are significant differences between tubes of the same number and type. And there are significant difference from when they're new and when they've had a year or two of use. Further, the NOS fanboys need to know tubes degrade just sitting on a shelf for decades. The metals inside out-gas and the glass envelope often (very) slowly lets air in.

    There are especially big differences in what is supposedly the same tube but made by different manufactures. In other words a Chinese 5998, Eastern Europe 5998, and NOS 5998 may all measure, and sound, very different.

    Single-ended tube amps are typically already very marginal audio performers. That means they're far from being transparent under the best of circumstances. So tube rolling is likely to have audible effects. On the one hand some love endless tweaking, trying new tubes, etc. That's part of the hobby to them. But if you're just looking to enjoy music, tubes are more of a liability than an asset.

    You're also correct it's difficult to do blind comparisons. The best bet would be two identical tube amps and you could do an ABX between them in real time with instant switching. I suspect, with different brands of tubes, you would detect some differences. Picking a favorite, however, might be more difficult.

    @tpievila, thanks for those links. In many "industries" a lot goes on behind the scenes that's often deliberately kept quiet. The BBC TV show Top Gear filed a lawsuit against their former racing driver (The Stig) to keep him from publishing a memoir that revealed behind-the-scenes information about the show.

    A few audio companies, that I won't name here, are well known among those in the industry for throwing money at their lawyers to squelch those casting their gear in a negative light instead of spending the money to improve their products. It's sad but true.

    ReplyDelete
  50. I am not surprised at all that Headfonia tried to put O2 down.

    Mike, the Headfonia chief, who is also known as tingm on audiophile-id.com; was biased negatively against NwAvGuy before everything happened.

    In one of his threads (Objective2 Thread) at audiophile-id.com, he explicitly mentioned NwAvGuy as "that uDAC basher" and claimed that NwAvGuy was being too arrogant for him and his friends to stomach; so he tried to get an assembled O2 to "review". (the post was made last year if I remember correctly, so it was posted before he received O2 from JDSLabs)

    The rest is history.

    I apologize if this post looks way too provocative for general public view, but I simply just wanted to share what I know behind the curtain. Feel free to delete/moderate this post, as this may not be appropriate for public view.

    ReplyDelete
  51. @Computer Dude, thanks for the comment. I'm OK with it as it seems to be you're reporting factual information that's a matter of public record.

    While on this topic, I received an email from someone who supposedly tried to post a link to this article as a comment to the Headfonia O2 review. He reported it was immediately deleted by Headfonia. It made this person sad they were behaving like Head-Fi and trying to suppress open discussion in favor of their own bias or agenda. I'm curious what would happen if someone who was more a regular poster at Headfonia tried to post a link there?

    ReplyDelete
  52. Thank you for another great article! It's always a breath of fresh air to read your posts.

    Now, I was skimming through the xiph.org article on 24/192 music, and that part about signal reconstruction got me wondering:

    1. Do you NEED to use the Whittaker-Shannon interpolation formula or equivalent to have an exact reproduction of the original (as long as you respect the sampling theorem of course), or will any kind of interpolation work (possibly producing other high frequency inaudible noise as a byproduct)? If that's the case, is the Whittaker-Shannon formula the best possible interpolation?

    2. What is, for instance, a pure 20KHz signal? Is it a sine/cosine 20KHz wave? Will any other kind of wave always have other higher frequency components?

    Thanks a lot! =D

    ReplyDelete
  53. @vittau, Yes, a pure distortion free 20 Khz tone is indeed a perfect mathematical sine or cosine waveform. And the Nyquist Sampling Theorem upon which digital audio is based, is more correctly called the Nysquist-Shannon Sampling Theorem as it's based on the work of Whittaker-Shannon.

    But it's not like you need a DSP doing a bunch of complex real-time math to decode digital into analog. The old school PCM DACs did it with analog filters.

    Measurements reveal how accurately a DAC reconstructs the signal. Any deviation from mathematical perfection produces detectable distortion components (at least detectable down to way below the threshold of audibility--the visible noise floor of the dScope using FFT averaging is below -150 dB or better).

    ReplyDelete
  54. Regarding the arguments about DBTs:

    I don't know how they did it. I don't know who thought of it first. Nonetheless, audiophiles have somehow managed to attach, for the overwhelming majority of people, blind tests and objectivists, not merely in an associative coupling, but in a way as to imply that the former is required to validate the stance of the latter.

    For individuals such as John Atkinson at "Stereophile", this was a godsend. He has written articles about the flaws of various tests he managed to dig up, about how a null hypothesis doesn't prove the absence of an effect and generally included all sorts of attacks on blind testing. Regardless of their validity, this has succeeded in its aim of creating the illusion of debate: lots of people arguing over how to interpret tests...must mean that nobody knows the true answer. It is this seemingly enlightened viewpoint that many feel they have thus arrived at that allows John Atkinson to write in the comments of his articles how the null hypothesis not proving the absence of something is a source of "immense frustration" to the skeptics and allows various condesceding individuals to descend into forum threads to declare that no real science is being done (and then go back to demagnetising their power cables).

    This is complete bollocks.

    Audiophiles would still be full of crap if nobody had ever conducted a single blind test to investigate their specific claims. Suggesting otherwise is a horrible distortion of the burden of proof.
    It goes like this:

    1. Science says we know about X parameters of an amplifier. They seem to sum up everything we could possibly hear and we have no reason to think otherwise (at present).
    2. Audiophiles say that this is all wrong and they regularly hear things that seem impossible.
    3. People point out that there are perfectly reasonable existing scientific reasons for what they heard. The burden of proof thus falls on audiophiles to demonstrate that what they are hearing is not due to existing and well-documented mechanisms, but due to hitherto undiscovered audio parameters.

    The fact that they respond to this by trying to invert the burden of proof says everything you need to know. Audiophilia is a collection of unfalsifiable (therefore completely meaningless, in the same way that it's meaningless to say I am followed by undetectable gnomes) claims and should be treated as such. If they can't satisfy the burden of proof, why should audiophiles get any more time than, say, parapsychology enthusiasts? Or UFO nuts? We may not know everything about audio, but audiophiles certainly aren't any help, at least as far as amplifiers and DACs are concerned.

    It's that simple. We don't need outrageous philosophical sophistries or ridiculous arguments about the scientific method. There is no friggin' debate: just a bunch of people with an irrational faith that audio is magic. And you can't argue with faith: but you have to, because otherwise faith becomes fact.

    I wouldn't care, were it not for a combination of large quantities of people wasting money, and audiophiles managing to completely sabotage a perfectly good hobby: their outlook has completely infiltrated all aspects of audio reproduction. Were it not for NwAvGuy, I would have most likely spent an extraordinary sum of money on amplifiers from AMB and a DAC from China which it later transpired was prone to frying itself. It never even occurred to me that things often didn't sound different. If I grab a magazine from Richer Sounds, the most popular mainstream UK Hi-Fi retailer, it is full of subjective comments on every single product from an affiliated mag. As far as someone who hasn't done serious research is concerned (not to flatter myself, but you really do have to look around), faith IS fact. Why would you assume otherwise?

    And they wonder why Peter Aczel and the admins over at HydrogenAudio seem so disgruntled!

    ReplyDelete
  55. @Willakan, thanks for taking the time to post all that. I agree. This link was buried in the Tech Section but is worth a look. It's a parody on a camera blog about contractors and hammers. And while some might just think it's funny, it's sadly not far from the truth on a lot of audio forums. You don't want a $2 Chinese no-name hammer the head is going to fly off and hurt someone. But a $15 name brand hammer from Home Depot will probably get the job done nicely. If someone looking for advice on a hammer encountered this they would leave the webpage. But in audio many take the bait. Hammer Forum Thread Parody

    ReplyDelete
  56. Oh, I read that. It's one of those laugh-and-weep-alternately sort of things.

    All it's missing is it transpiring that one of the hammers is prone to snapping into razor-sharp fragments, and the manufacturer chipping in to claim this was a deliberate design feature as not to compromise the fidelity of the nailing.

    ReplyDelete
  57. Hi everyone.

    For weeks I keep this blog, since I started researching soundcards and USB interfaces first and then DAC. And I bought my first hifi/studio headphones (Samson SR850, 32 ohms).

    Question: ODAC for sale will also amplified headphones? Ouput impedance?

    - Greetings from Spain -

    ReplyDelete
  58. I bought these headphones cheap (31 €) to check if to verify if I was comfortable with them in the head before buying a more expensive (I recommend strongly the AKG 701/2 for its sound and, above all, for comfort).

    At first, it sounded much better than my speaker system connected to the PC, but last week I set foobar2000 player so I like it better than the Samson SR850!

    Nauscopio Scipiorum foobar2000: ASIO, Kernel Streaming, Equalizer, Dolby Headphone, Channel Mixer, SACD, Visualisations... in english

    $ 300 = 230 € today. Hummm, if it is as good as the measurements can be a bestseller. I have a friend (via F2F RetroShare) interested too. Asus Xonar Essence One USB DAC and Headphone Amplifier: $561 in Amazon, and probably it has 10 ohms of headphone impedance like ST/STX.

    Thank you for your explanations. You have convinced me.

    ReplyDelete
  59. LOL@willakan, too funny: "...and the manufacturer chipping in to claim this was a deliberate design feature as not to compromise the fidelity of the nailing."

    They would also try blaming the offending nail for being too hard. Had the hammerer tooth-tapped the nail, surely they would have heard the strident dischord in the resulting sound. This is due to the crystalline structure of the nail being malformed, leading to an overly hard area under the nail's head. In effect, we are saving you from using overly hard nails. BTW, shattered hammer heads are not covered by our limited warranty.

    ReplyDelete
  60. 250-300 USD for the ODAC+ODA sounds great :D

    I know the S/PDIF issue has been discussed to death in the other article and I don't want to bring it up again here, but maybe sometime in the future you'll want to do a more complete/detailed article on the "Pitfalls of S/PDIF". S/PDIF input for the ODAC would have been an elegant solution to having a transparent DAC without any pc-related interference, but still not having to miss out on the pc soundcard's DSPs (actually just the HRTF by Dolby Headhpone or CMSS-3D). But then again the frequency response, when gaming and needing those DSPs, is not that terribly important, as long as you can get rid of possible audible distortions and noise from within the pc - so a D3 might be a perfectly acceptable compromise. Or would that still result in the audio signal running through D/A-A/D converters while still inside the pc in order to apply the DSP?

    Anyways, what I actually wanted to ask, since this article is on "What We Hear", what do you think this of this "attachable" headset-like boom microphone: www.modmic.com? The manufacturer claims it has been blind tested and is inaudible (the irony being that that claim is made on a head-fi forum thread), and here ( http://goo.gl/mQrnY - scroll down a bit) a 3rd party tried to measure it, but the graph doesn't really look all that revealing. I guess it's probably too small and too far away to have any em-effects on the coil/magnet inside the headphone, but my main concern is wether it could suffeciently distort the airflow of open-back headphones or cause sonic resonance effects with closed-back ones. Do you have any experience with this kind of stuff?

    ReplyDelete
  61. Samson SR850 are in the $50 range for headphones, at least in USA. ODA+ODAC should drive them and most other headphones quite well, but let's not be too hasty here.

    SR850 are supposed to be good for the price, but they're kind of cheap and probably sound mostly just fine out of most sources people have, unless there are weird background chirps and hisses. If you don't think you have a problem, then you don't really have a problem. You shouldn't need to buy anything more to get a decent sound. If you want a slightly better sound in various ways, then you can upgrade to ODA+ODAC (or realistically, FiiO E10 or something cheaper should be okay too for those headphones).

    But for the cost of ODA+ODAC, you can buy significantly better headphones anyway. In a modern system, the headphones (speakers) are where the majority of the improvement comes from. As a result, it's mostly people with much more expensive headphones that are realistically looking to get the ODA+ODAC. Of course even with headphones, higher price doesn't necessarily mean higher quality.

    ReplyDelete
  62. Regarding the QOL "Signal Completion Stage": At the end of "The Absolute Sound" review, they have an interview with the designer, and he indicates that the box basically feeds out-of-phase signal from one channel to the other channel. He appears to vary the amount of feedback in various frequency ranges (he calls it "Phase Layering"). Apparently he may also limit the "primary" signal and/or replace it with opposite phase in various frequency ranges. I could be wrong; some of it is difficult to interpret. In any case, I think variations of that have been done in the past; it can certainly cause some interesting imaging effects on stereo speakers. I think "Signal Completion" is a bit misleading, though!

    ReplyDelete
  63. @Grunty, Yes, it's probably fine to either use the PC's headphone/line out jack for gaming, or if it's not up to even that task, use a FiiO D3. You still use a USB DAC for music.

    Yes I do plan an article on USB vs S/PDIF from PCs someday. But there are lots of things on the "someday list"...

    I'm not expert enough on headphone acoustics to predict what a boom mic might do. With a closed back headphone you should be fine and even open back will probably be OK as they've made it really slim in profile. The force from the magnet is also unlikely to be an issue. Magnetically shielded speaker drivers for use near CRT monitors use a big "buck" magnet glued to the back of the main magnet to cancel much of the stray magnetic field. And while they do have some effect, it's not drastic. In this case, because of the air gap, I think you would be OK. Free field magnetic force decreases exponentially with distance.

    @mikeaj I agree with your comments. If it's a choice between a better headphone DAC for the Samsons, or better headphones, I'd go with the better headphones.

    @Paul, thanks for the added info. It sounds more like classic analog spatial processing (not unlike crossfeed for headphone listening or Bob Carver's Sonic Holography for speakers) which has nothing to do with reconstructing musical information as the new spatial information is rather arbitrary rather than accurate. Besides, I think you can probably buy a used Sonic Holography unit for less than $100--they were only a few hundred new.

    @Paul, You submitted your post twice. I take that to mean you didn't think it worked the first time? I was hoping this new pop-up window dialog for comments would be free of whatever was happening with embedded comments. Can you please share here, or send me a private message, as to what happened the first time?

    ReplyDelete
  64. Oh, and as a note about phase shift. It is in fact possible for phase shift to be audible in a monophonic signal. A good example of that is using a really sharp all-pass filter.

    It does, however, take a lot of phase shift over a short range of frequency, coupled with the right signals, to create such a problem. It's not something that is easily done in analog, and not something that usually happens in digital processing, either.

    ReplyDelete
  65. I've just discovered your site and took a look at your excellent articles. I've read only two completely. congrats.
    I want to ask you a question: I usually listen to music in my PC, using my X-Fi elite pro, that fully satisfies my needs.
    In fact, I think that what I like the most is the 10 band EQ and the bass enhancement it offers with its driver.

    Sometimes I end up listening music on my laptop or on other computers that have decent sound cards but lacks any decent EQ (or don't have any at all). I'd like to know if you know any decent software for EQ in Windows that offers an EQ within the OS, not only the music player (like Mediamonkey, the one I use).

    I tried mediamonkey's integrated EQ, but I don't like it so much. It easily distorts the music. however, knowing that Mediamonkey uses winamp plugins, is there a highly recommendable multi band EQ plugin for it?

    Regards,

    ReplyDelete
  66. Hey NwAvGuy

    Stat guy here again.
    Looking forward to the ODAC! (intently)
    Just to clarify in words and get it specifically, for those of us who have already built/purchased and O2, will the ODAC be available in a single board variation?

    If we take out the batteries in the JDS O2 for example, (and desolder the posts)..



    Can we purchase the ODAC individually? As most of us have the O2 here (and there are literally thousands who have it) we really want to drop the ODAC in instead of going out and buying another DAC/AMP.. For those of us will be buying the ODAC this is what we'd like. Can we get some clarification from you on that front, or is the only availability on an ODA product?.. I know you need a break from the ODA/C

    $250-260 + adapter is a really good price for the ODA. It will show people what's really possible and bring many people from outside into the hobby at this pricepoint. Three-hundred does make it noticeably more "prohibitive" for your average layman (from an outside persp. sense).

    I think There will be a huge market at 250 as opposed to 300. 250 is simply more accessible to many more hobbyist 'outsiders' and I get the feeling it will also spread like wildfire throughout high-schools at this price-point. Again, while it is not that bad, 300 starts to become intimidating.

    ReplyDelete
  67. Hi,

    you may like this clip

    http://www.youtube.com/watch?v=A13_QGMtlRE&feature=related

    It also shows a form of hearing expectations, this time in musical
    form and with Bobby McFerrin :)

    ReplyDelete
  68. @Anon, that's exactly the kind of product I've had such bad luck with from eBay. I can tell just by looking at the PC board picture it uses a single voltage power supply which almost guarantees it uses a dreaded Virtual Ground for the headphone amp. That means the headphone amp is probably very marginal.

    They also can only talk about the fancy capacitors and chips used and don't provide full specs. What's the maximum output voltage, output current, output impedance, THD into various loads, IMD, jitter, etc? It's all unknown. Just the output impedance could be a huge problem.

    Products from Aune, iBasso, Audinst, Matrix, HA-Info, Yulong, Audio-GD, etc. are consistently more about slapping trendy parts on a board than solid engineering. I'm not saying they're ALL bad, but everyone I've tested so far has had at least one major problem. I'll be comparing the ODAC to just such a headphone DAC in the next few weeks.

    @Stat guy, Yes, absolutely, the ODAC will be available as an assembled board (no enclosure) to wire into an existing O2. And the price of the board, last I heard, will be under $100.

    As for the total price of the ODA+ODAC, most USB headphone DACs fit into one of three categories:

    1 - Those only supporting 16 bit operation over USB (which includes most of the older options).

    2 - USB powered 24 bit headphone DACs that have very limited output capability and hence are unable to properly drive many popular full size headphones.

    3 - Expensive line powered desktop 24 bit but headphone DACs that have enough output power but are at least twice the price of the ODA+ODAC. And most don't have any credible measurements to back up their performance.

    The $660 Violectric V90 + USB 24/96 is a valid comparison. In fact, the ODA has a few features the V90 does not and the ODAC may outperform the Violectric USB 24/96 board (although perhaps not in an audible way).

    Another good comparison would be the $800 Music Fidelity M1 HPA which appears to have an unacceptably high 10 ohm output impedance (based on one review I read) but is otherwise similar.

    So $250-$300 might seem high, but it depends on what you compare it to. If you only need something like the FiiO E10, it's high. But if you have, or think you might someday buy, AKG K/Q 701/702s, LCD-2s, DT880-250/600s, etc, it might be a bargain.


    @Hugo, first off, if you like your X-Fi, stay with it--including the EQ in the driver.

    As for other EQ software there's a lot out there and I'm not an expert on all the options. I tend to use either the standard or DSP plugin for Foobar, or I use one of my professional audio editing suites. But I know there are many other options with some being free.

    When you hear distortion from EQ it's usually clipping. A good EQ should have a "preamp" or overall level control and, ideally, some way to either indicate clipping or automatically adjust the overall level to avoid clipping.

    When you boost a digital signal (even at just one frequency) and the recording already gets close to or hits 0 dBFS (the "loudness wars"), you either have to reduce the overall level to "make room" for the boost, or the signal will be clipped during the loudest portions of the music.

    Do others have suggestions for Mediamonkey plug-ins, etc?

    ReplyDelete
  69. Hi NwAvGuy,
    Normally, the audio source is already merged together, i.e. a number of instruments + vocals together in one signal.
    Will it help if all these are recorded on separate channels and merged only on playback? I'm guessing it should be more realistic. Is there any such technology currently?

    ReplyDelete
  70. @dman, it won't make any difference at all if the tracks are combined before the music is recorded or after. The digital bitstream is identical either way.

    If you mean having a separate DAC for each track and combining them all in the analog domain with something like an analog studio mixer, that's a different question. Such a thing is certainly possible using studio gear, but you would probably want high-end DACs that will accept an external audio clock so they could all be clocked from a single master source. This would a better question for a good recording engineer--perhaps Ethan Winer? Regardless, it's not practical for home playback.

    ReplyDelete
  71. "$ 300 = 230 € today. Hummm, if it is as good as the measurements can be a bestseller. I have a friend (via F2F RetroShare) interested too. Asus Xonar Essence One USB DAC and Headphone Amplifier: $561 in Amazon, and probably it has 10 ohms of headphone impedance like ST/STX."

    The Essence One does not use the TPA6120A2 headphone amplifier chip, but rather the LME49600 instead. So, there is no reason why it should have 10 Ohm output impedance. I do not know of any measurements available of what the impedance really is, though.

    ReplyDelete
  72. @dman:
    You could download individual mixer stems from various open remix contests like a recent (now closed) one from Beyoncé Knowles http://endoftime.beyonceonline.com/ or sign a deal with recording label as a remix artist. Good luck on that one unless you're David Guetta. Then load the stems into any DAW such as ProTools or Logic. Mixing can be done in software manually or with a control surface. The unprocessed stems won't sound like the final version you hear on the CD or radio, however. You would have to set your levels, EQ, comp/ex, limiting, pan/bal plus any various plug-ins. The 2Ch output can go to your DAC via USB for listening. OMG, why bother???

    Once the channel strips are combined together they form a complex waveform and individual instruments/vocals cannot be (easily if at all) extracted. Some center-panned vocals can be extracted via software like the freeware Audacity but it is not perfect.

    So, like NwAvGuy said, "...it is not practical for home playback."

    ReplyDelete
  73. @Anon, that's a big improvement for Asus. Even if they did raise the output impedance it should be possible for an adventurous DIYer to find the resistors and modify it.

    The downside is such a mod could cause stability problems and/or poor transient response. If there are resistors they might be counting on them to isolate reactive loads. The National datasheet talks about it more but the LME49600 is a very fast part and needs careful design to assure stability with all loads. That's what the 10 ohm resistors used with the TPA6120 are for.

    But, with any luck, there are no resistors and Asus got it right. But it's still a lot more expensive than an ODA+ODAC although it does have more features. It wold be interesting to compare it head-to-head with something like the DACmagic Plus which is about the same price.

    ReplyDelete
  74. @mikeaj

    The problem must be the output impedance of the devices that connect the Samson SR850 (32 ohm). In my old Denon equipment (1991) the headphones sound great!. And in a DELL Vostro 15" (24/48) four years old . A little worse on a HP 13" (24/96) a year old, but better than my RK7 :(

    The problem of Samson SR850 is the earpads not the sound. You can change it to AKG K240/270/271 or Beyerdynamic DT770 earpads.

    @NwAvGuy

    ASUS has a problem in Spain: their service is very bad (like most other computer brands in my country).

    ReplyDelete
  75. It would seem that Headfonia is now fully moderating comments on the article prior to them being visible. That's perfectly understandable (if a little inconsistent: you should really either moderate comments or not moderate comments).

    That said, one regular reader commented that it is one of the most negative reviews he's ever read on the site. And I think he might be right.

    They complement the black background, but then declare that 2/3 of the frequency spectrum sounds off (lows and mids). For a site that is very positive about most things they review ("it's great, but X is greater" is about on par) it does seem a little out of character.

    When you bear the audiophile-id.com comments in mind...I somehow doubt my comment regarding that will get moderator approval.

    ReplyDelete
  76. @willakan, I agree but what can you do besides try to present the censored information elsewhere? The O2 is somewhat threatening to a lot of people, websites, and products. If it's really true that you can predict an amp's transparency from the right measurements, and you can, that changes the whole game for sites like Headfonia.

    The O2 represents exactly that predictable transparency. So a natural response is to find a way to reject transparent amps as being desirable. But that's a slippery slope for sites like Headfonia as there are plenty of transparent amps. And to reject the O2, is to reject all of them (like the HPA in the DAC1).

    The other option is to say one transparent amp sound better than another transparent amp. But that's where blind testing can, and will, prove the reviewer dead wrong.

    So sites like Headfonia, as I see it, have three options:

    1 - Give in and concede audibly transparent gear is accurate and generally will sound the same.

    2 - Try to claim non-transparent gear that adds distortion is somehow better and condemn all transparent gear.

    3 - Try to wrongly claim transparent gear all sounds different and make up weak excuses not to participate in blind testing to prove they can hear the claimed differences.

    ReplyDelete
  77. I wish there was a way to confirm the audiophile-id.com stuff, but the site seems to be down.

    Headfonia's opinions on the amp seem to go thus (in chronological order):

    1. audiophile-id.com comments implying they're not favourably disposed towards the O2, to put it mildly.

    2. Facebook comments a few months ago, one of the later ones being "The O2 is definitely here to stay as one of the benchmarks of headphone amplifiers. At least in the $200-$300 range. Good job NwAvGuy."

    3. Review: good amp, but major flaws. Sacrifices everything for a black background. Closing paragraph/s imply that actually you just cheated by lowering the gain and that it sounds worse than a CMOY at equivalent gains...

    Nothing like consistency.

    A cynic (which I am quite clearly not, obviously) might suggest that they started off very ill-disposed towards the amp, warmed up a bit upon hearing it, and then got upset at the comments you made on your blog about Headfonia and churned out a review which attempts to frame their feelings about the amp, which remain largely constant, in a way to firmly emphasise the perceived disadvantages.

    ReplyDelete
    Replies
    1. http://www.headfonia.com/amplifier-recommendations/

      not sure this is EXACTLY what you were looking for, but it's really not helping his case much either. Whole article is a bit of :rolleyes:, but the comments are particularly eye-raising.

      Also
      http://www.amb.org/forum/my-m3-half-built-per-april-4th-11-t1039.html
      >Michael aka tingm of Headfonia (and Audiophile-id too)

      You probably won't be able to confirm or deny what ComputerDude says even if the site comes back up as the wayback machine (the internet archive) brings up login pages for accessing any of the threads on the site.

      Either way I'm sure he's correct about the relationship of Mike -> AMB.

      Delete
  78. Reading NwAvGuy's latest comment on transparent amps and Headfonia, I just came up with an analogy: Windows! The ideal transparent amp is a wire with gain ( or a window pane made out of solid air). So you can think of transparency of amps like the transparency of windows, and any coloration will be like stained glass windows. so preferring a tube amp will be like preferring to look through a green tinted window instead of a clear window.

    ReplyDelete
  79. I've always been amazed by people saying exactly that phrase quoted by Headfonia: Equipment that measure well somehow don't "sound" well.

    As a guy who operates on GCs I have yet to see a column with poor bleed delta to not show up on an MS.

    ReplyDelete
  80. Thanks Willakan and others for the added comments. I agree Headfonia has been inconsistent in their comments. And, as I said above, they're digging themselves into a deep hole with respect to accurate amplifiers of which there are many--not just the O2.

    At the advice of several I've shortened the entire article and also moved a lot of the more geeky stuff into the tech section. The first part is now about 1/3 shorter which should help make it a bit easier to digest.

    I also added some entirely new information like the Stereophile Exposed paragraph. For those with their comments open in a new window you may need to refresh the blog page to see the updates.

    ReplyDelete
  81. Thanks Willakan and others for the added comments. I agree Headfonia has been inconsistent in their comments. And, as I said above, they're digging themselves into a deep hole with respect to accurate amplifiers of which there are many--not just the O2.

    At the advice of several I've shortened the entire article and also moved a lot of the more geeky stuff into the tech section. The first part is now about 1/3 shorter which should help make it a bit easier to digest.

    I also added some entirely new information like the Stereophile Exposed paragraph. For those with their comments open in a new window you may need to refresh the blog page to see the updates.

    ReplyDelete
  82. The O2 can drive LCD-2s? Aren't those electrostatic headphones which means they would require ridiculously high voltages?

    >>But there are lots of things on the "someday list"...

    Heh, yes I know. I'm still looking forward to your article on headphones generally (in-ears vs. over-ears, closed vs. open backs, etc.), and the article on noise/distortions outside the audible range (i think you called it "out-of-band issues"). Though I guess at least some of the latter topic was covered in the recent post on xiph.org regarding 24bit/192kHz audio files.

    Either way, my intent wasn't to pressure you .. I'm in no hurry, and given the amount of detail you to put into your articles it is very obvious how much work is involved in the process. :)

    ReplyDelete
  83. @Grunty while the LCD-2 will give a lot of amps fits, the O2 drives them with lots of headroom to spare. They're planars not electrostatics. They use magnets and a flat diaphragm with a flat voice coil. Another term for planar is "orthodynamic".

    ReplyDelete
  84. NwAvGuy, what do you know/think about this DAC/chips and the upsampled?

    AudioStream Musical Fidelity M1 DAC (the newer one) Michael Lavorgna, 26.12.2011

    [ This one is new and improved as of a November 2011 street date namely adding an Asynchronous USB input capable of handling 24-bit/96kHz data and adding $50 to its price tag (the old M1 DAC's adaptive USB input was limited to 16/48). There are also some minor changes to the choke-filtered power supply but for those users who skip the USB input, the M1 DAC is very nearly its old self... ]

    [ The M1 DAC is also of the upsampling variety meaning it takes every incoming signal and converts it to 24-bit/192kHz using a Texas Instruments Burr-Brown SRC4392 sample rate converter before handing off to a pair of Texas Instruments Burr-Brown DSD1796 D/A chips in dual-differential mode... ]

    ReplyDelete
  85. Someone in defense of Headfonia wrote on their site:
    "Measurements alone do not necessarily guarantee an optimal sounding amp."
    Ok, but what else does?
    That's the whole point, isn't it?;-)

    ReplyDelete
  86. A discussion of this article seems to have disappeared (not just locked) from the "Sound Science" forum of Head-Fi.

    ReplyDelete
  87. I noticed you added a new section about Headfonia. The thing is though, from what I've read on that site, the author seems to completely reject the idea that there are devices out there that can measure past what the human ear is capable of and ensure transparency, never mind the other excuses. He seems to believe not just that one should trust one's ears, but that measurements are essentially useless and should be ignored. . .

    I hope that's not putting words in someone else's mouth, but it seems pretty clear to me if you read some of his non-review articles.

    ReplyDelete
  88. @Anon, I'm not surprised Head-Fi deleted an entire thread discussing this article--even from the Sound Science forum. Everyone should ask themselves why they would do so rather than allow an open discussion?

    I strongly encourage everyone to help get the word out and let people know sites like Head-Fi and Headfonia are censoring factual information about how our hearing works, etc. The great thing about the internet is it's more than a single forum and the truth nearly always prevails in the long run. Jude and crowd have already driven countless people away from Head-Fi with such tactics.

    @Satellite it may well be the reviewer believes the earth is flat--scientific evidence to the contrary be damned. That's his choice. But it's still good to point out the errors in his ways especially when he's in a relatively "trusted position" as one of the few primary reviewers for Headfonia. If someone is advising the public on how to spend their money it's fair to critique their credentials and methods. In my opinion a good reviewer will address his critics (at least the rational ones) rather than censor them and pretend there isn't a problem.

    @Maty, I can't stress enough the chips used in a DAC matter less than the implementation. Short of the cheap integrated C-Media chips in some products, and USB interfaces that only support 16 bits, most of the rest are capable of transparent performance in the right implementation. So without measuring the M1 DAC I really don't know how well they did. I certainly have a lot more faith in Musical Fidelity than in the "no name" companies like iBasso, Aune, Audinst, HA-Info, Matrix, Yulong, or Audio-GD. If they haven't already, someone will hopefully review the new M1 with measurements in the next few months.

    ReplyDelete
  89. Hello NwAvGuy, I just discovered your site and it is indeed refreshing. I have only just begun to read this so I hope I am not asking the obvious, but here goes. I am about to purchase a pair of Hifiman HE-500 headphones. 70% of my listening will be on the road and I have a reasonable library of uncompressed music on my iPad 2. Since I will need an amp (and I am thinking that a 24/96 upsampling DAC would be a huge upgrade over the iPad) I was planning on getting a HeadAmp pico USB DAC/Amp and camera kit. The pico can be powered off the iPad 2. Can your new ODAC be powered off an iPad 2, and if not, will there be a work around? If not, and that would really be too bad, any comments on the pico?

    ReplyDelete
  90. @NwAvGuy If you mean having a separate DAC for each track and combining them all in the analog domain.

    Yes, thats exactly what I meant. I'm surprised its not been implemented in home level equipment, especially when 5.1 and 7.1 channel audio is available. Hmm... good idea to explore.

    ReplyDelete
  91. Hi NwAvGuy, just wanted to congratulate you for your blog and for the hard work you have put on the O2 and ODAC.

    By the way, I made an "objective" comment on the headphonia review like an hour ago and it got deleted :S. (I pointed out that similarly accurate gear should also get bad reviews)

    Best regards from Chile,

    ReplyDelete
  92. @alejandro, thanks for sharing about the censorship at Headfonia.

    @anon, Sorry to bring the bad news but the HE-500 requires a lot of power to play even moderately loud. You're probably not going to be happy with only the 2.8 Vrms the Pico can manage. I believe you need about 5 Vrms for the HiFiMans. They were the most power hungry headphones I could find when designing the O2. The O2 is one of the only portable amps I know of that will do a proper job driving the HE-500 but it has to run from battery or AC power.

    The iPad has very restricted USB power. I believe even the Pico will still need to run from battery or AC power on the iPad.

    I would suggest either an O2 from the iPad's headphone jack or LOD. Or you could get less power hungry headphones and live with a smaller, or even no, amp. The iPad doesn't need an external DAC--it's a waste of money. Much like the iPod Touch, the internal DAC is already quite good.

    ReplyDelete
  93. All of this Subjective vs Objective debate for me results in one very simple conclusion :

    Objective guys (like me) prefers listen to the music the way the artists and recording engineers intended it to be. Therefore focusing on the music and not on the gears as long as the gears are capable of delivering transparent enough sound reproduction.

    Subjective guys prefers listen to how their gears can make their music "sounds better" and more suitable to their "supposedly great" musical taste. Therefore always thinking what to upgrade next and expect what kind of sound character change that upgrade can deliver.

    Also most "audiophiles" focuses more on the quality of the recording rather than the emotion the artists trying to convey to the listeners.

    "Whoa nice soundstage!"
    "This recording sucks, the mids is slightly recessed!"
    ...instead of...
    "Great lyrics! It's make me cry!"
    "Awesome guitar solo right there!"

    ReplyDelete
  94. @Anon, you're correct and I talk about that more in my Subjective vs Objective article. To me, and a few others have commented similarly, it's liberating to trust your gear because it's easier to relax and just enjoy the music. My iPod Touch and O2 with my HD650s, despite all three combined being comfortably under $800, routinely puts a big smile on my face because I'm lost in the music.

    If more people would accept the validity of measurements and blind testing it would be much easier to stop worrying about their gear. They would need gear that measures sufficiently well and/or compares favorably in blind testing to proven gear. Instead, over the last 30 years, it's been moving the opposite direction to where more people look to subjective reviews that substitute someone else's ears and tastes for their own. And, as this article shows, their ears and brains suffer some unavoidable and serious limitations they try to blissfully ignore.

    The link in Stereophile Exposed focuses on how Stereophile vastly expanded the number of their recommended components because it helped sell magazines and ads. He explains the psychology of subjective audiophiles being insecure about their gear and feeling great satisfaction to see gear they own show up on the Recommended list or buy something that's already on the list.

    The irony is, for digital players, preamps, DACs, and amps, the right measurements can provide that same satisfaction of knowing your gear is up to the task. But only if people are willing to accept the validity of the science, blind tests backing up the measurements, etc. When you compare the two, the objective approach is a much more direct path to enjoying music.

    ReplyDelete
  95. Well, since I'm not buying audio equipment for my dog, There is no bloody reason to buy more than what I have (Maverick D2/A1) unless they are products you designed or HD650's (not because you love them, but because I had a pair once and loved them even through headphone out on my computer.)

    ReplyDelete
  96. Anyone else have any theories on this nearly $4K device and their main marketing premise?

    http://ip.com/patapp/US20110158413

    inside pics:
    http://www.6moons.com/audioreviews/bsgtech/1.html

    analog processing.

    It uses opamps... ouch that cannot sound good at all it must be discrete or have glowing bottles.

    It clearly takes apart the original signal and changes it.
    Any change of a signal can be interpreted as an improvement or reducing of SQ.
    A change it is non the less.

    ReplyDelete
  97. NwAvGuy,

    Thanks for this article. A very interesting read as always. One note (sorry if it's been posted before), the headfonia reviewer's name seems to be Lieven instead of Leiven. Should be a simple find/replace.

    Have a good day and looking forward to your Timex/Rolex article!

    ReplyDelete
  98. Incredible read!
    Also, very nice blog overall, been reading for the last 4 hours straight now.

    ps; I was linked here by someone on Head-Fi and when I went back to the topic today, his post was deleted. I messaged the guy and he said this was the second time this has happened. I lost a lot of respect for that site over night...

    ReplyDelete
  99. Hi NwAvGuy, thanks for the effort you put into objective audio gear, I love the O2 amp!
    Can you reveal who will be selling the ODAC and whether there will be any pre-orders scheme?

    Thanks

    ReplyDelete
  100. Assuming the up and coming ODAC(?) is somewhat portable like its predecessor (the O2 amp) would it be possible to have a fully battery powered transportable setup? There are many portable 5v usb portable battery packs around and connect the data source to any Android device with ICS 4.0+ firmware and hacked usb hostmode.

    ReplyDelete
  101. @Solderdude, thanks for the links. Props to 6moons for finding the patent info--apparently I wasn't the only one who was curious about "completing the audio" . In terms of complexity it looks a lot like Carver's Sonic Holography from 30-ish years ago. Granted the devil can be in the details, and development time gets expensive, but $4K is a lot of money for what should be under $40 in parts (excluding the enclosure). There's likely a patent for Sonic Holography and it would be interesting to compare them.

    @Melie, thanks for the feedback. A lot of people have lost respect for Head-Fi. It's fairly absurd the mere mention (not even a link) of another audio website gets censored or even the entire thread deleted. That kind of censorship is supposed to be reserved for obnoxious dictators and cult leaders.

    @Will, JDS Labs should be selling the ODAC and I believe my partner on the project is talking with at least one or two others. If you're in North America, feel free to contact JDS. If you're elsewhere we should know more in the next few weeks.

    @Anon, I guess my own visual memory let me down with the spelling of Lieven. Thanks for the correction. It's fixed.

    @Anon, yes it might be possible, if you're clever, to rig up an external 5V battery pack to the ODAC and run it from an android device. The potential problem is how USB devices get enumerated when they're plugged in. The ODAC will likely need to be powered off when it's first plugged in and then power applied--not the other way around.

    The whole portable device USB DAC issue is kind of sticky for a few reasons. The biggest is the power limits. Any USB DAC with low enough power consumption is probably no better (and possibly worse) than the DAC in the device. So you're faced with using AC or battery power if you really want better performance.

    The Headamp Pico USB, for example, is a more elegant product for that application as it has battery power built in. I have no idea, however, how well it performs. Headamp seems to do a better job than many other boutique one-man shops. But I don't believe he has the means to do proper measurements.

    ReplyDelete
  102. Once again, great article. As someone who enjoys the technical details of how stuff works, I think the real issue with the "subjectivists" boils down to educating them on how stuff actually works. Ironically perhaps, in general they seem hesitant to take some of our conclusions "by faith," and generally become more receptive after learning about the whole scientific process. And I think what you are doing with this blog is slowly accomplishing that. It is too bad that sites like head-fi have basically chosen to lose some of their best contributors this way, but at the same time it's nice that people have a new forum for discussion.

    ReplyDelete
  103. Thank you again NwAvGuy for a great article. I really love your technical sections which go into great detail about your methods and reasoning for the conclusions presented in the first section.

    Something I would like to say, and forgive me if it is out of line, is that you seem to levy an unfair amount of criticism at Schiit audio. Given that the problem you briefly mention has been remedied, I find it unfair to disparage the company without having done a technical review of any of their products yourself or (as far as I can tell) anyone (who is reputable) else.

    Given all your work, I sense you are a pragmatic individual, and as such someone who takes pride in their well thought out and rational conclusions. But it would appear to me that none of the accusations levied against this particular company are founded in any fact whatsoever, only subjective opinion based off of other peoples experiences.

    I do not know whether you are right or wrong regarding the quality of their products, only that one should not judge them before any real technical analysis has been done. Your thoughts are of course always appreciated.

    Cheers,

    ReplyDelete
  104. Hifiman HE-500 / pico - part 2. Thank you for your comments and insight. Since there are very few places to go hear headphones these days, and the HE-500s get such strong universal acclaim, it looks like they would be very hard to beat at $699 (despite the power requirements). I have already listened to and ruled out any AKGs and B&Ws. If there is something else in this class that you would suggest, other than the Sennheiser HD-650s, I am not committed yet?
    Even though the iPad DAC is reasonable (some sites suggest the contrary!), I would still think I would benefit by a 24 bit DAC, especially with higher end power hungry headphones, right? I would appreciate your comments.
    If I do stay with the Hifiman, and a DAC would make a difference (?), it sounds like the ODA with the new DAC would be the way to go (providing that I could live with plugging it into the wall). When will it be available? If I only need an amp, then it sounds like the O2 is the simple solution

    ReplyDelete
  105. Yet another awesome article, thanks nwavguy! This has been keeping me away from studying for a couple of nights now, not sure I should really thank you for that but it definitely was very interesting and allot of new information as well as old information in new presentation.

    After reading the Headfonia article when it was published I was actually furious as there was such a huge gap between their review of the O2 and my experience such that it could not be explained away by taste/preference alone. I googled that night to see if there were any responses as it seemed to me there should be - but couldn't find any so I've let it go.
    ..It's nice to see I was not alone in my feelings.

    A few updates:
    Looks like no new posts have been allowed on the Headfonia site since 14 hours ago... Either they haven't moderated the responses yet or they simply are not going to allow any more comments to be published... quite sad if so as I've always held Headfonia to be a different kind of audio reviews site, maybe not un-biased but at least they seemed sincere and genuine and none of that mafia crap.

    I actually think that my own comment might have triggered the comment lockdown but it might have also been just the timing...

    Adding to Willakan's time line, I also distinctly remember them recommending the O2 to different people in the comments to other articles as well as saying that it is obviously the better amp compared to the E10/E17 due to economics alone (the E10 is cheaper and both have DAC components). I've tried to find those comments but unfortunately couldn't.
    Anyway not sure what caused the big U turn they pulled but would be very interesting to find out (doubt that we will).

    One last thing: the review was not only written by Lieven. He wrote the first half and it was followed by Mike, they both presented the same opinion (once each) so it might be confusing but there are actually two different people behind that review. Anyway I think they are the main two operating headfonia, don't think there is anyone else there (will be glad if someone that knows for sure chirps in).

    ReplyDelete
  106. @Anon, thanks for your comments. I think I've had enough involvement with Schiit Audio (including some that hasn't been published) to form at least a few reasonably solid conclusions about their gear and how they run their business. It's not a single incident, it's several. Their Lyr amplifiers have also damaged headphones. They spoke out publicly as being dead against the use of relays in their products then they release a new product full of relays.

    Perhaps most of all, several of their products are based on designs (low/no feedback single-ended topologies) that are known to be inferior and produce relatively high amounts of distortion and have an undesirably high output impedance. That's not theory, or guesswork, it's fact. Any competent audio engineer can explain why Schiit's approach is a poor choice to drive headphones.

    It would be like (assuming regulations would even allow) Mercedes offering a 1930 Ford Model T engine in their latest 2012 model. It would perform very poorly but, with the right marketing, you can sell just about anything: "Economical Proven 4 Cylinder Engine!" "Simple to maintain!" "Safe low RPM operation!" "An engine just like your great great grandfather had!" Etc... I'm sure some think I'm being unfair, but Schiit's choice of output stages dates back to the Model T and performs about as well.

    @AncientWisdom thanks for your comments. I've received some private messages regarding posts to Headfonia not being published and things are equally as bad, if not worse, at Head-Fi. I wondered about Mike's involvement as several of his comments are in the first person. But it wasn't clear to me they co-authored the review as Lieven is listed as the sole author at the start of the article.

    @Anon, perhaps others may have some suggestions for less power hungry headphones? If you don't expect to listen very loud, and mostly listen to highly compressed pop music, you might be happy with the Pico and the HE-500. It gives about 6 dB SPL to the O2 which is significant but not huge. If you haven't already, I would confirm the Pico USB really does work as a DAC with the iPad.

    24 bits is only an issue if you want to turn the volume down at the iPad and not at the amp/DAC. I don't even know if the iPad's volume control is functional with an external USB audio device.

    The ODA+ODAC still isn't likely to work with the iPad without some hacking (see the android/external battery comment above). The ODA (or O2) alone will work great with the line out or headphone out.

    If you have any links to credible measurements of the iPad's internal audio please share them. I have an iPad (1G) and can readily verify if they're reasonably accurate.

    ReplyDelete
  107. NwAvGuy,

    Thanks for your response to my earlier comment about Schiit. It was not made clear in your article (or any of the earlier ones) that you had practical experience with the products. So thank you for your feedback. If possible, could you reply with a link to sources which show output impedance etc. of their products? I would mighty appreciate it. Again thanks for taking time to respond, and for your contributions to the audio community.

    Cheers,

    ReplyDelete
  108. >>Do others have suggestions for Mediamonkey plug-ins, etc?

    I think I might have found something: http://goo.gl/giz6e (It's a Winamp plugin, so it works with Mediamonkey and some other players too).

    Unline other equalizers, this one allows for an arbitary number of "bands" on arbitrary frequencies .. and plots something like a frequency-response chart that will result due to the changes. I'm not sure how well it works or if it is really able to provide such precision, but it sure is much more intuitive to use than conventional fixed-band equalizers (though you'll probably have to take a look at the PDF-documentation first..). Also it doesn't have any clipping indicators, but it has a pre-amp. (In fact I couldn't find any software-EQ that had clipping-indicators - is that something that is not easily done in software?)

    If you can find time for it NwAvGuy, I'd really like to hear your assesment of this. Maybe you could create a frequency-sweep in Audacity, play it on Winamp (running it through the equalizer), and then measure the output of your DAC-1 with the dScope, and check how close it is to the plotted chart by the equalizer?

    ReplyDelete
  109. @Grunty, it's pretty easy to offer at least "peak" indicators. For example, with 16 bit audio, the maximum numerical value for a sample is 32767. If any samples hit 32767 you turn on the "peak" indicator for half a second and the user knows the levels are likely too high. Pro software, DAWs, etc, typically have this feature. Even the freeware Audacity will analyze a music file and let you know how many potentially clipped samples there are.

    In my experience, software EQ is generally very accurate. Ethan Winer uses Cakewalk Sonar to run two passes of opposite EQ in the myths video linked in the tech section of this article. It's accurate enough that, even after two passes, it nulls almost completely with the original unequalized signal.

    I don't have time right now but I'll add your plug in to the list of things to consider testing in the future. You can probably do many tests yourself if you're familiar enough with Audacity and have the ability to record the output from Mediamonkey. RMAA could provide a frequency response plot using its test file run through the EQ.

    ReplyDelete
  110. nwavguy:
    Yes Lieven is listed as the sole author however page 3 is titled "Mike's Impressions" and from there on (pages 3 and 4) it is all Mike. This is a style of reviewing that they have started using lately where they offer two perspectives on the same product. Not sure why they don't credit both reviewers though.

    Anon:
    Regarding nwavguy and Schiit, Perhaps read the following linked post, it might explain much. I suppose nwavguy just assumed most of the readers of this blog are familiar with that piece of history.
    http://nwavguy.blogspot.com.au/2011/07/banned-at-head-fi.html

    ReplyDelete
  111. Thanks for a very informative post. Just some quick comments on the ABX testing, the wiki article on confidence that was referenced is very poorly written.
    There is some confusion there with regards to statistical significance and the significance levels of confidence intervals of a test statistic.
    As an example, assuming p=.5 (random guessing) and a subject goes through 10 trials. If the subject gets 8 correct guesses the 95% CI is (.493,1) which is not a statistically significant result at alpha=0.05 level.
    The CI needs to be looked at in the context of the null hypothesis of interest to determine statistical significance.

    ReplyDelete
  112. Headfonia is now a site not to be trusted. I have tWo of the O2. I also have the Sennheiser HD650. The combination sounds very good, and better than any other amplifier I have owned. and I have owned quite a lot. I have also used the O2 with Denon D2000, AKG K601 and Beyerdynamic DT770. All with great results. So either the authors of the Headfonia article are not capable of testing stereo equipment, or thay have an agend. Keep on the good work NwAvGuy. Once your Odac is ready, I will immediately buy two of them.

    ReplyDelete
  113. Very interesting article again, thank you.

    The Hammerforum parody reminded me that we have almost nothing conclusive in the internet discussions.

    Basically all of the forums mainly consist of people rambling and creating more confusion. I would prefer actual information.

    If a person says he prefers product 1 over product 2, it means nothing to me. This is the problem with forums, no one states:
    -How much he has passion for the hobby.
    -How long has he been into it.
    -What equipment he has used.
    -What are his preferences.
    -How much he values equipment.

    Also it seems most people have already decided which product they will buy, and are just looking for confirmation for their choice. Here the undefined opinion may do more harm than good.

    ReplyDelete
  114. Censorship at Head-Fi seems to have increased recently, perhaps in reaction to this article. A number of my posts were deleted, apparently for no reason other than including links to measurements here.

    ReplyDelete
  115. I think people have given up debating subjectivity and are now attacking your objectivity.

    http://www.head-fi.org/t/603728/o2-vs-totl/60

    Care to elaborate on this topic?

    ReplyDelete
  116. If someone thinks that "Amp A" sounds better than "Amp B", despite the fact that "Amp A" measures poorly in comparison, it does NOT mean that that person is wrong in thinking "Amp A" sounds better.

    The bias that person might have which leads them to believe that "Amp A" sounds better, actually causes "Amp A" to sound better to them.

    So what you actually seek to do as an "objective" audiophile, is to replace expectation and McGurk biases (which are from things like appearance, price, popular opinion) with a measurement bias.

    There is definitely something to be said for a bias that is based on something substantive, like frequency response or distortion. But to pretend that measurement bias is not bias is incorrect, since nothing objectively sounds better than something else. Even if an artist intended for their music to sound as it was recorded, this is not always what we prefer.

    As someone who is a trained engineer, I can see your motivation for wanting the measurement bias to dominate over other biases, like appearance biases, or biases generated by advertisements.

    Blind listening tests puts the ball in the engineers court. But people don't listen to their music with blindfolds on (well, I don't), and biases actually change the way an amp sounds to people.

    ReplyDelete
  117. @anon, SRS is one of the "branded" DSP processing companies like THX, Dolby, etc. They license their technology to gear makers. None can reliably "invent" additional accurate information beyond what's already present in the recording. They can only add artificial "spatial", EQ, etc. Some find the effects pleasing while others tend to find them rather synthetic and unnatural.

    @Battou62 I've skimmed the thread and it seems like the usual O2 debate at Head-Fi. I didn't find anywhere someone accused me of an objective error.

    Currawong and others make a big deal out of how test signals are not the same as music. And while that's true on the surface, it turns out test signals ARE sufficient to determine if an amplifier is audibly transparent playing music. See: Music vs Sine Waves

    And if they don't like that argument, you can use audio differencing (null testing) to compare the input of the O2 to the output while it's playing real music driving real headphones. And the difference is so miniscule as to be insignificant. Such a test accounts for ALL forms of distortion and noise possible. You're literally subtracting the input of the amp from the output. What's left over is everything the amp is doing wrong. A piece of wire gives a perfect result.

    It would very interesting to run a differencing test on the O2 and the ToTL playing the same music into the same headphones and compare the result. Likewise it would be equally interesting to do a proper blind test comparing the two--but of course nobody can mention that as it's a banned topic--just like using "NwAvGuy" in a sentence. Does anyone really expect the O2 to get a fair shake at Head-Fi?

    Some also have attacked NFB in the thread but they're just recycling the usual myths and misunderstandings about NFB. For one thing, any problems with NFB would show up in the differencing test using real music. For another, they have zero credible evidence supporting their claims while there are countless technical papers and listening tests supporting NFB. And, finally, they should read Bruno Putzeys "There's No Such Thing As Too Much Feedback" article. One by one he addresses all the usual myths about NFB, including those cited in the thread, and proves them wrong with real science.

    Basically the thread reads to me like the usual "more expensive has to be better" argument. People who own such gear get insecure when someone tries to claim something much cheaper sounds just as good or possibly even better. That's been going on long before the O2 came along.

    @eisenhower, you're correct that good measurements, in sighted listening, create an expectation bias. In fact that's exactly what I argued might have happend with Lieven's O2 review. For someone who more highly values good measurements, the bias might work in the gear's favor.

    But that bias is not a way to dismiss measurements. There are LOTS of blind listening tests that support audible transparency based on sufficiently good measurements. I'm talking about AES peer reviewed papers. If there were really differences that can't be explained by measurements, someone would have surely demonstrated they exist with blind listening.

    And, regardless of what you think of measurements, I'm suggesting unless you listen blind, you really don't know what you're hearing is not biased. That's all.

    What perhaps you're getting at I tried to sum up with my crystal wine glass analogy. If one guy thinks crystal versus cheap glass changes the taste of the wine, is it OK to make others believe they need expensive wine glasses even though it's only one man's illusion? I'm not denying crystal may improve the experience for some. But where's the line? It's very clear a lot of people are being misled the way things are now.

    ReplyDelete
  118. Am I missing something? All I see in that thread are the usual "I heard it" arguments and some psuedoscientific magic-distortion-mechanisms-that-are-only-apparent-with-music. This being audiophiles justifying themselves, there's some hilariously technically illiterate stuff, including someone postulating that designs that don't use negative feedback sound better because your brain is part of the feedback loop or something...another claiming that SE amps suddenly become more transparent than the O2 when faced with real music.

    We've even got someone chipping in with the standard sinewaves-are-not-music argument with the dazzling evidence that TOTL amps must be better, as "All the extra parts in TOTL amps aren't there for nothing." Thank you Currawong.

    And they've started quoting philosophers. Here come the pathetically pretentious sophistries...

    I don't see how you can argue with blind faith masquerading as a superior understanding of audio, even if it has been ruining a perfectly good hobby since the 70s.

    ReplyDelete
  119. Eisenhower makes a good point: when you have beliefs about the gear, those beliefs will tend to bias your perception of how it sounds. It follows that unless you're going to hire a Jeeves to serve as intermediary between you and your gear -- leaving you ignorant as to what's playing when -- you're going to be biased.

    So NwAvGuy is arguing for an objective bias: for paying more attention to measurements than to price or bling, with the rationale that a bias in favor of measurements is a bias in favor of something that is at least relevant to sound, unlike a bias in favor of price or bling, both of are completely irrelevant.

    That argument strikes me as putting the emphasis where it belongs. We ought to be interested in what's really going on with the sound, and we ought to pay no attention whatsoever to price or bling (or the sheer number of parts inside the box, or the vintage of the technology, or the spiffiness of the website where they read a review, etc. etc.).

    So the debate isn't between 'objectivists' and 'subjectivists.' It's between those who think it's okay to be biased by information that is irrelevant to sound, in determining how something sounds, and those who think that's on its face irrational.

    I'm in the latter group. If you are devoted to how things sound, you're simply violating your devotion if you let yourself be biased by anything irrelevant to sound.

    This not only reframes the debate but decides it. Is the other side going to insist that price or bling (or any of the rest) is actually relevant to the how the gear sounds?

    ReplyDelete
  120. I agree with Ted this isn't about subjective vs objective. I was happy Nwguy moved away from that theme with this article and instead focus more on how we hear. The Sub v Obj argument can drive the two sides further apart.

    I can confirm null testing is real, it works, and IMHO, it's irrefutable. Anyone suggesting otherwise might as well claim the earth is flat. Like with blind testing, its a damn shame hardly anyone bothers to do it but its no surprise. IMHO most don't want to know the truth or have other reasons to avoid having bargain gear irrefutably rival their super expensive stuff.

    I think this blog is great and its awesome Nwguy isn't afraid to stand up to the established hype machines like Headfi, Headfonia, and AMB. I've seen real changes for the better all over the web like awareness of output impedance suddenly changing after years of indifference. I have to believe Nwguy should get at least some of the props.

    Suggestion to Nwguy: Is there a way you can publish some null tests? I think that would help silence many of your critics who try to argue the O2 only performs well with test signals. Can you show what it does with music?

    ReplyDelete
  121. Thank you nwavguy for your big efforts to bring quality audio to wide audience. Very kind of you, and not only your articles made me think in different angle then before.

    I have only one question regarding ODAC and this is external power: will ODAC be able to work with external USB power without any system errors or problems and if yes, then are there any real world subjective advantages in sound quality or objective better measurements?

    ReplyDelete
  122. @Willakan, Ted, Anon & Peak, thanks for your added comments. I'm generally in agreement.

    @Peak Limiter, I will look into offering some audio differencing (null) results. They can be quantified as an absolute level (unweighted or A-weighted) which makes it easy to compare say two amps. But I think it's also informative to let others download a file and hear the difference signal relative to the main signal. If the difference is audible it's useful to judge how "harmonious" it is--i.e. if it's mostly just benign noise, harmonic distortion, or if it's more objectionable and dissonant.

    I need to look into how to host uncompressed high quality music files without the bandwidth becoming a problem. I also either need unlicensed music or it can only legally be a short excerpt. Studio guys who produce their own music, like Ethan Winer's examples in the Tech Section, are at an advantage in terms of source material. If others have suggestions for license-free source content and hosting please share them?

    @Anon, as mentioned earlier in these comments, it might be possible to run the ODAC from power limited USB ports such as on a tablet if external power is used and the USB connection is made before the ODAC is powered up. But I haven't tested it.

    Based on all the research I've done I don't believe there's an audible (i.e. sound quality) benefit to running the ODAC on external power. It is possible it might measure slightly better in a few ways but external power is not necessary for 100% transparent performance.

    ReplyDelete
  123. NwAvGuy,

    You said: "There are LOTS of blind listening tests that support audible transparency based on sufficiently good measurements. I'm talking about AES peer reviewed papers."

    Please cite these articles that support your transparency critiera.

    ReplyDelete
  124. @Anon, I site a few in this article, including Ten Years of ABX Testing which itself sites a whole bunch of AES papers related to blind testing. There are additional links in the Subjective vs Objective article and you can search on "blind" at AES.ORG and read the abstracts of many more.

    Meyer & Moran is an excellent and relatively recent reference as the A/D and D/A they inserted into the signal path proved entirely transparent under realistic listening conditions over more than 600 listening trials lasting more than year. They used recording engineers, members of an audiophile club, and students as the listeners. They also published a follow up paper refuting their often irrational critics.

    Ethan Winer in the second half of the Audiophile Myths video linked in this article provides examples and criteria. There's also the work of Doug Self, Arny Krueger, and others.

    When you consider what the other side has in the way of objective credible evidence, it's not even close to being a contest. As I keep saying, if measurements didn't assure audible transparency, it would be easy to enough to arrange blind tests demonstrating the inexplicable differences. The upside for high-end audio manufactures of such tests would be huge giving them serious incentive. So why hasn't even one ever published such a test?

    ReplyDelete
  125. Hi NwAvGuy,

    maybe this post is out of topic, but maybe other people might be interested as well.

    I red again your Music vs Sine Waves chapter, and the full Meyer & Moran report. I found M&M very interesting especially point 4 note that the mastering of SACD -s tends to be more careful than standard CDs’ !!

    My question is if you know any similar test series done, but using LP or tape source instead of SACD/DVD?

    I ask it because I hear so much controversy on the analog <-> digital debate. Such a test could simplify that debate too.

    ReplyDelete
  126. @anon, that's on topic. There have been vinyl vs digital blind tests done. There's one linked in my Subjective vs Objective article, more on Hydrogenaudio, I know of an audiophile club that did one, and I've arranged a few myself. The clear consensus is people cannot detect when you insert a suitably transparent A/D and D/A into the otherwise all-analog signal path. That holds true even for very high-end systems priced well into five figure territory.

    The Audio Myths video linked in this article addresses the analog tape issue. I believe it's Ethan Winer who says something like "even a cheap soundcard beats any analog tape machine" and he's including the huge wide format studio machines.

    So, while I'm aware of the debate, once again the "analog sounds better crowd" has no objective evidence demonstrating there's an audible problem with even CD quality digital let alone 24/96.

    But there's some truth to preferring vinyl. It's mastered differently than CD (the same is true of original studio analog tapes). So just like superior mastering can make an SACD sound better than a CD, the difference is entirely attributable to the mastering not the format itself.

    Another factor is master tapes degrade steadily with time. The magnetic particles slowly try to revert back to their "erased" state--especially for high frequencies. Vinyl doesn't have that problem. 30 year old un-played vinyl can sound great. So if you listen to Dark Side of The Moon on vinyl vs any of the CD versions which came much later, all the CD versions suffer from some tape degradation. They try to compensate in the CD mastering but there's only so much you can do to try and correct for lost information on the tape.

    LP's were largely released before the loudness wars got really ugly. And many released since have been audiophile recordings that also have more dynamic range.

    So there are several valid reasons to like vinyl. But none of them have anything to do with 16/44 Redbook CD audio being audibly flawed.

    ReplyDelete
  127. Thanks for the write up mate, always straight to the point.

    Since you mention some of the overhyped brands at head-fi..you think burson is one of them? what do you think about their amps?

    ReplyDelete
  128. @Derek, I haven't tested any Burson gear but they seem to be significantly more into snake oil compared to say Centrance, Cambridge Audio, Benchmark, Grace, etc. Things like class A, discrete op amps, etc. are not required for amazing performance. Burson is also weak on credible measurements or even detailed specs. I'd put something like the well proven Centrance DACmini or DacMagic Plus up against a Burson any day.

    ReplyDelete
  129. @Derek:

    Three things about Burson:
    1. One of their discrete opamps has been tested: bizarrely, it was found to be identical to an AudioGd one (ie awful). They've said before they outsource some of their designs...

    2. Their website has possibly the worst justification for all-discrete headphone amps in living memory. It's barely in understandable english and seriously tries to persuade you that opamps sound bad because they are too thin and the musical texture gets squashed.

    3. They were deliberately deceptive when it came to stating that one of their designs had an asynchronous USB implementation. It didn't, but they told lots of people it did before someone used a USB prober and got a nasty surprise. This was apparently the fault of the people they outsourced it to (wonder if it's the same people they outsourced their first generation of discrete opamps to? It's a small world...)

    ReplyDelete
  130. TYVM for the link to Arthur Salvator, and +1 for logic.

    What's really remarkable is that there are ways of creating a real list of Class A, audibly transparent components. For a headphone amplifier, you could:

    1.) Create a benchmark for when a headphone amp sounds transparent (which NwAvGuy has done masterfully in the InnerFidelity article).

    2.) Measure every headphone amp to see if it meets/exceeds this standard.

    3.) Conduct blind tests with different headphone amps that all reach this standard, and if the listener can't discern the two, they are, without a doubt, audibly transparent for all intents and purposes.

    4.) All headphone amps that survive a trained ear (and oscilloscope) are classified Class A.

    5.) If any headphone amp ever manages to sound better than audibly transparent (with blind testing), all of the current Class A components are demoted, and the bar is raised. (I doubt this will happen)

    What I would give if this were to happen. A LOT of companies would go out of business.

    ReplyDelete
  131. @eisenhower: "nothing objectively sounds better than something else" - You are wrong here. If we can agree on something of higher quality to be "better" than something of lower quality (after all it's the quality that is supposed to justify the higher price, right?), than something would very objectively be better, as "quality" is a precisely defined term used in many ISO-standards (http://www.praxiom.com/iso-definition.htm#Quality). Thus if something manages to reproduce the musical information (the prime function of audio gear, right?) more accurately than something else, then it actually is better. If you like or enjoy that much accuracy is where it gets subjective, and that's where equalizers and headphone characteristics come into play (just as NwAvGuy said).

    If you insist on "measurements being just another form of bias", then maybe you'll want to read Dan Gilbert's book "Stumbling on Happiness", where he talks about "synthesized happiness". He makes a strong case about people who are clearly worse off than someone else (with different levels of extremity, ranging from paraplegics to students facing unfavorable lecture terms) and yet "feel" happier exactly because of their disadvantage. And while this happiness is very real (even measurable!), you'd have to clearly be insane in order to pursue that kind of path to happiness. It would be the equivalent of cutting one of your ears off, so as to be able to feel the joy of still having one more ear that you can use to listen to music.

    The bottom line is, yes equipment A might sound worse than B and you might still enjoy B more, because of some psychological quirks .. but then again, if you really do insist on someone exploiting your brain and charging money for it, you'll find that there are more effective, and "pleasurable", ways to achieving it than buying audio gear - Yes, illusions are slippery slope.


    @NwAvGuy, what file sizes are you talking about for uncompressed files? You could always go for one of those share-hosting sites (like rapidshare), since they don't really have download-limits, but it gets annoying if you have to split files up into multiple uploads. Worst-case scenario though would be just making a torrent, uploading the torrent file on the pirate bay and basically have everyone interested in the files host it. So I don't think that is that big of an issue.

    About licence-free music I don't really have any suggestions. The other issue would be what would really count as "music" and be able provide conclusive proof once you get out of the mainstream. For example would any old out of copyright records be useful? Or maybe just playing fan-made midi files? You'll maybe also want to have a look here: www.publicdomain4u.com - it seems to some kind of index for free music on the internet.

    Btw, it's funny that you mentioned the volume control of the O2 in one of the comments above .. this is what happened to me: http://www.abload.de/image.php?img=o2epiphany5yzk7.jpg - though that's probably Epiphany Acoustics' fault and not yours. ^^
    And the O2 is working great other than that.


    Finally one more thing I'd like to mention, I really like the creative ways people find to refer to your blog on head-fi:

    "Rumor has it that there are such tests published for the O2, [...] You might have to look into the deep, dark recesses of the Internet to find those, from some arcane posting in the month of July."

    ReplyDelete
  132. It's nice to wake up to some meaty comments! Thanks all.

    @Willakan, thanks for the added answers about Burson. They seem to be a classic case of a company that needs a site like Head-Fi to market their apparently flawed products.

    XacTactX, Thanks. That's kind of how it was done by the magazines back in the 80's. Stereo Review, Audio, etc. generally commented on the functionality, ergonomics, build quality, etc. and they left the performance up to the measurements rather than subjective biased hyperbole in a cryptic language.

    The main problem with all top rated components having to be transparent is the Headfonia/Stereophile/Etc. crowd will always want to argue lower fidelity gear somehow sounds better. Your method, for example, would force most tube gear out of Class A. On top of all those tube manufacturers screaming and yelling, lots of owners of tube gear would also be unhappy. They were told by supposed experts to buy that gear. It's a problem with deep roots grown over decades.

    But, obviously, I think it's a great idea. We might end up with more resources offering an objective approach to compete with the likes of Headfonia and Stereophile. If blind testing ever makes a comeback, it will help marginalize Headfonia/et al's following to those who want to put their head in the sand and believe Santa Claus is real.

    @Grunty, thanks for the added comments on psychology and quality. I think the money involved is a big part of the problem for multiple reasons. If someone wants to argue audio sounds better with tissue stuffed in your ears, that's relatively harmless. But if they suggest you need a $3000 DAC or headphone amp to really enjoy your music, that causes (financial) harm. Likewise all those involved in selling overpriced gear, and who own such gear, have a strong incentive to believe, and argue, it's somehow better because of the high price.

    As for file sizes, a single lossless FLAC 4 minute song would be 28 MB. If say 5% of the 5000+ daily visitors here downloaded it, that's 7 GB (56 Gbit) of daily bandwidth or over 200 GB/month. My 5% (250 downloads) is probably realistic when a new article is first posted but it would tape off. That's just one track, there might be several. In my experience a lot of the "unlimited bandwidth" sites have fine print stating they reserve the right to restrict bandwidth and/or charge more.

    The "pirate torrent" option could let someone argue I'm participating in illegal sharing and have the blog taken down. Given all the ToTL & Schiit amp lovers out there, I have to be careful. ;)

    I'll check out the free music link. Thanks. As for your volume knob, the cheap rubberized objective ones stay on very well. Those esoteric knobs made out of grain aligned unobtanium are well known for falling off during a full moon. ;)

    @Anon, the Timex vs Rolex article should be up fairly soon (by this coming Wed if all goes well). Your suggestion of doing "mini reviews" is a good one with the exception of unanswered questions. The less I write in the review the more questions people will have. I'll end up having to either ignore questions or spend a lot of time answering things that could have been in the review. But it's worth trying.

    ReplyDelete
  133. Hi there NwAvGuy,

    Regarding the previous comments on Burson's HPAs, I havent heard one either, but it seems to be on a lot of people's rigs and first preferences. I totally agree with your appreciations of their website, arguments and specs, but I would still give it a try, to hear for myself. I believe a good contrast -in a blind test- would be Violectric's V100, that goes for about the same price and has similar characteristics.

    My motivation is getting the best sound from my K702s, and I believe either the HA-160 or the V100, along with your ODAC would make a great combination. I'm not saying the O2 doesn't have what it takes to drive the AKGs, but as you well pointed out in a previous comment, it doesn't have the same power the V100 or V200 monsters have, delivering current.

    I, like you believe in applying science and being as fair as possible is what really counts, and although I would much rather use the term "intersubjective" than objective, I do believe measurements have a say and a clear value.

    Cheers!

    ReplyDelete
    Replies
    1. You might want to wait for the ODA, or even get an O2 now, and save yourself a bunch of money. Plus the ODAC will fit inside for an elegant single box solution. The K702 doesn't come close to exceeding any of the O2's limits--including current. Having more current available than you need doesn't improve the sound. It's like hooking a giant 6 inch water main to your house won't make one toilet refill any faster. If you said you had HiFiMan HE-5s I might be more inclined to agree with you as they come close to the O2's current limit.

      Based on what Willakan and others have found out about Burson, I would strongly favor Violectric, or any number of other manufactures who emphasize proper engineering over sketchy snake oil. Just because a company has a loyal following, doesn't mean they make good products. The Head-Fi forums are loaded with examples.

      Delete
  134. @vittau:
    1) Already explained ably above.
    2) It is any Sinusoid. No need to delineate between one defined via the sine function (the definition is y(t)=A*sin(wt+p)) or the cosine, since the only difference is phase p (pi/2 radians = 90 degrees). A waveform is a sinusoidal iff one and only one frequency component is present. In other words, the Fourier transform would have one, and only one, "peak".

    ReplyDelete
  135. I think it's a shame you have to rely on companies like JDS Labs or Epiphany Acoustics for offering the O2/ODA/ODAC commercially. While the price they're charging is (somewhat) fair, what I hate seeing is the O2 being marketed on the manufacturers' sites next to other "subjective" products.

    Like Epiphany's EHP-1 where they only provide the THD+N rating of the op-amps rather than of the whole product, or JDS' c421 where they used RMAA to determine the amp's quality. I'm sure there a lot of people that haven't read your blog that carefully, and referring them to JDS/Epiphany to obtain an O2 feels a bit like sending them right into the dragon's lair, where they might easily be swayed into buying one of these other more expensive and thus supposedly "better" products.

    Don't get me wrong, the EHP-1 or c421 might be great and outstanding products, I have never tried them. But anyone who has understood what has been written in this blog would still need to be at least somewhat suspicious about their quality, given what details the manufacturers chose to provide - or are even capable of providing. Ideally you'd do a close-up inspection of anything that is being sold alongside of the O2, but given that JDS and Epiphany are (currently) the only places to get a fully-assembled O2, I can understand the necessity of treading lightly here. (That being said, I assume you have done at least some rudimentary check-up on the commercially sold O2s - so as to make sure the manufacturers didn't cut any corners, etc.?)

    Still, I'd much rather see one of the big-brand electronics manufacturers that clearly lie outside of the "audiophile spectrum" like Samsung or Foxconn produce the O2, since they do have the means of following specs closely but no audiophile authority so as to market other snake-oil products alongside the O2. Even if this would mean a price-increase due to these companies' higher profit margins.

    One more thing - another commenter above suggested you do short reviews of products rather than your in-depth ones, so you can cover more ground. I think this would be a step in the wrong direction. What makes your blog great, and clearly distinguishes it from other sites, is that you leave very little room for speculation. Please don't give that up. In order to still be able to provide your assessment for a wide range of products, maybe it would be more sensible to create a page with hints, rather than reviews, about what to be on the lookout for, regarding different manufacturers?

    Like for the one commenter above that you told - just by looking at the PCB - that the DAC/amp uses a virtual ground. Thus you could make an entry on the hints page "AUXE eBay DACs/AMPs - some products have been known to have virtual grounds which is bad, more details here (+link to your virtual grounds entry)". When details are insufficient you could write something like "CompanyX - too few specs published, especially the output impedance might be an issue due to the used op-amps", etc. Also you may want to include links to some of the more trustworthy objective testing sites, for gear that you haven’t tested yourself, but generally seems to do well (like the aforementioned Vioelectrics).

    I know this style of suggestions would give you quite some potential for abuse - but then again you are in my opinion by far the most trustworthy person regarding audio-equipment on the Internet. So I guess that’s something most of us will be willing to live with. And I still believe you would be able to pull it off without blurring the line between subjective and objective assessments. Plus there would also be a clear distinction between full reviews and simple “trained eye”-guesses as opposed to quick-and-dirty measurements or mere (subjective) listening tests in the guise of actual reviews.

    Thank you for your great work!

    ReplyDelete
    Replies
    1. Thanks for the detailed comments. I understand your point about the O2 vendors. I'm the kind of person who patronizes Bill's Backdoor Coffee instead of Starbucks because I'd rather give my money to Bill than Starbucks senior executives who clearly don't need any more money.

      Any vendor who complies with the open source license can sell the O2 and ODA. JDS and Epiphany, all things considered, are doing a respectable job. I've read several compliments about both, they're respecting the design, and neither is making snake oil claims about the O2.

      The ODAC is a bit different as, by necessity, it's only practical to have the boards machine assembled in a single large batch. It fits into the O2 so it's logical those already selling the O2 would want to sell it. The profits for the ODAC will be divided between those selling them and the single source manufacturing them. As with the O2, I don't get any money out of it.

      I don't think Samsung or Foxxcon will ever build the O2. A more realistic example might be FiiO or even one of the "eBay" manufacturers I recommend avoiding like Yulong, HA-Info, Audinst, iBasso, Matrix, Aune, etc. And the big question there is if they'll honor the license and design, or if they'll take their usual approach and modify it to be "nicer looking" but perform worse.

      An eBay/Head-Fi vendor may substitute a $0.25 no-name Chinese volume pot for the $2.50 Japanese Alps pot specified and use many other substandard passive components because it's often impossible to tell visually. No-name electrolytics could have much higher ESR, no-name film DC blocking caps could have undesirable inductance, and no-name resistors could have poor tolerance, higher noise, and a much higher voltage coefficient (creating more distortion). The only way to tell would be to carefully measure a typical finished product purchased at random.

      I've been involved with having products made in China. The first article samples met all the specs and looked great. But the first big shipment off the boat showed they cut a bunch of corners with the production run. Wired Magazine, and others, have done detailed articles about this problem. Unless you're a big company like Apple, it's difficult to get consistent high quality electronics out of China--especially as labor rates have risen dramatically there resulting in using more untrained workers and cutting costs using the cheapest possible components, etc.

      Perhaps a small "objective audio" manufacturer will turn up and exclusively offer well proven products like the O2/ODA/ODAC. But, in the meantime, I think JDS and Epiphany are doing a reasonable job.

      Thanks for the feedback on shorter reviews. It's a trade off. The few magazines/websites that conduct measurements often keep them very brief. Stereophile is one of the better ones but they sometimes leave things out (especially, it seems, when it makes the gear look better). Ideally, reviews within a category of gear should all measure the same things the same way. A significant part of the review is explaining the measured performance and what it means to different users.

      For example it's good to have 24 bit USB support if you want to use the PC's volume control. Ultra low noise is good for B.A. IEM users, but not as important for most full size cans. It's also good to compare performance to other products and explain why the FiiO E11 is superior to the AMB Mini3, etc.

      Your idea of manufacturer recommendations is something I've been struggling with while working on the Timex vs Rolex article. I can offer factual information about the specs published, etc. But to offer opinions based on my "trained eye" is going to result in lots of "NwAvGuy hasn't even tested the UberDAC so how can he know" accusations. I've tended to express my opinion more in the comments here rather than the articles which I try to keep reasonably objective.

      Delete
    2. @Anon: What you're asking for is definitely ideal and there is ABSOLUTELY nothing wrong with it, but the thing is, it's also almost impossible to achieve.

      Firstly you state that the O2 amp should be manufactured by companies that "have the means of following specs closely but no audiophile authority so as to market other snake-oil products alongside the O2", thereby implying that JDS Labs and Epiphany make snake oil products. I feel that this is unfair on your part as you didn't not provide any concrete evidence that they are doing so. Whenever, NwAvGuy points out that a product is full of snake oil, he has concrete evidence backing up his point (That's why he doesn't state that BSG Technologies Completion Stage is complete snake oil, because he doesn't know enough about it to dismiss it as so. just because something is expensive doesn't mean it's snake oil).

      Secondly, how many companies do you know of that readily provides full data of their products? If you feel that only products that have their specs readily disclosed, or have objective measurements done on them and pronounce snake-oil free, then I'm afraid we're limited to only a few products. Also, how many of us actually know what does the data mean? I'm a chemical engineer, so I do value the importance of accurate and reliable data but they'll only be useful if i can interpret them.

      Thirdly, have you seen the O2 amp produced by either JDS Labs or Epiphany? I have an O2 amp that I bought from JDS Labs and one that I made myself. Looking at the JDS Lab version, one can immediately tell that it's extremely well made (minimum amount of solder is used to hold every part in place, excess parts are cut off neatly etc.) The only gripe that i have is that they didn't provide 250mAh batteries with the amp. In addition, I asked about how NwAvGuy chose the companies to manufacture his products and he also mentioned in his O2 amp blog post why he ended up choosing JDS Labs and Epiphany. And tbh, the O2 amp is designed to be a cheap and fantastic amp, so having it be manufactured expensively kinda defeat the purpose.

      Lastly, I can tell that you belong to the Objectives clan and that is the way to go (me too) but the approach that you're preaching is just a bit too extreme and ideal to be possible. For example, I'll like to buy a portable amp the size of a JDS Labs C421. There are many alternatives (like the iBasso D-Zero, the Hippo Cricri etc) but how many of them have been reviewed objectively? Does that mean that I shouldn't get a portable amp since there are no objective reviews? (Personally I'm leaning towards the C421. Besides it getting many rave subjective reviews, NwAvGuy trusts them to produce his O2 amp, ODAC and likely the ODA amp too, and their other product, the CmoyBB is popular too.)

      Delete
    3. The "it's probably good by association" theory in the last paragraph above is a bit disturbing to me. I should be clear JDS and Epiphany chose the O2, I didn't choose them. Anyone is free to produce and sell the O2 under the terms of the Creative Commons license. Both companies predate the O2 and were selling headphone amps before the O2.

      I also have not even seen any other products sold by JDS or Epiphany. What matters most is the design of the product and the properly measured performance. I didn't design (or even assist with) their other products, and I haven't seen detailed proper measurements of their other products. So it would be faulty logic to assume anything about the performance of their other products based on the O2. Things like customer service, build quality, etc. are likely similar but don't confuse those with measurements or sound quality.

      Delete
    4. @Anon: I can’t prove or disprove whether what JDS/Epiphany are selling are or are not “snake-oil products” – and I never claimed they did such things. However I am saying that, from what I’ve learned on this blog, what they are advertising on their pages, other than the O2, is if not subjective at the very least borderline-objective - among other things £40/meter RCA cables by Epiphany Acoustics (that is about ~60 US$).

      That of course still doesn’t say anything about the products’ qualities and how they sound. But that’s not my main concern here.

      As you yourself demonstrated, someone that goes there to buy the O2 might very easily be tricked into thinking these other products have received the same amount of careful engineering as the O2 – which is backed up by lots and lots of publicly accessible and credible measurements. While that is clearly not the case. And I think that is, if nothing else, at least somewhat questionable, from an ethical point of view.

      I am someone who strongly believes in voting with his wallet, and while I do want to support O2 manufacturers (yes I do have an O2 by Epiphany, right here next to me), I clearly don’t want to support their efforts in the subjective segment of the audiophile market. Thus I feel a little torn on the inside wondering whether buying the O2 was the right decision or not. Always keep in mind that what NwAvGuy is doing here is not just providing us with an amp, but also trying to bring a change into this industry, by opening consumers’ eyes. So ideology does very much matter in this regard – for me at least.

      Delete
    5. There's another wrinkle to all this that's worth mentioning. Both JDS and Epiphany, because they sell other (perhaps more "subjective") products, are between a rock and a hard place with Headfonia giving the O2 a potentially unfair review. If JDS and Epiphany sold ONLY the O2 they would have nothing to lose by more aggressively standing up for the O2, perhaps offering to arrange a blind listening test for Headfonia, etc.

      Because JDS and Epiphany sell other products, however, it's in their best interest to stay on the "good side" of as many reviewers as possible--including those at Headfonia. JDS is an especially good example with Headfonia preferring the JDS C421 in some ways over the O2. If there really is a difference in sound quality between the two amps, that means by definition, the C421 is a lower fidelity less accurate amp. If the alleged differences are purely due to expectation bias, or otherwise invented, than Headfonia is being unfair in their comparison. Either way JDS stands to lose if they defend the O2 with Headfonia.

      Delete
    6. @Anon of Apr 14, 2012 07:52 AM:

      Regarding your statement of "If you feel that only products that have their specs readily disclosed, or have objective measurements done on them and pronounce snake-oil free", this is what I do right now, after reading through Nwavguy's blog. I am now deeming, by default, that all audio products without objective data to be snake oil. This inevitably causes some blue-on-blue fire but at least it guarantees that all equipment that I procure are objectively excellent.

      Now to measure the output impedance of my DAP.

      Delete
  136. On the BSG Signal Completion Stage...I went and checked out the website and read the SoundStage review (linked from the website). The reviewer flipped over the "enhanced detail". To the reviewer's credit, he then measured and found that there was a 3-4db volume boost with the unit engaged. After getting rid of the volume boost, he found music through the BSG "more naturally lifelike" than with it bypassed. Excuse me if I'm just a TINY bit skeptical...

    Andy B

    ReplyDelete
    Replies
    1. Unlike say expensive cables there's little doubt the BSG box changes the sound in audible ways that go beyond the cheap gain boost trick employed by lots of products. The question is if can it accurately recreate lost information (I don't see how) and if yet another crossfeed variation is worth nearly $4000.

      Spatial processing of any kind--including the crossfeed built into a $150 headphone amp, Carver's $300 Sonic Holography, Dolby's various 2 channel surround processing bundled with soundcards, etc. has no way to more accurately locate a performer in 3 dimensional space. It might increase the apparent space between the performers, but the effects are more arbitrary than accurate. An extreme example is selecting "Stadium Surround" on an A/V receiver while listening to some intimately mixed unplugged acoustic recording. The effect is mostly a novelty and has nothing to do with the venue used to record the music.

      Delete
  137. Hi NwAvGuy, will like your advice on something:

    I was searching Google for any potential smartphone (besides the iPhone) that I can use as my DAP and music server, and I came across this thread on head-fi. According to someone there, he said that there is no point looking for a smartphone that can play lossless files because none of the DAC in smartphones are resolving enough such that you're able to hear an audible difference between 256kbps files and lossless files. Granted, that thread was 2 years old, so I'll like to ask if things are different now. I remember that you have on several occasions, recommend that people not buy digital transports because the DAC unit in iPhones and iPods are more than good enough for portable use.

    ReplyDelete
    Replies
    1. That's sort of loaded question. First of all, most people cannot hear the difference, even with a very high-end DAC, between a properly encoded lossy 256Kbps track and a lossless track. You can easily verify that yourself with the free ABX software discussed in this article. So that argument in largely moot.

      I haven't fully tested an iPhone like I have my iPad Touch 2G and 3G. But I can say, based on other measurements, the third generation iPhone seems very similar to the Touch 3G. And the same seems to be true of the 4G products. If those somewhat generous assumptions are true, the DAC in third and fourth generation iPhones should be audibly transparent (or very close).

      Obviously, if true, such knowledge hurts the sales of all those Head-Fi sponsors hawking iPhone DAC-In-A-Box products. And it also hurts the sales of Head-Fi sponsor favorites like HiFiMan's clunky half-baked DAPs. So I would expect many on Head-Fi to claim the DAC in iPhones is no good. Most likely either spent money on other hardware, and/or otherwise have an interest in supporting Head-Fi sponsors (last time I checked, Apple's not lining Jude's pockets).

      Using an external DAC with any portable device is much more risky than simply getting a better portable device. Phones and DAPs internally use the far superior I2S bus to hand off the digital audio to the DAC chip. But with an external DAC you have to embed the clock into the audio data. And, with the extremely low power chips used in phones, the odds are much higher there will be more jitter in those external signals--especially after it's sent through a dock connector and a thin cable with the jitter-prone digital signals a fraction of a millimeter from a bunch of other signals.

      I've never seen any convincing measurements that show superior performance for external DACs with a portable device as the transport. I have, however, shown the DAC in the Touch 3G is quite respectable. And, rumor has it, the 24 bit DAC in the Touch 5G (due out soon) will be even better. I'd much rather spend $200 on one of those than some snake oil magic box from a Head-Fi sponsor.

      It's worth mentioning the DAC, and especially headphone amp, quality in a lot of Android smart phones ranges from quite good to audibly flawed. And, while my pool of examples is limited, there seems to be little consistency by brand. Samsung, LG, and HTC, for example, have made phones with respectable audio and others that sound (and measure) notably worse than the $29 Sansa Clip+. So YMMV once you look beyond the 3rd and 4th gen iPhones.

      Delete
    2. The whole problem with Android is the forced SRC problem. You play a 48 kHz file in Android, it gets forcibly resampled to 44.1 while passing through the Android Audio Finger layer. The result is much disturbance in higher frequencies. This is hardware-independent. Both Qualcomm and Tegra devices show this problem.

      http://anythingbutipod.com/forum/showthread.php?t=68549

      Delete
    3. If the DAC in the Touch 5G does turns out to be a 24bit DAC and it does measure well, then it is likely (I hope!) that Apple will make the same implementation in their iPhone 5. If that's the case, then I feel that my waiting for the iPhone 5 instead of getting a iPhone 4S will be pretty worth it.

      It is ironic that for years, I have the impression (due to Head-Fi) that Apple DAPs were one of the worst DAPs in the market, and that Apple produces products that look good, but perform badly. Now it turns out that the iPhone might actually be the most suitable DAP for me.

      Delete
    4. Output impedance might still be an issue even if the built-in DAC (Cirrus Logic) is transparent. The 3rd generation Touch has about seven ohms of output impedance, measured by (if I remember correctly) Nwavguy right here.

      Of course, that's the third gen Touch. I have no idea about the new iPhones.

      Delete
    5. Yes, I was talking about the DAC, not the headphone amp, in iPods/iPhones. You may still need an amp to get optimal performance with many headphones. That was part of my own incentive to design the O2. It would be great if the 5G iTouch/iPhone finally have a near zero ohm output.

      Delete
  138. Hi NwAvGuy,

    probably you are right in saying that current external mobile DAC implementations has no significant improvement over the iGadget's own DAC.

    However I must say that your technical comment in paragraph 4 is not quite valid. External DAC -s can be any precise and they do not have to rely on transport interface (USB, SPDIF, I2S) actual embedded clock. BTW this is how Benchmark DAC1, Youlong D100 and Violectric V800 works.

    World learns slow. However I vision Smart Gadgets will sooner or later replace the bulk of PC applications and if it happens Smart Gadget will be avg Joe’s media center. Therefore interfacing to it will be a key issue, should it be ageing USB or coming MHL or whatever future will bring.

    ReplyDelete
    Replies
    1. You bring up a good point. It's possible to regenerate the clock, but most of the devices designed to interface to the iPhone/iPod digital output (which is NOT S/PDIF compatible) use the supplied clock so that doesn't apply. That's especially likely to be true of portable iPhone/iPad DACs for power consumption reasons.

      On my list of things to do fairly soon is to run a blind comparison of the iPod/O2 amp with both running from battery against a PC/DAC1 playing the same file. I suspect both are sufficiently transparent so no difference will be heard but the truth will be in the listening. For those who think the iPod's DAC is "not good enough" they really need to audition it blind before they jump to any conclusions.

      Delete
    2. I would love to see a comparison of the iPod/O2 with a PC/Benchmark DAC1.

      I run my IPhone 4s into a decent integrated amp and speakers via a line out dock and I'm impressed with the sound. It has great clarity and stereo imaging. There are some audio measurements here: http://www.gsmarena.com/apple_iphone_4s-review-665p5.php#aq
      It definitely beats my apogee one running from a macbook pro (I know this is not a high end source but it gets decent reviews).

      However, surfing the various audiophile forums had got me wondering "Am I missing out on something?" So I was very tempted to go out and spend some money on a stand alone dac. This is where I think a lot of the audiophile madness begins.

      Thanks for keeping me in the real world!

      Delete
  139. The thing I find most intriguing about ABX testing in general is the degree to which differences seem to be minimized.

    My interest is in professional audio ie. microphones, mic preamps, interfaces with AD-DA converters etc. The online examples of microphones for blind testing, for instance, compare mics $200-400 with mics costing over ten times as much often to great advantage. There are people out there that seem to have an easier time in the mic ABX tests than me, for sure. What I CAN say is that the differences are often surprisingly small and not always in the prestigious expensive mic's favor. Of course an expensive brand name mic is almost always very well built and usually quite dependable. But it will be a long time if ever, before I buy another $2000 mic. What this discussion has done is make me much more comfortable shopping for a mic in the 300-400 dollar range and realizing that if I use it properly, in a decent mix it can sound great, if not indistinguishable from the one costing 3 grand. (which I don't have to spend on a mic anyway)
    This blog has been a terrific help and a real eye opener and I thank you. Wish I had seen it 2 years ago. It would have saved me a lot of money.

    Andy B

    ReplyDelete
  140. Another great article.

    I bought an Audio GD NFB-12 DAC/Headphone amp unit to use with my computer; one reason was that I needed something that had a line out as well as headphone out that would enable me to easily switch between speakers and headphones. The other reason was that the output impedance was 2 ohms and after reading your other articles I thought that was a good thing so I am wondering what is not so good about Audio GD kit?
    To be honest I was disappointed to find that I could not really hear much difference between it and the output from my PC's onboard soundcard.
    The speakers are Acoustic Energy Aego M and the headphones are AKG Studio 240 so both are not "audiophile" but neither are they cheap.
    I couldn't really find anything else at a reasonable price that met my requirements except for the recently released Arcam rPAC and I couldn't find the output impedance of that anywhere.
    Hopefully your desktop implementation of the O2 will have line out?

    ReplyDelete
    Replies
    1. The discrete op-amps used by Audio-GD have been measured to perform very poorly. That combined with their reluctance to use negative feedback properly results in a lot more distortion. Some of their products also, from what I understand, have poor frequency response.

      From what I can tell, Audio-GD doesn't design to any consistent engineering goals. And it's not possible for one person to design as many products as they have, in a relatively short period of time, and really do a good job optimizing them. It would seem Audio-GD is more interested in offering Flavor Of The Month (FOTM) products that cater to the latest popular audio myths, DAC chips, etc.

      That's not to say all Audio-GD products perform audibly poorly. But their product line, as a whole, is likely to perform much worse than even FiiO's much less expensive products.

      I can't say for certain, but it's likely the $75 FiiO E10 outperforms the NFB-12. I'll have more in my Timex vs Rolex article.

      Delete
  141. Hi NwAv Guy,

    I would wait for the ODA, especially if it will include all the features listed in your ODA article, as the ones most people agree they would like to have included in a desktop Amp.

    Regarding your last comment, I believe that there are several elements to assume that Burson's products are flawed and more snake oil than accurate engineering, but I also pointed out it would only be fair to try one or two of their components out, in order to get facts right, and be objective about things.

    It would otherwise be possible to conclude that your position is simply biased against certain brands because of what others have said without necessarily contributing with facts or proof.

    I woud certainly support your ODA and ODAC, but I would also do some blind testing with some other components in order to be as fair and objective as possible, and that following your own recommendations and advise. If you accept challenges to your designs, which I believe a very valuable position, I would also consider other products to have the same chances, without having a lot of negative and biased comments hanging over them.

    Cheers!

    ReplyDelete
    Replies
    1. NwAvGuy,

      Instead of the ODA having essentially the same spec as the O2 but with more inputs and outputs, why don't you modify it such that it will still essentially have the same transparent character, but with more power so you can drive ALL headphones, even the tough orthodynamic ones (like the HiFiMan HE-500).

      Delete
    2. The O2 can drive the HE-500 to very respectable levels. According to InnerFidelity they need 0.31 Vrms to hit 90 dB. So that's 3.1 Vrms to hit 110 dB SPL. Even adding a generous extra 5 dB, they still only need 5.4 Vrms which the O2 can barely manage on fully charged batteries and do with ease on AC power.

      A lot of amps, even desktop amps, fall down with planar (orthodynamic) cans due to how much current they need. At 5.4 Vrms the HE-500's need 163 mA of peak current per channel. The O2 can deliver 200 mA per channel--even on battery power.

      A lot of portable amps don't have enough output voltage and/or current for the HE-500. The FiiO E5, E6, E7, E10, and E11 all fail as does the AMB Mini3, Bottlehead Crack, all Cmoy/Mint amps, and most battery powered amps including those from HeadRoom, HeadAmp, Leckerton, and others. Likewise most amps with significant output impedance will also fall short.

      Delete
    3. That's closer to the limit. It would really depend on how loud you want to listen at.

      Delete
  142. Hi there nwavguy, what are your thoughts on wadia as a company? They have a new dac, the wadia 121 with headphone amp ( possibly designed by ray Samuels). Looks like it may be the next fotm

    ReplyDelete
    Replies
    1. FOTM is probably a good description. Wadia has at least been around a long time and some of their gear has measured reasonably well. But they also lean towards overpriced snake oil rather than solid engineering. For what it's worth the Matrix Blind Audio Test compared a very expensive Wadia CD transport/DAC to a cheapo Sony CD player and nobody could hear any difference.

      The Wadia 121 is $1300 and there are plenty of well proven products, many with more features, in that price range. I would suggest the $800 Centrance DACmini likely outperforms the Wadia and has more features for a lot less money. Headphone amp design, as the O2 has demonstrated, isn't rocket science. And the ODAC is about to demonstrate the same is true of DACs.

      Ray Samuels is a self proclaimed "design by ear" designer. This article explains, in detail, what's wrong with that approach. And other products, such as those from NuForce, Audio-GD, and Schitt have demonstrated what can go wrong when you mostly design by ear. I've not seen any detailed specs or credible measurements on Ray Samuels gear.

      It's my understanding at least some of his products have some serious design flaws. I've been told, for example, he's using Li-Ion batteries without a proper smart charge controller. That's a serious fire and safety hazard (just look up "lipo fire" on YouTube). I've also been told by someone who made some measurements the $150 (assembled) O2 outperforms even the $650 SR-71. It's my understanding he's a nice enough guy but his amplifier designs seem mostly random as if he's designing art pieces instead of audio products.

      Delete
    2. HiFiMan claims that their HM-801, with a rolled-off treble, is modeled after Wadia's 508 player. Their argument: Wadia is an insanely expensive piece of gear and our player is made to sound exactly the same as that.

      Excuse me, but I can only laugh.

      Delete
  143. Hi NwAvGuy - long time listener, first time caller. I love your approach to audio electronics - I'm an electrical engineer myself - so it's great to see actual measurements and science, rather than speculation. Looking forward to the ODAC release, I've been looking for something exactly like that for a while now.

    Anyway, I'm wondering if you have any thoughts on earbud-style headphones. I've got a pair of Sennheiser CX300s that I've used for a couple years now; before that I had a pair of Sony EX-71LPs (I think that's the model #). I recently started traveling on a much more regular basis, and so my earbuds are getting a lot more use. I'm thinking about an upgrade, but I'm actually not sure what benefits there are to be had. I've done some research, and it looks like the CX300s have great distortion performance, and the frequency response certainly isn't BAD for earbuds. A lot of people seem to think the Etymotic IEMs are great, and they certainly have great isolation, but they also appear to be a much higher distortion design (according to the HeadRoom measurements, at least).

    Do you have any earbud recommendations? I'd be tending to use them without a separate amp.

    Thanks!

    ReplyDelete
    Replies
    1. To me ears, the CX300 isn't that impressive. I'm sufficiently unimpressed with my CX300s I use them as a test load even when they might be destroyed by a poorly designed headphone output. In other words, if they're destroyed by a poorly behaving amp, I'm fine with that.

      To my ears, the Sony EX-71 is a much more engaging earbud compared to the CX300. The same is true for other IEMs like my Ultimate Ears SuperFi 5s. Sennheiser seems to do much better with full size cans.

      I've not seen any credible test results that demonstrate Etymotic IEMs have excessive distortion. They might lack engaging bass, but everything else I know about Etymotic indicates they're relatively accurate and clean.

      Delete
    2. HeadRoom graphs show huge amounts of distortion for some of the models, like for a 500 Hz THD test, 2nd harmonic at -30 dB, 3rd harmonic at -55 dB.

      However, checking InnerFidelity and at least one other site:
      http://www.innerfidelity.com/headphone-data-sheet-downloads
      http://www.geocities.jp/ryumatsuba/review.html

      it doesn't seem out of the ordinary. Seems like something's wonky with HeadRoom graphs again.

      Delete
    3. Ah, thanks mikeaj - I hadn't seen those sites before. It's tough to figure out what earbuds I'm going to like if any objective data I find is skewed, because it's rather difficult to audition earbuds.

      Also, NwAvGuy - I'll do some more investigation into the Ultimate Ears line, it sounds like you're happy with them. I'd be really interested to see a blog post comparing some headphones...but I also know that's not really what you're set up for.

      Anyway, thanks for the insight!

      Delete
    4. Honestly, Head-fi is really the best place for that kind of thing. It's great for the headphones themselves, but not very good for all the supporting equipment like amps or DACs.

      If you just watch out for the output impedance of whatever DAP/amp they're using and ignore pretty much everything else they say about "synergy" you can usually tease out plenty of useful information about headphones from even the craziest cable believing, "sonic brick" using audiophile.

      Delete
    5. Thanks all for your help. I ended up placing an order for both the Etymotic HF-2 and the UE SuperFi-5vi from Amazon. I'll let my own ears do the deciding; Amazon's return policy seems to be pretty liberal as long as I don't damage anything.
      There really ought to be some way to audition headphones without going through the hassle of buying and returning. I got lucky with my main headphones - I had a friend who owned both the Grado RS1 and Sennheiser HD650, and let me borrow them for a bit. In that situation, it was pretty easy to decide that I liked the 650s a lot better!

      Delete
    6. Not so sure about that, maverickronin. Head-fi's subjective reviews had me purchase iems with serious problems. One of their main reviewers is a quiet listener and would appear to be unaffected by significant frequency humps. Hello PL-30 and H2O audio flex; both shouty iems with 8db spikes in the low mids, yet great reviews. The former is fixable in parametric EQ (using sine wave testing) but the latter even EQ-ed is poor sounding yet has an enthusiastic review.

      In time I tried 2 of one and 3 of the latter headphone with identical results. Even my brother, who knows nothing of Hi-Fi, came back to me claiming the H2O was poor and the ipod buds far better; he was right. Both brother and GF now using cheap Denons instead.

      I no longer trust head-fi even for headphone reviews. I'm now a measurements devotee.

      Delete
    7. That would be awesome if there were more measurements of headphones. I love looking at the measurements for all the headphones I can and it's very helpful but it's not always enough.

      No single site has all the numbers I'd like to see, different measurement rigs aren't directly comparable to each other, and only the most common headphones are usually measured.

      The measurement situation is even worse for amps and DACs but I'm not nearly interested in trying new amps or DACs. Unless I need more power than the O2 can deliver in the future I don't really see a need for an upgrade. That makes the situation is pretty moot.

      Just like you interpret graphs for a new headphone in light of other headphones measured with the same system you have to interpret reviewers in the same way by comparing your opinion and their opinion of a common model.

      Besides those practical issues, the sound you like is subjective. It's possible to make an amp of DAC that's perfect to the limits of human perception but no transducer of that level currently exists and likely never will.

      Delete
    8. Headphones will always be objectively flawed--i.e. none are audibly transparent and each has its own unique sound. So, from that perspective, subjective comments are valid. But it's also true headphone measurements can tell you a lot--especially the more complete ones like those done by InnerFidelity.

      The commercial and sponsor-oriented bias at Head-Fi also extends to headphones. And headphones are not immune from expectation bias. Many people will largely hear what they expect (i.e. read online) to hear. That creates a self propagating subjective perception that may be far from accurate--especially for less popular gear that's not widely reviewed on many different sites.

      So I think it's fair to say Head-Fi is more useful for headphone reviews than gear reviews--as subjective opinions can be useful. But you still need to be aware of all the bias behind those opinions.

      Delete
    9. In case anyone is curious (or even still following these comments, with the ODAC announced), I ended up going with the Etymotic HF-5. While I actually may have preferred the sound of the UEs a little bit, I was MUCH happier with the fit of the Etymotics, both the earpieces and the cable. I didn't do any careful level matching, so it's entirely possible that what interpreted as slightly better sound from the UEs was just due to the fact that the UEs are much more sensitive, and as such, louder. Anyway I'm really happy with my choice - thanks all for your help!

      Delete
  144. Hello NwAvGuy. I have three related questions that arise from the subjective review discussed in your post:

    1. What does it mean to say that a headphone amp has or lacks “authority”? Is authority anything more than a bass boost or coloration? Similarly, what is “scale”? Is scale image size or another form of coloration? The reviews use the term “authority” with authority, but I’m not sure what they are talking about.

    2. Is there anything about a small amplifier that would result in lack of bass or scale? I assume that with a proper design, a portable amplifier would have flat response down to very low frequencies. I don’t see anything about the size of a component that would impact scale, whatever that is. The reviews often distinguish full-size from portable amps.

    3. Similarly, what characteristics of an amp would result in loss of detail? Noise is an obvious cause. But when people are comparing levels of detail are they talking about anything other than high frequency roll-off or boost?

    ReplyDelete
    Replies
    1. See the paragraph in this article titled: Condescending Lingo. "Authority" and "scale" are audio reviewer buzzwords and are nearly useless. Audio reviewers use vague descriptions as it's intimidating to some (making it seem like the reviewer knows more than they do) and it's harder to hold the reviewer accountable as they can claim words like "scale" mean different things if anyone challenges their review.

      My best advice is to ignore any review that uses such vague descriptions. The English language has plenty of unambiguous words to describe what someone hears. Reviewers who choose not to use more obvious words are not doing anyone any favors.

      The size of a headphone amp has nothing to do with its sound quality. For amps that drive speakers there's some correlation between maximum power output and size but that still only determines how loud they will play. Most portable battery powered headphone amps have less output than the best desktop amps, but the O2 is an exception to that rule. But as long as an amp has enough output to play loud enough, the physical size doesn't matter except to create subjective expectation bias. Some might expect a bigger amp to sound "bigger" so that's what they hear even if it's not true.

      The word "detail", again, can mean different things to different people. I've encountered gear with massive amounts of high frequency distortion that was mistaken by some as "added detail" because of expectation bias. Generally it's a term that applies more to headphones and speakers rather than audio electronics. But if a DAC or amp is so bad as to have audible roll off in the highs, it will subjectively have less detail.

      Delete
    2. I am no subjectivist and may be approaching this too logically, but I think "lacking authority" may be related to output impedance, since a high output impedance would result in reduced physical "control" or "authority" of the amp over the headphone's coil/driver.

      Delete
  145. Do you (NwAvGuy) know of a web site that gives more of a "for dummies" explanation of the different specs for headphones and headphone amps? I enjoy reading your views of audiotopia, and I appreciate the detail that you go into for your articles, but a good deal of the information you cover is over my head. Perhaps, if there isn't anything like this already, you could create a single FAQ-like page which contained brief, non-technical explanations of different headphone and headphone amp specs.

    For example, you often talk about clipping. What exactly does clipping sound like? As it stands now, I have no idea if clipped music would sound like TV static, tearing paper, a skip in playback, or something else. I know it's bad, but I wouldn't know it if I heard it. Or how about output impedance? Is it similar to putting a "brick wall" in front of the audio signal so you need more power to break through the wall? Sadly wikipedia didn't help me out here and skimming various audio forums makes me feel that most people have no idea what any of the terms mean.

    Since music is something anyone and everyone can enjoy, maybe you could educate more people who aren't as concerned with nitty gritty details if there was some resource that explained some of the things you're demonstrating with some easily approachable real world examples. Hopefully that makes sense.

    ReplyDelete
    Replies
    1. Thanks for the feedback and suggestions. I'll work on that. I'm going to try to have some sound files available for download and I hope to have examples of some of the more common "flaws" which should help. Several have suggested I do some sort of "glossary" of audio terms and I agree that's also a good idea.

      Delete
  146. Hi Nwavguy, it would be nice to know your opinion on the recent influx of ESS Sabre DACs, in particular of 2 brands - wyred 4 sound and anedio. Would you get any noticeable improvement in sound when compared to the Dacmini or Benchmark dac? When considering pricing they are similar to a benchmark but is the newer technology Sabre a better DAC chip than the AKM or AD chips?

    Thanks!

    ReplyDelete
    Replies
    1. The big joke among genuine audio engineers is everyone (including the marketing department) wants to know "what chip is in it?". But the DAC chip is rarely the weakest link in a complete DAC. The rest of the design is often more flawed than the DAC chip itself.

      You can build an audibly transparent DAC around chips from ESS, Wolfson, TI/Burr Brown, AD, or AKM. Which chip you use comes down to the application, features you need, if there's a microcontroller, power consumption, power supplies available, type of output you want, etc. The only non-transparent chips are the low-end really cheap ones, some of the highly integrated all-in-one chips/CODECs, and some of the old i.e. NOS) chips.

      The problem is a lot of the smaller manufacturers just slap a well regarded DAC chip on a board, add some trendy audiophile blessed parts, and call it good. They don't even own a decent audio analyzer to verify the performance. Those DACs very likely perform much worse than the chip itself is capable of. The devil is nearly always in the details, not which chip you choose.

      This is especially an issue with Asian-designed audiophile DACs. You look on eBay and the chip part number is featured right in the one line description. But, notably missing, are real measurements that verify the performance of the entire DAC. They often just quote the specs right off the DAC chip datasheet which is extremely optimistic and misleading. The eBay DACs I've measured don't even come close to the datasheet performance, and so far, every single one has had at least one serious problem. I'll be showing some measurements for just such a DAC soon.

      Delete
  147. Nwavguy,
    Thanks for article, I really enjoyed it. I love the way you peel the veneer off of some of "outlets" and lay them bare for all to judge for themselves. Obviously a lot of "behind-the-scenes" machinations occur that most people don't have a clue about. At the very least it gets one thinking.

    I still value head-fi as a buy/sell resource, and there are some people that have a genuine desire to help people by recommending gear they think "sounds good" to others (for better or worse). OTOH it's really hard to navigate the site without being clobbered over the head with what I like to call a "blind" Schiit recommendation (it gets really old after a while).

    My posture is that I have no problem if one wants to spend money on gear they think looks good in their home, tickles their inner geek, or sounds good to their ears. However when people, or manufactures try to ascribe superior sound quality to (or sell you on the basis of) a particular discrete circuit, or paper in oil silver film cap, or other [insert exotic upgrade], that has no measurable, supporting data, it just wreaks of everything that's wrong with hobby; that there are people trying to make a buck off of you by preying on your ignorance.

    I also find it astounding how many people out there on the inter-webs exercise carte blanche in attempting to assassinate your character at every opportunity because (I speculate) that they feel you threaten the "validity" of their hobby in some way.

    Know that you have supporters right here on this site, and keep up the good work!

    Regards.

    ReplyDelete
  148. Surely these distortion graphs for the earphones show what a blunt edged instrument THD is. The THD of the electronics is orders of magnitude below that of the transducers and yet we still think that we can hear a difference between electronics with 0.001% distortion. We worry about 1 dB variation in frequency response in the electronics but we tolerate 10 dB ripple in the high frequency response of the earphones. (You will probably tell me it is the eigentone of the ear canal!)
    Well some people can. I am getting older and my hearing is limited to 12kHz. It seems like my brain has upped the gain to compensate so resonant peaks at 7kHz and 11kHz tend to get emphasised and unpleasant (CX300s). Maybe that's why older people cannot stand high frequencies.
    You can listen to CX300s if you use some equalisation and sit in an aeroplane. The also work well with the sound turned off in this context.

    I also think that what we hear depends on context. My wife cannot stand my fairly loud and clean music at home ("turn it down") but will enjoy a loud (the deaf guy on the mixing desk said 105 dB max) and distorted (to the point that I could not hear the vocals without earplugs) Jimi Hendrix soundalike concert.

    Finally - Epiphany is advertising a thing called E-DAC. Is that anything to do with you?

    ReplyDelete
    Replies
    1. NOTE: This commented has been edited to incorporate feedback from others. Because the comments here are close to the maximum limit of 200, I've tried to consolidate this sub-thread.

      Steve, your distortion beliefs are common. But there are some important distinctions (ones I make in Music vs Sine Waves and some of my other articles).

      First of all, there are headphones with less distortion than many amplifiers and DACs. The HD650 is one example that's been tested by several independent websites for distortion. It has distortion at least 10 times less than some amps, DACs, etc.

      Second, you need to note at what level distortion measurements of speakers and headphones are made. They're often done at fairly loud levels and distortion is often much lower at lower levels. And, as someone pointed out, some measurements are THD+N where "N" is noise and that can easily dominate acoustic measurements with microphones. You also often don't know what the driving equipment was and how much it may have contributed to the distortion measurment, and the same is true for the microphone, mic preamp, etc. Ideally, you should only compare headphone distortion between models on a single website where they were hopefully all measured the same way.

      Third, I'm not saying you can hear a difference between 0.001% and 0.002%. But, under some conditions, anything over 0.01% might be audible. The Myths video linked and explained in this article has a test where "crud" is mixed in with the music at various levels. As reported in the article, the crud is still audible even at -70 dB which is 0.03%. You can download the file and try it for yourself. Even though you're listening with "high distortion transducers" that doesn't stop you from hearing the crud--if not at -70 dB than at -65 dB (0.05%). I plan to offer some test files of my own.

      Fourth, distortion is additive and the human brain can hear things that are lower than the ambient level. Tests have been run exploring the audible thresholds of distortion in electronics. They were done using speakers and headphones. If those speakers and headphones really mask any distortion below say 1%, why can people still hear electronic distortion at 0.05%?

      Finally, it matters what kind of distortion it is. Some distortion (like the 2nd harmonic) is relatively "harmonious" and more easily masked by the music. But other kinds of distortion are much less harmonious, or not harmonious at all, and are much more likely to be heard. The Myths video linked in this article used dissonant crud. And even using my DT770 headphones (which supposedly have much higher distortion than the HD650), it's audible down to around 0.03%.

      People can usually pick out a 1 dB frequency response variation in a blind test--even using headphones with 10 dB of variation. It's also difficult to accurately measure the frequency response of headphones. Those huge swings above 2 Khz are typically more a result of the measurement technique (standing waves at the microphone) rather than the headphone. The same is true of speakers when swept sine waves are used. Our ears and brain seem to largely correct for this acoustic phenomenon compared to a microphone.

      Delete
  149. iPhone 4 output impedance is 0.97. iPhone 4S output impedance is 1.9

    I have the latter and it's headphone out is gorgeous paired with my Westone UM3x.

    You can check impedances at goldenears.net or sonove blog.

    In my view iPhones are the best performind daps around. For iems that is.

    ReplyDelete
  150. Hi, I already own the O2 amplifier and I enjoy it very much. I use it with my iPod Touch 3G via a LOD cable. I'm like you Nwavguy and like the piece of mind knowing that my source is completely audibly transparent. Is the iPod Touch 3G's DAC via line out transparent enough or should I either get the ODAC or HRT Music Streamer II?

    Thanks.

    ReplyDelete
    Replies
    1. I would just stay with the Touch 3G + O2. You can't use the ODAC with your Touch but could use it with a PC. But I'm not sure you would notice enough of a difference over the Touch 3G which already does a very respectable job (see my Sansa Clip+ review for some Touch 3G measurements if you haven't already).

      Delete
  151. NwAvGuy:

    Why don't you use the TE8802 chip which supports asynchronous USB up to 24/192? TE7022 doesn't support 24/88.2.

    ReplyDelete
    Replies
    1. The TE8802 is an interesting chip. But when we started this project it was new and unproven. We also wanted a driverless solution for Windows. The TE8802 requires a proprietary windows driver for asynch USB.

      24/88 can be resampled to 24/44 with NO loss of audible fidelity (this has been shown in multiple blind tests). And 24/192 offers no audible benefit at all. Neither does async USB. If the ODAC is already audibly transparent why does it need to be "improved"?

      The ODAC is about doing what's required for audible transparency where the DAC can disappear from the signal chain. Once a DAC is "invisible" why spend more money, or require proprietary drivers, for no audible benefit?

      Delete
  152. @Grunty: Your contention with my claim that "nothing objectively sounds better than something else" is a perfect demonstration of measurement bias. I agree that higher quality = better only because "higher quality" is a perceived quality, as is sound, and is thus completely subjective.

    You seem to believe that sound exists independent of a listener. This is false. Measurements do not need a listener, but sound does. Sound is then dependent on our biases.

    So it isn't an "illusion" that tube amps sound warmer. The sight of glowing filaments actually makes them sound warm (to some people).

    ReplyDelete
    Replies
    1. It's worth pointing out that properly run blind listening tests can indeed bring some genuine objectivity to an otherwise subjective experience. For anyone who doesn't believe that I'm happy to provide some fairly credible references.

      Delete
    2. You can remove biases, but sound is ultimately subjective. I know many people who enjoy the sound of scooped mids (e.g. Bose speakers), whether they are blindfolded or not. Who are we to say they are incorrect?

      Your ODAC design might measure worse than a DAC1, but I don't think you would say it objectively sounds worse than a DAC1. I am sure many would prefer the sound of the DAC1 just knowing that it measures better, regardless of the fact that when blindfolded, it sounds no better to them. This is no more valid than someone preferring tubes amps.

      Delete
    3. But that doesn't change the fact you can objectively demonstrate the ODAC and O2 (as just one example) sound the same when you remove the usual subjective bias. Are you suggesting someone should spend roughly 10+ times more for the Benchmark merely based on the erroneous perception it delivers better sound? Or is everyone better off knowing that perception is merely listening bias and they really sound the same if you don't know which is which?

      Delete
    4. Well, you are "objectively" demonstrating that they subjectively sound the same while blindfolded.

      I agree with you that someone shouldn't spend more on equipment measures better (like benchmark) if it cannot be distinguished in blind listening tests. There are better things one could spend their money on.

      Delete
    5. First of all, I built an O2 several months ago; it worked the first time I fired it up, thanks to the many suggestions on building it (I've built other amps before this though), and it sounds great on all of my headphones. This is my first time to post something here, so I want to thank NwAvGuy for all of the work into making this amp as well as the testing and measurements to back up the performance. Job extremely well done! I eagerly await the ODAC.

      Based on many comments about it on this blog, I bought Goldstein's book The Wine Trials, as well as his similar book on beer and felt I needed to comment on it as it relates to blind testing. I bought it because I thought it would contain some interesting recommendations on inexpensive wine, which I am very interested in, but also to understand how he can claim that 2buckChuck, which I buy frequently and find it excellent for a cheap wine, and many of the other highly touted cheap wines could possibly stand up to wines that I know are great (Dom Perignon for one). I've been blind tasting wines for decades, and have kept a journal with the results since 1982. I've done some of the tricks like putting the same wine in twice or having a "ringer" which was unexpected. I enjoyed a blind cab tasting where the person that said she loved cabs but hated Zins, picked the one Zin I put in as her favorite. The claims made about the cheap wines being better than some of the best wines made are just ridiculous. I tried several of the wines in his book and was familiar with some others, and so far I find they are very good for the price, although in some cases I prefer some other wines not mentioned that are even cheaper. But they all have one thing in common: they are all well made but simple wines without much complexity, made in a style that is accessible and ready to drink when released. The expensive wines that were included in the comparisons he talked about are often wines that are meant to have several years of bottle age before consuming (to shed tannins which most people don't like, as well as develop additional complexity), and has a lot of intensity and complexities that complement food better than drinking alone without food. Almost all of the wine I consume is with food, but blind tastings (not mine) are usually done with little if any food. To my mind, touting the 2 Buck Chuck Myth does disservice to your blind tasting argument for audio. Fortunately Goldstein's The Beer Trials is a far more useful book, as it compares beers that have similar styles. If he had done what he did in the wine book, I'm sure that Bud and Coors (which are given average ratings) would have rated far better than most of the craft beers that he often rated very highly which often are full of complex and intense flavors that most people in a blind tasting would find overwhelming. So he compared lighter styled beers with similar styled beers, unlike his wine book where tannic and very intense wines were compared with easy to drink light wines. Lighter styled wines (and beer) are preferred by most people; just look at the sales figures. The Domaine Ste. Michelle sparking wine is okay for the price but not one of my favorite sparkling wines even in that price category because it is just too simple, but the fact that 41 of 62 tasters preferred it over the Dom Perignon is not all that surprising. But the information is worthless to me when I want to choose a sparkling wine, as I'm sure I wouldn't have been in the group of 41. I'm sure a blind test of a bunch of people would show that for most people low bit rate MP3 file sound as good as a lossless file of the same music, but that doesn't matter to me as long as I can tell the difference in a blind test. The fact is that a lot of people eat crap fast food that I wouldn't eat, drink crap beer and wine that I wouldn't drink, and listen to crappy sounding audio that I wouldn't listen too, and someone could prove in a blind test that they are right and I'm wrong.

      Delete
    6. This is getting way off-topic here, so I can understand if the comment doesn't get approved or deleted later on.

      I think KurtW might be spot on with his assessment that most people prefer simple over complex. I have to admit I have little experience with wines. But what I have noticed, over the relatively few years of life that I have had so far, is that in every area where there is competition–be it in art, music, sports, engineering, etc.–there is not only a point of diminishing returns but in fact a point of declining returns.

      To provide some examples of what I mean - I enjoy a nice “simple” painting as much as the next guy, but to be perfectly blunt, I wouldn’t be able to tell the difference from “complex” Picassos and my own doodles, from back when I was seven, if nobody told me. It is the same with wrist-watches. All those expensive mechanic Rolexes are beaten by even the cheapest digital (quartz-driven) Casio ones in terms of accuracy – ironic isn’t it? I also do prefer recorded (and professionally mastered) music quite a bit over live performances. And I could go on and on about this.

      The point I’m trying to make here is that at some point you cross the line of just assessing the final products quality (the wine, watch, etc.) and start assessing how large the challenges of the manufacturing process were and how well they have been mastered. Of course it is much more complex to create a mechanic watch that doesn’t drift too much, compared to a digital one – but given the prime function of a wristwatch (which is keeping time) why should I care about that? Of course Picasso had quite a bit of a struggle to put his thoughts onto the canvas as accurately as he did, but given that I enjoy a paintings mostly because of the beauty the convey, why should I value a piece of abstract art? And so on..

      I understand that you may want to call me a Neanderthal incapable of comprehending the finer aspects of life now. But let’s be honest for a minute. What we like and dislike is for the most part not so much a deliberate choice but the result of millions of years of evolution. Of course there are some differences from person to person, but by the end of the day we are all still humans. Now considering that, and that whoever is in charge at McDonalds for designing the menus is not just some cook, but probably a rather smart guy with extensive knowledge of what our taste-buds have come to appreciate, and how the logistics of assembling the burgers is tailored towards making one BigMac taste the same as any other one, you can rest assured that very few other restaurants will be capable of matching or beating McDonalds in terms of mere taste.

      Of course too much of anything isn’t good for your health, and monotony gets displeasing rather quickly too, but those aspects are besides the point here. Also there is a case to be made for when genetics, or other life circumstances, do produce results “out of the ordinary” .. but those are rare occasions, and being part of that minority is not always something you should pride yourself in. To provide an example – a large magazine here in Germany made a MP3 vs. CD listening challenge back in 2000. The person to get the most guesses right had suffered some hearing loss due to an explosion and could only hear up to 8kHz on his left ear, which exposed certain MP3-compression-artifacts to him that would have otherwise been masked by the higher-frequency contents of the music. Link to the article (German):

      In a desperate attempt to get at least somewhat back on-topic, I think that some part of the discussions audiophiles have is because some people want to just enjoy the music while others mostly enjoy how well certain challenges have been dealt with. Of course that $75,000 Rockport vinyl player is a remarkable achievement in its own right, but I highly doubt it’ll be capable of matching the $50 Sansa Clip+ in terms of sound quality, if the Sansa’s MP3/FLAC and the Rockport’s vinyl-LP have been made from the same source.

      Delete
    7. Thanks KurtW & Grunty for your comments (I'm combining two replies here to save space, we're at 200 comments again). You both raise some good points. Goldstein claims to have used "wine experts" working in the industry for at least some of his tastings. I'm not sure if KurtW listened to the podcast I linked, but there's also some other interesting info presented beyond just Goldstein's.

      Grunty left out the link to the German listening trial: http://www.heise.de/ct/artikel/Kreuzverhoertest-287592.html

      My main point was to show the "experts" tend to, overall, rate more expensive products higher, yet the general public, overall often does the reverse when they don't know the price of what they're tasting or listening to. If you believe the Robert Parkers and Michael Fremers of the world you generally get better taste and sound, respectively, by spending much more. My main point is that's often not true. In fact, it appears to be not true the majority (as in more than 50%) of the time.

      I agree wine isn't a perfect analogy for audio but there is a lot of commonality. And wines are very subjective--much like speakers and headphones. I agree you could skew the results of blind wine tasting by the types of wine you poured, the complexity, etc. Bottle aging is another factor although I've read many of today's wines, even expensive ones, often fail to improve significantly once bottled--instead they degrade. But, again, that's arguably subjective. There's little question the taste does indeed change over time.

      I think there's a bit of "wine snob" in your comments even if that wasn't your intent. You're associating people who like a $7 Merlot with those who like Big Macs. The same thing happens in audio. When Joe Average claims he can't hear the difference between his $75 DAC and a $7500 DAC, the "real audiophiles" are quick to point out that's because he doesn't know what to listen for, didn't listen properly, etc. It's a lot like an oenophile claiming Joe Average hasn't fully developed a "taste" for fine wines.

      The big difference is, the few times they do participate in blind tests, even the golden ears often can't tell gear apart they otherwise claim should be very different. Throwing a bed sheet over the equipment doesn't change their supposedly advanced listening skills, hearing acuity, etc. It just removes their subjective bias. James Johnston joked (I'm paraphrasing) "Any good audiophile should be able to tell you the brand of amplifier he's listening to... as long as he's close enough to read the name on the front panel"

      I agree with Grunty that ultimately we all should be realistic about respecting different levels of taste and hearing acuity. Where there are differences that survive blind comparisons, and with wine there nearly always are, it's an individual subjective preference beyond that point. Robert Parker's opinion shouldn't much matter. In blind tastings, the wines are often scored or ranked very differently by different tasters--even expert tasters. Sometimes you need a very large number of tasters to even see if there's any reliable consensus.

      In effect, the highest volume products are those that appeal to the most people over a broad demographic. And you can bet when Starbucks decides to roll out a new coffee or beverage worldwide, it's not because some Robert Parker or Michael Fremer "expert" declared it the best. They probably conducted extensive tasting sessions, pilot programs with a limited number of stores, etc.

      But group consensus doesn't matter if there really isn't any perceptible difference between two products. If Starbucks found using blind tests even highly regarded coffee experts couldn't tell a $2/KG bean from a $20/KG bean, which do you think they'd offer in their stores? Why shouldn't the same thing be true in audio?

      Delete
  153. Have there ever been any extensive blind listening tests to compare different brands and models of headphones? (I found one paper but I don't have access to AES: http://www.aes.org/e-lib/browse.cfm?elib=13724).

    I know HeadRoom measures different aspects of headphones and gives a value rating, but are any of the differences easily distinguishable (other than looks, comfort, and how well external sound is blocked)? I know listening pleasure is subjective, but do ultra expensive headphones really alter/reproduce sound "truer" or objectively sound better (i.e. if you had people blind listen to 10 different headphones, would the most expensive/"best" headphone always be ranked highly)?

    From my experience, when I switch between headphones, I notice a difference, but it feels more similar to changing the position I'm sitting in rather than a difference in the sound coming out of the headphones. I've even caught myself thinking that I'm wearing one pair of headphones due to what I hear, and they turn out to be a completely different pair. After that episode, I'm not sure that there really is any obvious difference between the phones I own.

    Furthermore, since audio engineers produce music using what I would assume to be standard production headphones, as in not HD800s, D7000s, LCD-2s or Tesla T1s, wouldn't using standard production headphones best reproduce music "the way it's supposed to sound"? Maybe audiophile headphones reproduce better high frequencies, but if the engineer never heard them or took them into account because of the headphones they used, why would an audiophile headphone be better?

    ReplyDelete
    Replies
    1. Headphones are difficult to blind test unless perhaps the person listening has never worn any of them and has no idea which headphones are being compared. Otherwise they're all too easy to identify by feel alone which makes it's not-so-blind if you someone pops them on your head and your brain thinks "ah those velour pads mean the Sennheiser HD650".

      So, no, I've never head of anyone trying to find people to participate in such a test. You would have to survey them in advance and, if they had ever tried anything you wanted to compare, you would have to tell them "never mind" and find someone else.

      Delete
    2. There are few cases of headphones that you could test blinded like the different impedances of some Beyer models or maybe the Denon Dx000 series

      It would be cool if we could do the equivalent of the HK's blinded speaker tests. Unfortunately for that idea, the majority of different headphone models are easily identifiable by touch alone.

      The same sort of biases that apply to sighted listening of DACs, amps, and cables also apply to headphones. Fortunately headphones usually vary a lot more than most of those other links in the signal chain as verified by measurements. Also, high end headphones tend to vary a lot more than high end speakers. That can give us a decent amount of confidence that we're hearing something real but its hardly perfect.

      I don't think that it's possible to do much better comparing different models of headphones but I'd love to hear new ideas about better objective testing.

      Delete

PLEASE NOTE! If you see "LOAD MORE..." OR "LOADING..." just above this paragraph, you need to click that link to load the newest comments. It may take several seconds. When all the comments are displayed the Load More/Loading link disappears.

COMMENT APPROVAL: Comments may take anywhere from a few minutes to a few days to be approved. You might want to save a copy (highlight Ctrl-C) of your comments before submitting in case something goes wrong. The approval process is necessary to prevent obscenity, etc. Please keep comments on topic. Comments promoting commercial interests will not be approved and may be reported as SPAM.