I'm prompted to write this because of a video by GoldenSound titled Why you can't trust audio measurements I recently watched. The video highlights a number of issues regarding the reliability and applicability of audio specifications and their measurement. In so doing, it presents an opportunity to share some of my thoughts on the subject.
So, what's my position on measurements and specs?
First, if you haven't watched the video, I recommend you do so. It will frame what follows nicely.
I use measurements to confirm that my theoretical predictions have translated as expected into the real world. In other words, measurements are a valuable tool for development because they help me validate designs. However, measurements themselves don't demonstrate suitability for purpose. The design itself does that.
When it comes to making these measurements available to the public in the form of "specs", things get complicated. There's no question that consumers need to know the major performance aspects of gear. These include things like maximum power output, amplifier gain, etc. We're talking here about things you need to know to determine whether a product will do what's needed in your system. Yes, by all means, publish these details!
But I don't currently think supplying performance specs beyond these basics is in AVA's or its clients' best interests. This is for a few reasons. First, as GoldenSound's work shows, it's difficult to ensure comparable measurement contexts. This renders making meaningful comparisons between published specs difficult at best.
Second, to reach meaningful conclusions beyond the most coarse, you need much more complete specs than any manufacturer is currently providing, along with an engineering degree or something close to it. I'm not joking.1 Even the datasheets for audio opamps and similar components that manufacturers provide engineers to convince us to adopt a part and help us use it are consistently incomplete, despite pages and pages of data and graphs.
OK, fine. For specs to be really useful, they need to be incredibly comprehensive. But what's the harm in publishing a subset of numbers anyway?
The scenario we are trying to avoid is people making product selections from irrelevant data. I've seen it happen. Everything from, "I'm getting the one with the best SINAD," to, "I'm getting the one with the least/most feedback," to, "I'm getting the one with the highest slew rate," to almost anything else you can think of.
I'm flat out not interested in designing for the best SINAD or most/least feedback or highest slew rate or {fill in the blank}. I'm very interested in designing equipment that's best fit for purpose. This means balancing all performance aspects to deliver something that cheats the music as little as possible. This is an incredibly wicked problem and one that forces users to put a lot of trust in the engineer.2 No simple set of measurements has a hope of capturing all the decisions that go into a design.
That said, I'm conflicted. One of the things that's hardest to establish with clients is credibility. "Good specs" is one of the ways of doing this. However, I remain concerned about the message publishing them sends. It tacitly legitimizes shopping by specs while providing little of value to those who understand the limitations.
This isn't an ideal state of affairs, I'll admit.
