Recently, my husband and I wanted to buy some new sheets. But how to choose? Would they lose their shape over the years? Begin to pill?
A friend pointed us to Sweethome's recent sheet review. This was no joke. These testers had examined the cotton fibers under a microscope, washed the sheets multiple times, and even given them a literal smell test to make sure they didn't have any noxious post-factory odors. We were suitably impressed, and bought their recommended sheets without thinking about it twice.
But the experience left me curious about this magical little site. Who were these people and why were they so serious about sheets? I decided to ask Jacqui Cheng, editor in chief of Sweethome and its partner tech-site, Wirecutter, about the work they do and why it matters. A lightly edited transcript of our conversation follows.
Tell us about Wirecutter and Sweethome. What's the idea behind them? What sets them apart from other review sites?
You know how when you—or your parents, or your friends—decide you're going to buy something new, like a new portable hard drive or a new humidifier for your bedroom, you often do some research to find out which products have the best features for the best price? And then you have to find out which features aren't worth paying for, and why is there an $800 humidifier anyway!? Well, that's the kind of thing we're trying to do for you at Wirecutter and Sweethome. As consumers, we're all basically doing the same research on the same products all the time, and it's not only redundant, it's a wasteful use of our collective time for everyone to be doing that stuff all the time.
Wirecutter and Sweethome aim to do all that research and just tell you which one or two items are the "best" for the majority of the people, the majority of the time, for any particular category. We're trying to keep the world from wasting its time doing all that research, but also trying to keep everyone from wasting their time on features that are full of hype or don't matter when it comes to real-world usage. For example, at the moment we don't recommend any 4K monitors, and that's because it's just plain not worth it for most customers right now. You aren't going to find tech sites that tell you that so clearly—their job is to talk about why 4K is awesome and why it's the future, not why the home consumer has no use for it right now.
Sometimes, we even tell readers why they shouldn't rely on something that we're reviewing. We did a piece on portable breathalyzers and presented all our research, testing data, and analysis as usual. But during that process, we also discovered that nearly all portable breathalyzers that a regular person can buy on the market today are inaccurate, not to mention that people's blood alcohol content is very hard to predict over time (your BAC continues to go up for 90 minutes after you stop drinking).
So, not only is it wildly unsafe to try to use your own breathalyzer to determine whether or not to drive, even if you did use it, your reading isn't likely to match whatever the police might read if you were to be pulled over and given a police breathalyzer.
In the end, is it really worth spending $150-$300 on a device that is only slightly better than using a crystal ball? We think it's not, and we said so pretty clearly in our piece. However, we still named a "best" portable breathalyzer based on our tests & research (it was the most accurate of the bunch) for those people who don't want to listen to our advice, or just absolutely feel like they need to have one (they might want to see if their teenage kids have been drinking at all, or something along those lines).
In terms of how other tech or review sites operate, we love them—don't get me wrong!—but they also tend to review things on an individual basis instead of looking broadly at a category and determining the best. That's not to say their opinions aren't useful—we rely on other review sites heavily in order to help inform our own judgement on a category. In a way, we all work together virtually, but Wirecutter/Sweethome's goal is to do all that research and condense all that information into a simple, easy to understand format for the regular Joe or Jane who doesn't necessarily want to read 10,000 words on routers—they just want to know what's the best one to get at home to use with their iPads, you know?
The hope is that we'll all get back some hours in our lives that could be used on more important things, like spending time with people we like, or getting work done.
How does the testing process work? Which products took the most work to test? Any outlandish tales?
By the time we're testing a handful of items for a particular product category, we've already put in dozens of hours of research and interviews in order to even identify what to test. We really go through each product category with a fine-toothed comb and identify what an ideal product might look like, if such a thing were to exist, before we even begin to narrow down what's actually available on the market to test.
Each product gets a different testing procedure, but we try to put together a plan that does not replicate what other reviewers have already done. So, for example, if it turns out that a site like Anandtech or CNET already ran all the benchmarks you can possibly run on a new tablet, we aren't going to run those same benchmarks again just for the sake of doing so. We're going to name and cite them, and talk about their results, and possibly use that data to help formulate different tests.
We also try to make our tests as "real world" as possible. We recently published a piece on surge protectors where we had an electrical engineer use a noise generator to send voltage spikes through each of our testing candidates to see how each surge protector clamped down on the spike. No one else is doing real world tests like that—you just read the box and you know that this surge protector claims to be able to clamp down 600 volts over a period of 10 years, but does it really? With our tests, we were able to actually see—through the data—how much capability each surge protector loses every time it has to clamp down on a giant spike like that. (Did you know they can only do that so many times before failing, and the only way to know your surge protector is on its last legs is when it finally blows out?)