One of the nice things about working in web technology, is that it’s easy to get immediate feedback on whether a design is successful or not. A wealth of data is available to us and can be helpful in identifying design victories and setbacks. But how much should designers trust their gut? When should they go with the numbers instead? The answer (as is usually the case with these things) is that it depends.
My own experience is that data can help refine existing designs, but is often useless in the face of really large innovation. The iPhone is a good example of this–conventional wisdom said that mostly business users were interested in smartphones, and they wanted real keyboards and small screens. The number of people in both the industry and the press that ridiculed the iPhone is staggering–but completely understandable. Because people evaluate ideas based on how they fit into existing models, but these models can be broken when a new idea is big enough.
Google Ventures has an article today suggesting some guidelines to use when considering whether to design based on numbers or to design with your gut:
“Curious about customer behavior? Use data. When it comes to digital products, web and mobile analytics tell us exactly what customers do. Even if customers say they would never, ever, ever buy rainbow suspenders for their avatar, we just never know what people will do when we’re not watching. Better to trust the data and see what people actually do rather than trust what they say they’ll do.
“Decisions about product quality? Use instinct. To build quality into a product, you have to pay attention to hundreds of details like crafting clear help content or moving that button 3 pixels to the left. None of these small changes individually would prove worthwhile with data. But taken together, they create an overall impression of quality — a halo effect that improves a product in many ways. So when wondering how much time to spend on the details, designers should trust their instincts.
“Deciding between a small set of options? Use data. There’s nothing like an A/B test for making an incremental, tactical improvement. When trying to pick the just-right words for a homepage header, there’s little to be gained in arguing over the right copy. It’s better to test a few versions and pick the right one based on data. The key is to measure the metrics that really matter to the business longer term (such as sign ups, purchases, or user retention) instead of just measuring clicks.
“Concerned with long-term impact? Use instinct. A good reputation takes years to build, but just one bad experience can destroy it. So when balancing between tactical easily measurable goals like more clicks, and long term goals like trustworthiness, it’s essential to listen carefully to your instincts. And if your instincts need a little boost, get curious: go out in the world, talk to people, and gather data.”
