I agree this would be nice but sadly criticism doesn’t work like that outside the dataviz community, pointing an end user to a critique taxonomy isn’t going to fly. We need to ensure we’re equally looking outwards for critique as much as we apply any inward looking discourse.
Personally I find critique difficult because people are able to fall back on “appeal” and “engagement” as modus operand. As you’ve written, “We also don’t know how to evaluate appeal”, is patently true. We need to get better at understanding and communicating what works well and separating clicks from usage. In business encouraging engagement with visualisation is much more of a cultural battle than it is simply design, separating the two is simply impossible.
As data visualisation approaches the “third wave” my own view is that we risk leaving users behind if we’re not careful. While Netflix might be able to afford and encourage more enlightened approaches, Tufte and Few still hold sway when it comes to 99% of business dashboards, simply because the “rules” work more often than they don’t. Winning that cultural battle depends on several things but the single biggest battle is around the value of any kind of data driven approach vs “gut feel”. The bar is simply that low. Recently I was at a data visualisation event where the overall discussion boiled down to the value of tables vs charts; we’re literally fighting at that level on the ground. Trying to encourage executives to invest in any level of visualisation is hard, trying to push an agenda that involves “gifs, playful color and novel visual methods to delight users” isn’t the world I can afford to live in at the moment.
The vast majority of practitioners out there are in business, they aren’t academics or coders, nor do they follow any particular tool or agenda with any allegiance — they’re simply trying to do their job and produce dashboards and data visualisations to help educate everyday business users. For them talk of a third wave is way too early and is potentially damaging — we need to focus on simple messages that can help the majority, not just focus on improving data viz for people who like data viz.
Developing new guidelines and throwing away the old “science” should be at the forefront of what we do as a community but we need to ensure it is done with the end real world user at the forefront of any research. Otherwise we risk a schism between what works in the community and what works in real life.
