How do you choose a Bi Tool?
Reflections after a “BI Bakeoff”
Last night I had the pleasure of participating in an event run by Pracitus and hosted by Xudu Data (Matt Weathers and Kelly Lucktaylor) — “BI without the BS”. The evening was billed as chance for attendees to see Tableau, Qlik and Power BI in action, “providing a practical demonstration of the capabilities of each tool as they go head to head to build the most insightful visualisation utilising the same data set”.
The idea came from Matt and Kelly watching Iron Viz at Tableau’s European Conference and wanting to repeat the experience but with three different tools. It was a chance to have some fun with data, get some experts in the three products together with business users and executives and see the art of the possible…and there’d be beer. How could I refuse?
Why Tableau, Power BI and Qlik?
The three products were chosen because of their positions in Gartner’s Magic Quadrant — Gartner don’t break out products in the quadrant (Microsoft have a range of BI products from Excel upwards and Qlik has Qlik Sense and Qlik View) but Power BI and Qlik Sense seem the sensible products to throw in the ring with Tableau.
The Challenge and Scoring Criteria
The challenge was laid out to us as below
- Build the most visually appealing insights and storyline
- It must be able to be built within 20 minutes from scratch in front of a live
audience - The data set provided can be blended with any other information the competitor sees fit to enhance the story
- Any data prep and joins can be researched prior to the evening but must be
demonstrated on the night - Note that only the selected tool can be used for data prep and visualisations
- The final output will be judged by a team panel
The judging / scoring criteria were laid out as below:
The dataset
We were given the data four weeks before the event but due to the Tableau Conference in New Orleans, plus a few other events I started looking at it a week before the event.
The data was on eSports — “The fastest growing sport you’ve never heard of”. If, like me, you’re in the dark about this growing phenomenon then let enlighten you. eSports, and in particularly DOTA2 (Death of the Ancients) hold huge tournaments viewed by millions people worldwide, teams earn millions of dollars in prizes and sponsorship. The data showed the players, their teams, the games they play, the tournaments and positions; as well as the prize money they earn at each competition.
The dataset itself consisted of 6 files and had its limitations. It showed the Top 100 players only, the data was also represented at different levels (Prize Money is awarded to teams not individuals) and so it was hard to draw definite conclusions.
Preparation and ideation
I know nothing of eSports (in fact I find it hard to care how much people I don’t know are earning) and so finding a story that was an interesting problem, especially considering what was possible in the data, was a challenge. Similarly any preparation had to be done live — and so any cleaning was limited to what was possible in 20 minutes.
My other consideration was showing the audience something of Tableau, I was conscious that if the audience came away thinking “I could use that” then that would be a great result but I also wanted to win (I’m hugely competitive), balancing that competitive edge with the aim of showing off the strengths of Tableau was hard I’ll admit.
My initial reaction to the event was one of trepidation, the last thing I want to encourage is the perspective that BI and visual analytics is only for experts and so I seriously considered grabbing an audience member who’d never used Tableau in the networking, giving them a 30 minute training on the tool and the data and letting them take my place with me acting as coach. Of course that would have been unfair on the audience, the judges and my fellow competitors — all of whom had come to see the contest between experts — and so I quickly abandoned that plan. However my concern remained, not everyone has a Chris Love, a Jen Stirrup or a Nick Blewden in their company, it’s not real life and so judging a tool only on experts using it is likely to lead to a narrow set of conclusions.
Those concerns aside I still needed an idea and approach, having used Qlik Sense and Power BI one of my worries was that one or both of my fellow competitors would build business-like, summary, panelled dashboards — and the audience would be left looking at two or three very similar dashboards. Therefore I deliberately and consciously choose a more informal, engaging visualisation, thus playing to the brief “Build the most visually appealing insights and storyline”.
My visualisation idea came from my first hour playing with the data — as I do with any dataset I explored, I dragged and dropped and I asked questions.
- Who’s the top earner?
- Have they always been the top earner?
- What makes them earn so much? Is it the big tournaments or the small ones?
As I asked those questions I built visuals. Given the limitations of the dataset that story of the ups and downs in being the highest earner was an interesting thread to pull on and so I decided to make it the centrepiece of my visualisation.
I worked on it and pulled together several elements — practicing probably much more than I should have to ensure there were no hitches along the way.
You can see one of those practices below to get an idea of how I built out my visualisation — there’s a few complex elements / calculations and I had to densify the data in Tableau Prep to get a row for every day / player combination so I could sum up the earnings.
The Final Visualisation
I’ve uploaded my final visualisation to Tableau Public so you can use it here.
I also had 5 minutes to present to the judges and audience and so I used the time to show my journey with Tableau and this data —highlighting the fact the dashboard was the end result and that data exploration is an essential part of an analysts role (something that only Tableau provides IMHO). You can hear one of my practices of my talk below to get an idea of what I said.
The Others
One of the things I was looking forward to was seeing my fellow experts present their tools — we all work in isolated, and rather partisan communities, seeing the other side is something we rarely get chance to see.
Jen Stirrup, representing Power BI, had only started looking at the data that day due to her other commitments, her bravery amazed me, I’d have been terrified, so to see her be able to pull out her story was fascinating. Jen concentrating on the battle facing female gamers, using simple charts to tell how few females there are and giving us the background of some of the abuse they struggle with. You can read about Jen’s experiences at the event here.
Nick Blewden, representing Qlik, took an approach that’s very familiar to me — he brought several charts together in panels and provided filters to show how they can be interactive, he also showcased Qlik’s story feature as well as producing a dashboard with a single Qlik click of his mouse (while the feature idea was impressive the result came up slightly lacking IMO).
His demo was very similar to something used to do almost every week in ten or fifteen minutes with new customers using Tableau’s Superstore data. It’s an approach I’ve been reviewing, while it shows the advantages of the product I’m aware the dashboard rarely show much insight into the data beyond the summary highlights and click through capabilities. So I was interested to see how the audience and judges would receive them, especially given I’d purposely chosen a very different approach.
The Judging and result
Nick was the lucky winner, getting both the audience vote as well as the judges vote. Congratulations to him.
My own interpretation of the result is that Nick showed something that was immediately familiar to many people expecting to see a dashboard, it had BANs (Big Ass Numbers) with a number of smaller charts and a map. It’s what we expect to see when we talk about BI — to be fair given the audience had come to see BI products pitted against each other I can’t blame them for choosing his dashboard. My own story was a more complex chart to follow and understand, it’s the kind of thing we in the Tableau community are used to building and seeing all the time in Makeover Monday but not something that, with hindsight, I should have expected to do well with a set of judges and audience used to a more traditional approach.
So how do you select a BI tool?
Well it’s certainly not by watching three experts use them for 20 minutes, I feel the audience may have gone away feeling they didn’t actual see much of the products themselves. There’s also a risk, and this was reflected in some of the Q and A that followed and in some of the judges comments, that events like these trivialise and equivalence the three products.
Does it really matter which product you choose? As we saw last night they’re all great.
Well for me the answer is a resounding and loud Yes, it does matter. The approach in all three is very different, with different skill levels required for starters. Also dashboarding is only the end of the story, something I attempted to get across in my talk afterwards — data exploration is a key component of any analysis. I’d love to see a similar event with beginners on the stage — starting cold from a dataset and all being given questions to explore shouted from the audience. That is something closer to real life data analysis.
Until someone is brave enough to put on an event like that then we’re left wondering what is the best approach to take. For me Stephen Few’s approach is perhaps the most independent and well structured advice you can find. Use his approach to assess each tool’s capabilities while using them on your own data (with the help of a partner / vendor to help with support and questions — contact us at The Information Lab if you’re looking for help assessing Tableau or if you want to know more on our thoughts on its advantages over Qlik and Power BI info@theinformationlab.co.uk ).
Conclusions
It was a great event to take part in, congratulations to Nick, and Jen, and a big thank you to Matt, Kelly and the the team at Practicus for inviting me. The judges were great (although I disagreed on some points — especially on the point that tables helped find errors faster than data visualisation) and provided some great debates and discussions that continued well past the scheduled finish and moved on to the bar afterwards.
Next stop IronViz!
(thanks to Laura Schofield @schofe23 for the pics)
