DataChant is launching a new series that will focus on bad practices in Power BI. Instead of sharing with you tips & tricks on how to deliver best-of-breed analytics solutions in Power BI, let us try taking the opposite approach, and share the most common worst things you may do in Power BI.
You may ask yourself why we should waste time teaching bad practices in Power BI instead of focusing on best practices. There are two reasons why: The first one is simple. I think it is more fun to learn from others’ mistakes rather than their success. The second reason is crucial – Bad things are likely to imprint in you a powerful emotional memory that will not fade away so quickly as an average best-practice.
How this is going to work? In the past, I published a series of blog posts on the 10 pitfalls of the data wrangler in Power Query. That series further explained in Chapter 10 of my book. This time, we are going to take a broader perspective and focus on all-things Power BI, bottom-up, top-down, the ins and the outs. For that reason, I invited a few of my colleagues and some of the best Power BI bloggers / MVPs to join the fun as guest authors here on DataChant.
Would you like to join as a guest author? Contact me at gilra@datachant.com. The series can cover anything on Power BI. Power Query, M, Modeling, DAX, Visualizations, Storytelling, Architecture, Governance, Security, Power BI Embedded, Premium, Pro, Free, Migrations from Tableau, Qlik or Excel, The little things that annoy you or the big things that can wrongly divert an entire business from its course. There are only two main requirements if you choose to accept this challenge and join as a guest author: Focus on real-world bad practices and have fun.
The natural place to start this series is in a prologue. In classic greek tragedies, prologues were used as the stepping-stone that would warm up the audience and set the emotional tone towards the inevitable catastrophe that awaits the main characters. Inspired by this analogy, I thought it would make sense to start this series with a classic yet-controversial visual that befits a Greek tragedy prologue – The Pie Chart.
Source: William Playfair‘s Statistical Breviary of 1801
The Pie Chart – A Prologue to Bad Practices in Power BI
The Pie Chart is a fascinating visual. Its circular shape can fixate our attention. No matter how many other visualizations are displayed in your report or dashboard, there is a good chance you will first stare at the Pie Chart. I suspect that this may be related to something innate in our perception. Our forefathers used to gape at the moon or worship the sun. Even today, when we have a conversation, our line of sight is focused on the circular shape of our pupils. Can evolution theories explain our fixation to Pie Charts? Is it nature or nurture? Perhaps we are conditioned from a young age to crave cakes and pizza slices.
The innate fixation to Pie Charts may also explain why may choose this visual when we build our report or share our message in situations where the Pie Chart is pointless. Here is one of the many examples I found recently.
Warning: This Pie Chart can elicit visually-induced seizures.
You can see that the Pie Chart above consists of too many distracting thin slices that provide no added-value to the visual. The authors of this Pie Chart could easily switch to a Bar Chart to show the top emojis or at least consolidate all the infrequent emojis into a single “Others” slice – but they decided not to do so. Perhaps intentionally.
We see too many cases of Pie Charts that are used for the wrong reason. Perhaps this is why Pie Charts have become so controversial among professionals of analytics experiences. But for our Prologue of Bad Practices in Power BI, we are not going to dive deeper into the theory of Pie Charts. For further reading, I recommend Stephen Few’s “Save the Pies for Dessert” article here, Ed Tufte’s “The Worst Chart in the World” article here, Robert Kosara’s blog here, and if you want to take the mission of killing the Pie Charts in your organization, I recommend sharing Jason Clauß’s “The Five Stages of Grief over the Death of Pie Charts” here.
A Pie Chart Survey
In an attempt to provide some added-value in a topic that was already scrutinized by the best experts in the domains of analytics, I decided to create this survey and shared it on my blog and social networks. The results of this survey and my possibly-biased analysis are presented below (You can click here to view the live 9-page report in full screen).
The 574 respondents of this survey included users who are both report producers & consumers (306), report producers (252) and report consumers (16). The low number of respondents who don’t produce reports was low. It can be explained by the fact that most of my readers develop Power BI reports, and I don’t have the reach of enough report consumers. The low number of non-producers can also be explained by the possibility that consumers of reports are no longer passive clients of centralized BI reports, but instead, they are the authors of self-service analytics artifacts.
30 executives, 53 directors, 102 managers, 118 architects, and 271 analysts were measured by two types of questions. The first type (Q1, Q2, Q3, Q6) focused on the respondents’ preference for Pie Charts. The second type (Q4 + Q5) measured the error rate of users to determine the top two slices in a Pie Chart and a Donut Chart.
54.88% of the respondents (315) preferred the Pie Chart over the Bar Chart in question #1 when there were only two slices in the Pie Chart (I must admit that I expected a lower rate here, but to honor the winners in the first round, I decided to use a Pie Chart in the report above to present the results).
While the Pie Chart won the first battle, it has lost the war. Support of Pie Charts has been quickly dwindling in the next three rounds. In question #2, with only 4 categories in the visuals, 16.2% of the respondents preferred the 4-slice Pie Chart. In question #3 with 8 categories, only 6.3% preferred the 8-slice Pie Chart. To take revenge for the loss of the Pie Chart, a surprising blow to the Bar Chart was given by a Treemap in question #6 with 40.6% of the respondents preferring the stunningly beautiful geometrically organized rectangles of the treemap. Still, the Bar chart won the day with 56% of the votes.
Nevertheless, the Treemap deserves a dedicated blog post in the series of Bad Practices in Power BI.
Preferences by Levels and Roles
I am used to hearing from many developers that Pie Charts are adored by executives. The results suggest that this is an urban myth. As the respondents had higher roles, they were less likely to prefer Pie Charts – No matter how many slices were presented in the visual. 23.69% of the analysts, 42.37% of the architects, 44.12% of the managers, 45.28% of the directors and 56.67% of the executives preferred to never use Pie Charts.
The preferences for Pie Chart were similar among producers who are also consumers and producers who are not consumers. Due to the low number of consumers-only (16 respondents), we may not have a large enough sample to confirm but we may see here a trend for a higher preference for Pie Charts by consumers (18.75% preferred Pie Chart with 4 slices over Bar Charts, while only 11.51% of the producers preferred 4 slices). Still, the majority of consumers preferred Pie Charts with 2 slices and didn’t choose the 4-slice or 8-slice Pie Charts.
Error Rates of Pie Chart vs Donut Chart
Questions #4 and #5 measured the error rate of respondents in detecting the top two largest slices (The correct answer was London & Paris).
To fool around, I used the same data for questions ##4 and #5 and I deliberately revealed the answers in question #3 (see below) when I displayed the top 2 cities, London & Paris, in both visuals. But many respondents didn’t pay attention (or perhaps paid more attention than they should – thinking I tried to mislead them).
I didn’t expect to find so many errors. 23% of the respondents failed to identify the correct top cities. And the Donut chart was even worse. 30.3% of the respondents failed in the Donut Chart test. For more details about the distribution of error rates by roles and user types, take a look at the screenshot below, or navigate to the relevant page in this live report.
The most accurate respondents were managers. 69.61% of them correctly identified the top slices with zero errors. Then came the analysts (61.25%), directors (60.38%), architects (60.17%) & executives (50%). Given the high rate of mistakes by architects, directors, and executives, you may want to rethink the use of Pie Charts – Especially when you have 8 slices or more.
I wanted to learn how the preference for Pie Charts by the respondents (reflected by the number of preferred slices) can impact the likelihood to wrongly detect the top two slices. From the chart below, you can see that 44.44% of respondents who preferred 4 slices had errors. Mistakes were less common in the group who preferred no Pie Charts (37.86%) or stayed with only 2 slices (36.96%). This gap may suggest that users who dislike Pie Charts are more likely to be aware of the inaccuracy of this visual, and as a result, they were more careful when they answered the accuracy-type questions.
Using Microsoft Forms, I was able to collect the total duration of the survey for each respondent (But not the duration per question). The average time to complete the survey was higher for 355 respondents who didn’t make any mistake (214 seconds) and was the lowest (171 seconds) for the 87 respondents who failed in the two questions. Here we have evidence that Pie Charts with high number of slices require significant time from users in order to correctly identify their size.
When we look at how the number of errors and the number of preferred slices can affect the time to answer the survey, we see that there are two distinct groups of respondents. The first group is 15 respondents who preferred the 8-slice Pie Chart (Question #3 or Question #6) and correctly identified the top slices. They may brag about how easy it is to identify top slices in an 8-slice Pie Chart, but they were also the slowest. It took them the longest time to answer the survey (288 seconds). The second group is 60 respondents who made one error, and preferred the 2-slice Pie Chart (Question #1) but didn’t select the Pie Chart in any of the other preference-related questions. This group invested 257 seconds to answer the survey. They are probably aware of the accuracy challenges of Pie Charts, so they invested the longest time to identify those top slices – but they were still incorrect. Their time investment didn’t pay off.
Next, I applied sentiment analysis and key phrase extraction on the textual answers to the last question in the survey: “What do you think about Pie Charts?”
The results can highlight the controversy of Pie Charts in a vivid way. For those who didn’t choose Pie Charts in any of the questions, the average sentiment started relatively negative (Sentiment score 0.41). From here the sentiment steadily increased as the respondents liked more slices in their Pie (Slightly positive with 0.56 score for 2-slice users; Very positive with 0.71 score for 4-slice users, and extremely positive with 0.85 sentiment score for the 8-slice users).
Finally, as this is just a prologue for our new series on bad practices in Power BI, I wanted to share with you the survey results in a way that you may either love or hate. You can view the Power BI report here, and flip to the last two pages to view the live Pie Chart and Treemap views.
Please share the survey results with your colleagues and managers to start working together towards an effective user-experience for your analytics solutions. Stay tuned for more articles in this series and subscribe to DataChant to gain access to the Power BI report file that I used to analyze this survey.
Do you want to share bad practices? Please contact me at gilra@datachant.com.
Is it wrong that I was craving a whipped-cream topped slice of pumpkin pie while reading this?
Definitely bad!
Interesting. “The most accurate respondents were managers. 69.61% of them correctly identified the top slices with zero errors. Then came the analysts (69.61%)…”
So the analysts came in second, with the *same* percentage as the managers.
(Yes, I realize this is a cut’n’paste error, just giving you a hard time)
Thank you for the hard time 🙂
I made the correction: “Then came the analysts (61.25%)”
Some of those differences would not be statistically significant -just chance variations amongst the groups, esp for example the duration by error rate and slice preference, which goes down again for the higher number of slice preference.
Thanks for this really useful and fun way of exploring visualisation best practice. Would it be better to lose the decimal points ie are the numbers at the decimal point level useful in these charts? Do they either provide useful information or are they likely to impart any real accuracy in this sample size?
I don’t get how this is relevant – I mean, every single time I use pie chart, I use data labels that shows percentage (and sometimes even number itself), so whoever is reading cannot do any mistakes.
Thank you for sharing. Would you still think the Pie Chart will be effective with 20 labels? And what would you do if the end-user wants to sort the items alphabetically?