Brett Sheppard briefed author Cindi Howson in The Briefing Room on August 7 to discuss Tableau’s data discovery tools. Watch the archive.
Visual data discovery seems to be all the rage this year with new products and high-growth companies. What’s driving this interest – the pretty pictures or the self-service? And will this new category of tools finally take business intelligence (BI) mainstream or are we simply trading spreadsheet chaos for another kind of chaos?
There is still a fair amount of confusion about what is visual data discovery and what it is not, so I’ll start with a definition:
Visual data discovery tools speed the time to insight through the use of visualizations, best practices in visual perception, and easy exploration. Such tools support business agility and self-service BI through a variety of innovations that may include in-memory processing and mashing of multiple data sources.
Some befuddled BI teams though are shrugging their shoulders and asking, “Isn’t that what ad hoc business queries were supposed to do?” Well, yes, to a degree. Two of the biggest differences in business query tools and visual data discovery tools are the use of graphs and the degree of user autonomy. In a business query tool, a user can certainly add a bar chart to a dense page of numbers. But the chart is an after-thought. In fact, according to a TDWI survey last year, users spend two-thirds of their time analyzing data in tabular versus chart form. This may be appropriate when you need a precise number (How many widgets do we have on hand?), but not when you are trying to identify patterns, trends, and anomalies. With visual data discovery tools, the query and visualization process are one in the same. Drag a time period onto the page and up pops a trend line. Add a product category, and perhaps that trend line is now automatically converted to a trellis or small multiple chart. Research has shown that when data is represented graphically, we use less cognitive resources to make a decision and retain information better. So these graphs are more than just pretty or engaging; it’s about speeding the time to insight.
The other big distinction with visual data discovery tools from business query tools is the degree of user autonomy. Business query tools generally require a metadata layer that IT will often design and build. This metadata layer provides a layer of abstraction from the physical database schema with potentially hundreds of tables. With a visual data discovery tool, business users are often working with a subset of data, either a flat file or spreadsheet, so IT is not a bottleneck. If a real-time query is involved, the visual data discovery tool may automatically model a metadata layer, giving its best guess at what’s a metric and what’s a dimension, again with little to no IT support. Somebody would have to write the SQL for the initial query and define the joins, but once extracted, the data is often loaded into an in-memory engine. (See BI Scorecard’s Visual Discovery evaluation framework for detailed features to consider.)
If the pretty pictures and degree of business autonomy make you want to rush to buy these tools and throw out your business query tool, keep in mind that there are gaps the visual data discovery tools don’t fill (yet?): many are desktop authoring and don’t work as well with multiple fact tables in a single query, for example. Others provide a high degree of interactivity and exploration on the desktop, but not on the Web so just how to share analyses is a work in progress for some products. So right now, I recommend companies add visual data discovery tools to their BI tool portfolio and view them as a complement to the business query module. Time will tell if the modules fully converge.
Wait. Haven’t we been here before with desktop OLAP tools? The industry pendulum does seem to be swinging back to the late 1990s – think Microsoft Analysis Services, Cognos PowerPlay, Essbase, TM1, all with their beautiful, highly visual front ends (ProClarity, Wired for OLAP, Executive Viewer to name a few). These departmental initiatives grew into chaos, so the desktop OLAP tools became enterprise grade … and IT once-again became a bottle neck.
The visual data discovery vendors, then, should take note of history: The more successful products and vendors will empower users without overwhelming them. The tools will be agile while also being scalable. And the savvy IT departments will embrace them rather than run from them. After all, in this era, it’s survival of the smartest and the fastest, not the perfectly controlled and architected.
How important is visual data discovery in your organization? Take the Successful BI Survey.
About the author: Cindi Howson is the founder of BI Scorecard, a resource for in-depth BI tool evaluations based on exclusive hands-on testing. She is the author of several BI books including Successful Business Intelligence: Secrets to Making BI a Killer App, a TDWI faculty member, and a frequent contributor to Information Week. As a consultant, she advises clients on BI strategies and tool selections. Prior to founding BI Scorecard, Howson was a manager at Deloitte & Touche and a BI standards leader for a Fortune 500 company. She has an MBA from Rice University.
August 7, 2012 - 3:03 pm
Great summary – interesting that you mentioned Cognos. We co-announced Impromtu at the National Center for Database Management – back in the day. We saw the handwriting on the wall – and though we still use PowerPlay, we never “Upgraded” to the server version but kept it where it belongs – on the user’s desktop. Rather than create the massive cubes, we created proceedures which allowed massive data to be shaped into visualizations tailored to the decisions being made. IT loves proceedures, the end-users loved that they could poke around – and no ‘cube’ required. Thanks for the great reminder!
@JRMigs
August 8, 2012 - 7:38 am
Cindi, are you actually saying that using less cognitive resources to make a decision is a good thing? I guess we are too accustomed to equating effort with result. But you make a good point, the less effort (use of cognitive resources) we can put into getting a result the less expensive that result will be, and the more results can be produced per unit of time. Thanks for the insight.
August 8, 2012 - 8:51 am
Hi, Wayne, yes, I think it is more about efficiency of decision-making than effort. If you can look at more data, while sifting out the irrelevant stuff and better retaining the important stuff, that can yield a faster and better decision. If we are in information overload, it slows decision making. I do think though much of this depends on the decisions we are trying to make. “Easy” ones – like we are low on product inventory in warehouse A – can we ship from another, which is closest? That kind of decision may not involve as much data or as many options to sift through. If however, you are looking at clinical drug trials, food bourne illnesses, patient treatment plans where we don’t even know what may or may not be related, software that helps us sift out the less important and highlight the significant is crucial.
Regards,
Cindi
August 10, 2012 - 1:44 pm
Hi Cindi. I agree with you but isn’t that challenge the same as it ever was? And that is, how do we define/agree how info points are related? What is very important, important, not so important, etc. ? Not just for now, for who wants it, for selected resources, but in a way that provides the measure of validity the info needs to stand up to repeated and not always kind examination. You need look no further than the presidential campaign to know what I mean. Same facts, different visualizations.
August 10, 2012 - 12:02 pm
Great warning on the dangers of too much visualization power. I was not in the workforce for the late 90′s boom in BI tools. I can see where at times simpler is better. Nobody really cares how the work gets done except for the one doing it. If they just have questions and don’t like playing with data they would rather just ask someone who could tell them the answer.
August 10, 2012 - 4:20 pm
[...] read a great blog post by Cindi Howson this week and a phrase she wrote really stuck with me. It was this: Time to [...]