by Donald Farmer
Simple truths are often the most unexpected. Here’s an observation which, to me at least, is straightforward but still surprising: today, business users have access to better technology than IT.
Don’t blame IT! They are constrained by budgets, the depreciation of capital purchases, the need to serve a broad range of users, and increasingly demanding regulation.
Business users, on the other hand, have it easy. You don’t like the company issue laptop or cellphone? Bring your own device! You think the dashboards and reports on the management portal look old and tired? Download the freemium version of the latest visualization tech and try your hand. You need data about demographics, weather, exchange rates or a host of other subject areas? You can access numerous free, or low cost, data sets online. Running out of shared storage space? Sign up for cloud storage and collaborate with others at minimal cost. Subscribe to all these services out of your expenses budget if you like, although many of the services are so cheap you’ll happily pay for them yourself and likely use them for personal work too.
In many cases IT teams have not resisted this trend, but instead discovered that empowering business users and domain experts with self-service unburdens administrators from unnecessary operations.
For example, a self-service data preparation technology such as Trifacta, enables business users to work directly with the data they want to analyze in a remarkably simple manner. As data preparation can be an involved business – Trifacta tellingly refer to the practice as data wrangling – empowering users frees up administrators from the often frustrating task of trying to nail down ever-changing requirements.
Breaking the eighty percent rule
Analysts have often said that data wrangling, in its various forms, takes up 80% of the development time in any analytic project. You might expect a data wrangling tool like Trifacta to reduce that time as much as possible. Well, the simplicity and power of working with Trifacta certainly does make data wrangling more efficient, but with this 80% rule, I personally take a somewhat contrarian view. I think it’s just fine that working with data takes up so much of our analytic effort.
In my view, data preparation and analysis are simply two sides of the same coin. To prepare data you need to hold an analytic purpose in mind, however tentatively formed. Otherwise you’re not even experimenting or exploring, you’re just playing. Equally, to analyze data is investigate not only its aggregations and patterns, but its structure too. And the more you learn about the structure of data, the more you might tweak it, reshape it, indeed wrangle it to reveal more patterns.
Whether you are comparing start and end dates of a process to analyze an elapsed time, or arranging demographic data sets into appropriate age-groups to find useful correlations, or simply concatenating name fields to create a more useful identifier, the distinction between analyzing and wrangling is a weak one; especially so with self-service technologies, because rather than being a cumbersome exchange of requirements between business and IT, this new, empowered analysis typically happens on the desktop of one savvy, and satisfied, business user.
The new role of IT
What then are IT to make of these newly empowered users? After all, it is still the hard-pressed system administrator who has to worry about performance and governance and security. My suggestion is that first of all they need a change of mindset and then a change of practices will follow.
IT need to see themselves, not as restrictive gatekeepers, but as supportive shopkeepers.
In the age of self-service, no one can impose or maintain all the restrictions of access and usage that were once common. However, IT can provision well-formed, well-governed and secured data to business users who can then serve themselves. Provisioning tools like Trifacta is also an essential step in this new process. Business users will understand their data through data wrangling, and communicate their knowledge through visualization and collaboration tools. If we provision simple, but powerful, self-service tools, business users are far more likely to use those than to seek out their own solutions.
The shopkeeper model is not a free-for-all liberalization of all data, despite the fears of some. If you have bought alcohol, cigarettes, or medicines, you will be aware that shopping is not an ungoverned activity. The new world of empowered experts is exciting and challenging, but also highly productive and rewarding for business. It does need IT to play a new role, and provisioning both data and technology is an essential first step.