The importance of Business Intelligence and Analytics has grown significantly over time with factors such as increasing data volume, decreasing data storage cost, and cloud adoption acting as tail-winds. While businesses have put their faith in BI and Analytics tools and related technologies over the years, the adoption of those applications by business users has been a challenge. I experienced it first-hand when I was helping a UK retailer upgrade its enterprise reporting platform.
For someone looking at it from a technical perspective, the upgrade to a new and completely revamped version of SAP BusinessObjects was something to cheer about. Similarly, for the project sponsors or the data leaders in the organization, it was a move towards better performance and a more robust platform with ‘self-service’ capabilities. However, during the user-trainings or my usual discussions with business users from various departments, I sensed anxiety and dread. The end-users belonged to procurement, merchandising, store planning, logistics, etc. and they had become comfortable with the operations and flow of the existing application and wanted status-quo. Some of their major concerns were doing simple activities on new applications such as navigating around and building simple tabular reports or charts on their own.
More than 5 decades of BI and Analytics: A Journey
There have been a lot of advancements in business intelligence and analytics solutions in the last 5 decades. We have come from decision support systems to visualization tools where users can simply import data and create visually appealing dashboards in minutes. The table below represents the evolution of BI and analytics solutions over the years and the business and technology drivers behind them.
|Business Drivers||Technology Drivers|
|DSS||Need for a framework for day-to-day operational decision making||Codd’s paper titled ‘A Relational Model of Data for Large Shared Data Banks’ in 1970|
Initial development of database management systems
|BI Reporting|| Even with the one-dimensional nature and rigidity of database systems, businesses were exploiting data profitably, e.g., Nielsen used it for rating TV Shows|
Need for a way to structure and bring together all business data
Reducing the cost of data storage
Increased adoption of ERP systems
|Emergence of RDBMS |
Kimball’s and Inmon’s proposals theorizing Data Warehouse design
ETL and OLAP go mainstream
|Visualizations|| Large time-to-insight in traditional BI reporting|
BI tools were difficult to use for business users
Huge dependency on IT for insights
| An exponential increase in the volume and variety of data|
The emergence of in-memory processing
Software-as-a-Service models become popular
|Search-based Analytics||Time-to-insight is still large compared to the pace of business|
Need for decentralized decision making
The emphasis of businesses to increase adoption of analytics
|Techniques such as Natural Language Processing(NLP) go mainstream|
Advancement in Artificial Intelligence & Machine Learning
Despite the advancements, the onus on an end-user to build technical capability to operate on the tool efficiently has remained. With the ever-changing technology landscape, the nature of technical proficiency also needs to keep changing. Such factors have been a hindrance in the easy adoption of BI and Analytics applications amongst business users.
The emergence of Search and AI-driven Analytics
The progress in Natural Language Processing (NLP), Artificial Intelligence (AI), and increasing processing and storage capabilities on the cloud have given rise to a new breed of search and AI-driven analytics applications that seem capable of solving the issue of adoption. These new class of analytics applications aims to capture the market by promising an extremely simple, fast, and democratized access to data at a nominal cost. Following are some of the key differentiators for these applications:
- Search and voice-driven analytics
- The near-instant response even with billions of rows of data
- Easy to implement and explain AI/ML processes and models
- Easy collaboration through in-built features
- Innovative pricing models
Tellius and ThoughtSpot are some of the known vendors of search and AI-driven analytics solutions. ThoughtSpot has disrupted the market significantly in a very short time. Within 7 years of its launch, it has acquired several Fortune 500 companies as its customers. It has also been featuring as the leader in Gartner Magic Quadrant for Analytics and Business Intelligence Platforms report for 2 years now. The seemingly instant success of ThoughtSpot is a testimony to the fact that it has made it easier for the business users, who are technically not proficient, to use data and generate insights, thereby driving adoption.
The other major players in the visualization space have also started integrating capabilities such as AI-driven insights, conversational, and search-driven analytics into their offerings. Tableau, Power BI, and Qlik have introduced similar capabilities in their applications. While Tableau calls it Smart Analytics, Power BI & Qlik call them Q&A and Insight Bot, respectively. In almost all cases, the capability comes into play once you have built the dashboard, and you want to delve deeper into the insights and not while building the dashboard. So, the problem for business users remains the same. They are still required to have technical proficiency for navigating the application.
Moreover, the recurring licensing costs make it difficult for enterprises to make a pro or premium license available to everyone on frontlines, i.e., store managers, floor managers, or supply chain analysts. The innovative pricing model of the new players in search and AI-driven analytics such as ThoughtSpot solves that problem. They charge customers based on the number of rows of data processed in a given duration.
Is search and AI-driven analytics the need of the hour?
A search and AI-driven analytics solution becomes even more critical in a situation like the COVID-19 outbreak. In the current scenario, when countries and businesses are under lockdown and supply chains are badly hit, it becomes imperative to decentralize the decision-making, which is frequent, time-specific, and location-dependent. A solution like ThoughtSpot can allow businesses to do so easily by making the necessary insights available to the frontline employees when they need them.
It remains to be seen whether this new breed of search and AI-driven analytics applications will co-exist with the incumbent BI and Analytics tools, or will it eventually disrupt them. The likes of ThoughtSpot and Tellius still may not have the necessary in-built data preparation and transformation capabilities. However, they can do enough to take analytics to the last mile of business, enabling faster data-driven decision making and solving many existing problems for businesses.