The next wave of business intelligence

Dan Vesset, IDC, reports on why business intelligence technology is something without which organisations cannot succeed.

However, as a set of tools and concepts for managing organisational and individual performance, BI still has a long way to go in reaching all the necessary people and processes in companies, government agencies, hospitals and schools.

IDC’s research into the BI market has identified that the market moves in 15-year cycles. The first of these periods, from 1975 to 1990, was characterised by production reporting on mainframes; companies like SAS, IBI and IBM began in the BI market during the early years of this cycle. The second 15-year period, from 1990 to 2005, saw the beginning of the ‘modern era’ of BI, characterised by end-user friendlier client-server based BI tools from vendors such as Business Objects, Cognos, Hyperion and others. Eventually, query, reporting and OLAP technology migrated from client-server to web-based architecture with the development of broad suites of BI platforms.

When we look back in a few years, we’ll see that 2005 was another turning point in the BI market and the beginning of the new wave of investment in BI by organisations across all industries. The current market cycle is expected to last until 2020 and will be focused on expanding the reach of BI to more users, both inside and outside the organisation. Figure 1 shows the market size and forecasted growth rate from 1990 through 2010.

As the BI market matures, the focus of organisations, BI software vendors, and systems integrators is also changing. To date, the BI market has primarily focused on delivery of information to analysts and managers. Progress has been made in delivering information faster and through various means (e.g. reports, dashboards, alerts on PDAs). However, analysts and managers represent only a relatively small portion of an organisation estimated at about 15-20 percent of employees. In fact, IDC believes that the market for reporting and OLAP tools for power users and analysts has reached a level of maturity that cannot sustain the growth rates of the past in terms of new licence revenue. Instead, this market is now being targeted by larger IT vendors such as Microsoft, Oracle and IBM, along with the existing specialty BI vendors. As the market continues to mature, it is highly likely that the larger IT vendors will continue to gain share in this market.

But what about the rest of the organisation? Whether we're talking about information workers with higher levels of freedom to decide about the work flows and processes they use on a daily basis or line of business employees who may be restricted by systems and policies in how they perform their duties, a vast population exists whose business intelligence requirements have not met their potential.

The next wave of BI will indeed reach out to these employees as well as other organisational stakeholders such as suppliers, partners, customers and government agencies to improve information delivery and decision support functionality for all.

This shift in market focus can be addressed only partially through existing BI software, which as already mentioned was created with the analyst or power user as the intended audience. Clearly, a frontline employee will have limited use for an OLAP or an ad-hoc query tool. In fact, to address the needs of front-line employees and line of business manager, organisations must redefine and expand what they mean by BI.

The expanded vision of BI must take into account not only the technologies involved, but also business drivers and performance management methodologies.

The business drivers that have ushered in the next wave of BI include:

Compliance
BI can help drive consistency in decision-making. It’s important to make the right decisions, but often it is also important that decisions were not made arbitrarily, i.e. that different employees followed similar decision processes that can be audited or monitored.

Competitive pressures
Within the process of performance management it is important to go beyond dashboards and reports that focus only on information delivery. To put BI into an operational context, it’s not enough to have dashboards that simply report on what happened. This information is valuable, but of limited use. Dashboards should also show context around the information and provide guidance for action – in other words, they need to be in the context of whatever business process the dashboard is built for and support predictive analytics.

Inter-company connectivity
The final driver is inter-company connectivity or the linking of business processes with your partners, suppliers and customers. It is important in these incidents to have a more comprehensive view of operations that goes outside your organisation.

At the same time, the discussion around BI implementation is changing from one of technology to best practices in the process of performance management. Some of the questions being asked include:

• What KPIs should I track, measure and act upon?
• How do I set up and manage BI competency centres?
• How do I manage master and meta data as part of broader data quality and compliance efforts?

BI technology must evolve to address the needs of the ever-broadening audience of users. As more data becomes available for analysis and more users expect to receive decision support based on this data, the scalability and availability requirements of BI systems are becoming more robust. IDC research shows that already today over 40 percent of organisations indicate that if their BI systems go down for just a few hours, there will be significant material impact on their operations. At this point, BI systems have not yet reached the requirements of operational systems. However, the trending is towards convergence of availability requirements between analytics and operational systems.

Another key emerging trends in the market is the convergence of traditional BI tools and search technology. In fact, it’s quite likely that some of the emerging BI applications will closely resemble common online community sites with the associated functionality for graphs, tables, comments, notes, ratings and other collaborative community features. It is also clear that search as a ubiquitous interface has the potential to penetrate many more users than end-user BI tools that require more training.

Finally, BI will find its way to the vast majority of users by being embedded within the operational applications already used by these end-users. IDC calls this evolving concept ‘intelligent process automation’ (IPA). IPA is the convergence of BI and business process management technologies. It is the automation of repeatable, operational decisions (not the more infrequent strategic decisions made at the executive level). This is not to say that IPA replaces executive dashboards or ad-hoc BI solutions. The two technologies have different audiences and serve different needs. However, only IPA can help automate repeatable, operational decisions to address both performance management and compliance issues.

When you are developing your BI strategies, you should look not only at the traditional software for query and reporting or the decision support needs of power users. Instead, expand your efforts to include features such as business process management, collaboration, workflow and annotations, in addition to new scalability and availability characteristics.

Dan Vesset is Research Director for IDC’s Analytics and Data Warehousing Software service.