Making Big Data and BI Work Together

For enterprise IT and the end-users it supports, the interplay between big data and B.I. can prove as exciting as it is frustrating.

As enterprise executives and end-users eagerly look to gain meaningful intelligence and fast time-to-insight from deep wells of rich data—enabling them to react more quickly and intelligently to market conditions, deliver better customer service, streamline internal operations, and differentiate the organization from among the competition—IT is charged with facilitating such desires for agility even as rivers of data continue to pour into the organization.

With storage costs low enough to easily and cost-effectively store vast amounts of data, many IT organizations opt to store virtually everything they can. While that satiates some of the desires demanded by end-users, it increases the pressure on the makers of B.I. tools to create offerings robust enough to make meaningful, quick, and accurate sense of all available data.

In the real world, it’s not just about which BI tools to use: it’s about figuring out how the existing technology infrastructure will (or won’t) support meaningful B.I. derived from big data, which business needs should be satisfied first, and how to foster a truly collaborative culture whereby disparate business units share insights for the collective betterment of the organization.

Rethinking Infrastructure

When seeking ways to leverage the power of big data and B.I., IT organizations sometimes expect that existing infrastructure will fulfill their needs in providing end-users with fast, meaningful analysis. This expectation can become a pitfall.

“How do you serve up large sets of data to many users in a short amount of time?” asked Howard Dresner, Chief Research Officer for Dresner Advisory Services. “You need to take a look at the anticipated usage patterns, and then build out the infrastructure accordingly to make sure that you’re going to be able to deliver the appropriate levels of performance to users.”

He added: “If you’re now serving up many terabytes of data to users, and you couple that with, for example, a host of mobile devices that are running BI apps, that’s a real strain on more traditional infrastructures.”

Dresner advises IT organizations to think differently about their approach—not only to technology infrastructure, but to the structure of data itself: “I see organizations moving towards more non-traditional data structures—columnar, in-memory, or appliances that use those approaches—just to get it faster.”

Mobility Minded

Also taxing traditional technology and data infrastructures (while paradoxically helping to drive more effective BI across the board) is the proliferation of mobile devices. Once considered the holy grail of collaborative computing among enterprises, today mobility is very much reality.

“In some vertical industries, like retail, it’s all about mobile,” Dresner asserted. “I talk to retail organizations every day that are buying hundreds of iPads, for example. That in and of itself is driving so much more activity from a B.I. perspective in exercising the infrastructure more so than anything else we’ve ever seen. And I think that’s a good thing: putting more eyes on useful and accurate data more quickly is clearly better.”

Dresner says that, while the increased use of mobile compute devices has taxed existing tech infrastructures, organizations that overcome such stresses and do well “are not being religious about using a particular architecture or a particular product. They’re being pragmatic and saying, ‘What’s going to work? We might have to change it in two years, and that’s OK. But what can we do right now that’s going to give us competitive advantage for the next 24 months?”

Focus on One Application At a Time

While it may sound like a daunting and potentially costly task to revamp or change infrastructure every two years just to attain smarter B.I. from big data initiatives, Dresner advises prudence: focus on one application at a time.

“The organizations that I talk to who’ve achieved success on this front have said, ‘We need to deliver X business intelligence to Y class of user over Z class of application, and this is how we’re doing it.’ To them, the ROI is good enough,” Dresner said, pointing again to the retail industry as an example. “When you look at retail, it’s very operational in nature. If you can correct a small problem across 1,000 stores on a daily basis, that’s real money. It adds up to millions and millions of dollars in some cases, so it can be justified.”

Changing Policies, Working Together

Success in marrying big data and B.I. isn’t only about tech and data infrastructure, or B.I. and analytical toolsets. Success also is reliant on organizational culture, and the willingness of IT and end-users to jointly work together to achieve the same goals. For some organizations, this isn’t easy. In many cases, the end-user community seeks to steer the B.I. ship, and wants IT to step back a bit. For IT, the question becomes how best to support end-user desires while maintaining control and management of the underlying data and B.I. infrastructure.

“The most impactful solutions occur when IT and users are aligned, and are both pulling in the same direction,” Dresner said. “If you have enough momentum around things like B.I. and performance management, eventually IT is going to come around because they’re going to realize that politically, it’s expedient to do so. But until you get to that point, IT is sometimes not engaged. Or sometimes there’s a standard that IT has deployed as perhaps part of some other solution, and users are off doing their own thing.”

“We’re going through a phase right now where IT is settling into a new role, which is that of systems integrator and facilitator of business unit requirements,” added William McKnight, President of McKnight Consulting Group. “And so there’s hybrid IT that’s forming all over the place. IT roadblocks—like not enabling BI self-service [for] the business community—will have to go away because information is power, and is a competitive differentiator for the business. We just simply need to give business units unabated access to information. Some current IT practices don’t allow for that, and those probably will go by the wayside in the next five years.”

Collaboration among end-users in different business units also is key in making B.I. work well.

“Collaborative BI is a big deal,” stated Dresner. “The technology can be a powerful enabler, assuming you have an organization that embraces collaboration and transparency. Most organizations say that they do, but they don’t. Here, the problem is more about people and less about technology. But if your organization is a truly collaborative culture, then collaborative BI is a big deal because you can more readily work with other groups beyond your own to share insights that will help drive towards consensus.”

original source : http://slashdot.org/topic/bi/making-big-data-and-bi-work-together/