the analytics edge

You’ve invested heavily in the development of your application but your customers want more than the simple reports you have provided. They want a dashboard with some analytics that make them more comfortable that their money was well spent. It shouldn’t be complicated – just a few charts of trends and stuff like that. That’s how it starts: the journey of discovery and disappointment. You’d think this stuff was a commodity by now, but technology companies don’t like to give up their high prices, and the experts believe the complexity and specialist skill sets are required, so the products follow suit.

I was on a few sales calls this week, and this sad state of the industry was driven home by a couple of reactions to our new product offering. As with any vendor, we have tried a number of demo scripts to try to hook people’s interest, and we recently cut the demo way back to just a couple of simple activities. In each case, I was about 5 minutes into the new demo – open a transaction file, create a pie chart of sales by product, then create a table showing top 10 salespeople’s performance over the past 12 months with some colored highlighting. I refresh the dashboard and show these two reports, then pause for acknowledgement. In the first call of the week, I was surprised by a “way cool!” response. The second call was a slightly more reserved “cool!”, and I found the common response a bit amusing.

It was the third demo that has stirred up some deeper thinking, though. A large organization that sees analytics as a competitive necessity, they started by telling us that they own many different packages, from large and small vendors. They have bought analytics companies, and have built their own products from scratch. They have processes to follow, and asked for copies of any presentation materials we would be using up front. Very professional and detached, clearly setting the bar high before we started talking.

Knowing our demo script well, I apologized for what I said will appear to be a tedious demonstration of me clicking and typing, I assured them it would be short, and it was faster to show them than to explain our unique value. Working slowly through the dozen-or-so simple steps and pausing with the new dashboard reports on screen, we sat through a long silent pause. When they did speak, they apologized for the delay – they busy were taking notes, and were excited by what they just saw!

So what did these professional and experienced individuals see in our product that elicited such a response? They saw that we have significantly lowered the bar for adding analytics to applications. The discussion revolved not around the big analytic projects underway, but opportunities throughout the organization where groups couldn’t justify the effort and investment because it was so high. We joked that we create “disposable analytics” – it is so easy to add a new analysis to the dashboard that you can throw them away when they have served their purpose. It has an extremely small footprint that doesn’t try to be an application all unto itself. It is easy to use without specialist skills. And it is available at a cost that makes sense.

And it dawned on me — we have turned business analytics into a commodity, and it is long overdue.

Number one Hot IT Job for 2009, according to Baseline, is Business Intelligence Experts. That is bad news for business managers. It means there is a shortage of the skills needed by businesses. These people help “in a lot of decisions having to do with how the customer base is changing, market differentiation, pricing and services”.

To BI Experts, this is received as good news - salaries will go up because of demand; jobs will become more interesting with less mundane report writing and more strategic analysis work; and career paths will open up as you gain more exposure to things outside IT.

Business executives, though, may not have the reaction you expect. They believe they are paid the big money because they make the tough decisions, and when others try to move into that space, they react. This is not necessarily political in nature - it is just good business sense. They are always looking for better ways to do things, and if the BI guru suddenly becomes a cog in the gears of success, alternatives should be considered.

Fortunately (or unfortunately for the BI experts), there are a number of new BI solutions appearing on the market that are geared to providing BI-like answers without the need for ‘expertise’. They require less technology infrastructure at a lower cost using a cheaper skillset. I am proud to be associated with one. The pendulum is swinging back from a technology-centric, expert-driven ’business intelligence solution’ to embedding easy to use business analytics into applications. They do not require experts to run them.

So I caution the BI workers out there — your position might be hotter than you think.

A long time ago, I graduated from an engineering program with the concepts of effectiveness and efficiency held in high regard. Truly elegant solutions could produce phenomenal results with very little resource or effort. By contrast, overly large and complex designs that relied on brute force to get the job done have always seemed to be the scourge of civilization. As the computer industry developed, I saw a number of elegant solutions emerge, packaged as lean, purpose-built objects that delivered results as no other product before it. These alternatives usually took the market by storm simply because they offered a better way to get the job done; the Return-On-Effort ratio was very high.  This is what attracted me to Nextanalytics.

In the business intelligence and reporting space, the various players have been battling it out for decades. Innovation has dwindled to feature creep as products have matured and grown bloated. One fact has held over the years – only 15% of the population actually have the desire and ability to conceptually analyse the data available. This has nothing to do with the ‘tools’ or making them easier to use; most people’s brains simply do not work that way (http://lsn.curtin.edu.au/tlf/tlf1995/whitefield.html). The big B.I. players have known this for a long time, and with many of the analytical types working in other roles, the few that actually use the BI tools to create new reports, dashboards and scorecards are in very high demand.

As we all know, high demand means high priced, so when it comes to generating new reports, those in control ‘do the math’ and decide where the effort ‘is worth it’ and where it isn’t. Because their effort is so expensive, a lot of potentially insightful reports never make the cut – if you don’t know the answer will be worthwhile, you’ll never do the work to prove it. That explains why there is a growing investment in analytics (http://www.tdwi.org/News/display.aspx?ID=9269). But is spending more the right answer?

The market has shown, time and again, that when demand for complex and expensive solutions climb steadily, something disruptive usually occurs. People don’t like spending lots of money on big, complicated things they don’t understand. I feel it is time for solutions like Nextanalytics that change the rules, change how things are done, change who can do them. It’s time to reduce the effort, reduce the investment, but still get the returns.

We’ve seen the cycle before — product life cycles have matured and the products bloat in size, complexity and price until people start to question their value. To those that have grown up with the products, this seems almost blasphemy. How could we question years of product refinement and adaptation? Don’t you remember all the lessons we learned along the way? There is NO WAY you can live without all those features!

Maybe it’s just us baby boomers getting older and less tolerant of complexity, but I am finding more and more telltales of a groundswell in search of simplicity. Being part of a software startup, we are constantly looking for new and innovative approaches (our potential competition), and I have been really surprised — there isn’t a lot of innovation in the Business Intelligence space, and it seems no one wants to simplify the status quo model of SQL databases and OLAP cubes feeding Excel spreadsheets.

First of all, Excel now sports even more features and can handle even bigger spreadsheets than ever before. That sounds great until you read about how many spreadsheets have errors and what the costs are. Then there are the increasingly scarce BI skills needed to build all the queries to feed all those spreadsheets.

Things like that make me believe we’re on to something with nextanalytics. Simple analytics, not complicated statistical analysis. A powerful script language that works on pages of data, instead of complex query statements, tedious cube construction, or a nightmare of formulas copied over thousands of cells. And it’s soooooo easy to build reports that would be a major challenge with all those other tools, and with a lot less to learn and a lot less to do. An order of magnitude faster to accomplish the task - and we’re not talking about compute speed; far less to learn, less to do, and much easier to reuse previous work.

And the whole thing is available to developers for free in a 300k download (on request), production servers get licensed for under $600. Why wouldn’t you consider it?

I read an article in DM Review today that included the question “So why are CIOs spending so much time and energy on BI?“.  It seemed like a strange question to be asking.  The article was discussing the trends in BI towards a new generation of performance management, but the question made me stop. It seemed like such a silly question.

Why do CIOs spend time and energy on BI? 

Isn’t that their job?

A looong time ago, I wrote up a one line description of what I wanted to do with my career. This was around the time “CIO” was a new idea in only very large organizations. Coming from an engineering background, I enjoyed technology. I also valued efficiency and effectiveness. I carved out a place for myself when I termed my career objective at the time to champion ‘improvements in business effectiveness and success through innovative use of computer technology.’ I found myself in I.T. because that was where it made sense for me to be.  I defined the role of the group based on my personal objectives.

To this day, I still see the I.T. organization as a middleman between the possibilities available with technology, and the challenges of the business. It is our job to continuously scan the technology landscape help the organization take advantage of opportunities where the business can gain a competitive advantage (or at least to cost-effectively not fall behind).

So why spend time and energy on BI? Because information is the lifeblood of any organization, and computer technology today offers so many possibilities for improvements. So many, in fact, that organizations hire specialists to manage it. They put those specialists in the Information Technology group, and the CIO heads up that group.

So the CIO should spend a lot of time and energy on BI. It’s their job.

I noticed a recent job posting for a Business Intelligence Analyst, and it struck me how far the BI industry is from where it needs to be.

‘This role will support business analytics for the rapidly growing business by conducting daily, weekly and monthly assessments required to support strategic level decisions by management.’

Sounds like a position that should exist in every company, doesn’t it? The way I read it, though, two things popped out — daily/weekly/monthly, and “strategic level decisions”. Those are two concepts that, when connected, expose the weaknesses of our current business intelligence technologies and have a direct impact on the ability of companies to staff this role.

First, I should explain that I believe strategic decisions - the kind that affect the way a business operates — do, in fact, happen daily, weekly and monthly. Challenges and opportunities do not wait for a quarterly strategic review meeting — they pop out of the woodwork and demand a reaction - instantly. Competitors act, you must react. Acquisitions happen, you must re-evaluate. Markets shift, you must adjust your investment plans. If nothing has become clear in the past few years, it is that your ability to react quickly determines your success in business.

Second, I also believe that information required for strategic decisions actually have a very short useful lifespan. When that unexpected event occurs, you suddenly need to know a few specific things about your own business that you likely weren’t paying much attention to. You need to evaluate scenarios that you did not consider before. You ask for reports you never needed before. But when the decisions are made, that information serves little purpose - maybe a little follow-up to make sure things turned out the way you hoped, but it is not required for operations. The reports languish.

This is real life. This is what Business Intelligence must support. These decisions, these reports, are crucial to company futures. That is why executives invest in BI. They want answers when they need them. IT groups know the scenarios all too well. Some refer to it as the report of the quarter; a high level executive suddenly demands a new report that is based on information no one has ‘cleansed’. A huge effort gets put into cleaning up the data and performing the analysis to deliver accurate information. Then they sit back and watch the report seemingly never get used and wonder why they bothered. A strategic decision was made – the most common one — to do nothing different, but those kind of decisions are invisible. 

So what does a BI solution require of the business analyst position?

‘Create Star Schema, develop cubes, KPIs and dashboards for Products Develop automated reports using SQL Server 2005 Reporting Services Design data mining structure for management to create their own reports Analyse source data and determine fields required Data mapping’

Can you believe that some people actually debate whether IT matters? I thought we were supposed to have left the techie stuff behind years ago and computers were supposed to make it easier to do the things we needed? Sure doesn’t look like anyone told the BI vendors…

It’s time for change. We need a different approach to business analytics. A simpler way.

I came across the first of a two-part article by Ned Madden today that does a great job of exposing the range of opinions about analytics and what it means to different organizations.

I particularly liked the IDC quote defining business analytics software as “solutions used to access, transform, store, analyze, model, deliver and track information to enable fact-based decision-making.” That sums up the general feelings people would associate with the words ‘business analytics’ when not encumbered by dictionary definitions. And it is that emotional attachment to the words that will elicit investments by senior managers. I have to note that the definition could have been written in 1985 and is the fundamental reason behind virtually all business investments in computer technology.

Managers want more information to better run their business, and the role of computers in maintaining and understanding daily operations continues to grow. The speeds and feeds have been producing lots of reports with even more numbers, yet managers are increasingly challenged to find the time to look at them. BI and application vendors know that most of the reports in their systems are never looked at. IT organizations know that a change in management or disappointing quarterly results means a spike in new report requests.

You can see the dynamics in the market — there is a pent-up demand for something new that will make people’s lives better. Mr. Madden has captured some of the excitement building in the industry –terms like custom fit and competitive edge are headlines, and there are lots of niches for vendors developing. I look forward to seeing part II of the Breakthrough in Analytics.

And so it goes…trying to define the word ‘analytics’. Well, maybe we can start off by agreeing that we won’t agree, as James Taylor says. But try we will.

This is predictable, and was evident more than two years ago when the term was just beginning to be picked up by mainstream marketing minds. With the aging of the “BI” term and the changing landscape in the industry, the players need new magic phrases to get a piece of the lucrative IT spending priorities (full disclosure: yes, I work for one of them and I want some of it, too). We have probably all heard that analytics are an enabler in high performing organizations, and books like Competing on Analytics are reinforcing the point in the executive ranks.

Marketing knows what that means — they may not say it outright, but their unspoken desires are building. Every competent executive is uncomfortable with what the future may bring. They need information to make decisions, and they see the word ‘analytics’ popping up in the business press. They don’t try to define it, but they start to feel positive towards it. The more they see it bandied about by others, the more they are attracted to it, and their very own marketing VP is probably talking about things she’s learned from the web analytics.

If someone, like the CIO, picks up on an opportunity to use a buzzword to fix a dwindling budget, then an investment in ‘analytics’ has been born. It could be anything, so long as you can stretch the term to fit the actual spend in some way. Now, I am not suggesting people invest in unnecessary projects just because they used some fancy buzzwords, but anyone that’s been in the IT leader position knows you need to drop those words to get over the resistance hurdle for the projects that the business does need. Once the project starts, vendors pick up on the new ‘analytics’ investment, and they use the same technique: drop the words to get over that initial resistance.

So, should we define ‘analytics’? Absolutely NOT! Leave it as wishy washy as we can get it, and leave the door open to interpretation. Everyone wins! The CEO gets to brag about his new investment in analytics, the CIO gets his project budget approved, the vendors….we get past the gatekeepers to talk to the people that matter.

[oh, and NextAnalytics actually does sell a product that does ‘analytics’ :-)]

As a long-time IT exec, I’ve seen the studies, listened to the sales pitches, questioned the case studies, and lived through the hype cycles. In organizations, there is a perceivable cycle that repeats itself -the current strategy is getting long in the tooth, so management searches for the next ‘thing’.

Someone in marketing recognizes a festering annoyance in a few leading organziations and coins a new phrase that others start to rally around. They know that management spends money to deal with festering annoyances, but they hate pouring money into the same thing, year after year after year. That is why it is so important for marketing to supply their new phrase with some nice sounding “differentiation”.

I remember the need for “reporting systems”, the fancier “decision support systems”, robust “data warehouses”, the more agile “data marts”, prestigious “executive information systems”, the all-encompassing “business intelligence” solutions. All of these were supposed to give management a better understanding of the business while dangling the elusive carrot of competitive advantage.

So that festering annoyance put business intelligence applications at the top of Gartner’s technology priority list for the third year in a row. Management is still not happy with the information they have available to them. They think they can get better reports, and will soon get tired of investing in the same old thing. The BI industry is in a really bad state to respond. The Big BI Consolidation has distracted all the wrong people, and there hasn’t been a lot of innovation to hook a new marketing phrase to.

We all know that the lack of a product would never stop a good marketing group, so is it any surprise that, after Google Analytics helped them understand their web site usage, they are identifying a bigger requirement for ‘business analytics’?

Business Intelligence is dead (as a phrase). Marketing is killing it. Long live business analytics.

Well, three years at least…