Enterprise Communications

Will Abundant Data and Advanced Analytics Kill Profit Margins?

by Elka Popova 24 Sep 2012
Share this:

Some of the most exciting technology topics today include social search, data mining, and, in general, big-data analytics. These technology trends can determine some very personal decisions (e.g., which restaurant to go to), but can also have a very powerful impact on broader investor decisions and financial markets. This article on Facebook and friend mining made me ponder this issue again.

Online content, Web search, e-commerce, and social networking produce and retain tons of data that can be processed and analyzed and used for various purposes. Some of the data are already being harnessed for targeted marketing and advertizing purposes, but the potential is much bigger. The main reason why the potential has not been fully exploited so far is the lack of analytical tools that are sophisticated and powerful enough to uncover all the hidden gems of useful information.

For example, it must be simple enough to run a search of most liked restaurants among your Facebook friends and come up with a single favorite among all. But how about running more complex algorithms and deciding if you should travel a few miles to get to a place less poplar in terms of total votes but more popular based on some kind of a weighted average? For example, wouldn't it be wiser if you based your decision on the percentage of “likes” among your friends living in each particular location or having visited certain locations (e.g., weighing the data by a new criterion)? In other words, if 5 out of your 15 friends (30%) living in San Francisco voted for a Thai place in the city, does that make it a better choice than a Thai restaurant in Menlo Park that 3 out of your 3 friends (100%) living in Menlo Park or having visited Menlo Park liked and favored?

My example may be not be very scientific, but it just illustrates the value of designing the right algorithms when running any kind of search or performing data analysis. At the end of the day, humans design the processes in complex data analytics, and the analysis is only as good as the variables and correlations selected by these individuals. 

Overall, there is a huge potential—for technology vendors, applications developers, and users—in data mining and analytics. But there are also multiple concerns related to data sharing and data usage by third parties. For example, social networking users are becoming increasingly concerned about privacy and are posting fake names, birth dates, and other facts about themselves to try and hide from prying eyes. Therefore, it will become increasingly challenging for data analytics to sort through the real and the fake pieces of information.

A comment someone posted on Twitter related to the Facebook article above made me think about yet another implication of over-connectedness and efficient data sharing. That person asked “Doesn’t this make our world smaller and smaller?” I believe it does, in many ways.

It is becoming easier and faster for information to cross this increasingly “shrinking” world. Which begs another interesting question: how will abundant, pervasive, and rapidly disseminated information impact profit margins? According to common economic theories, in a perfect market, buyers and sellers have equal and prompt access to market information, which creates a kind of market equilibrium and eliminates the possibility for excessive profits. The truth is, perfect markets do not exist and that has created ample opportunities for profit creation. But are we getting dangeriously closer to that state of almost-perfect access to information thanks to continued advancements in communications and information technologies and data analytics? Because it is important to remember that profit could be a dirty word to the under-privileged, but, in capitalist markets, profits (or the promise thereof) drive competition and innovation, which in turn help raise the standard of living.

At Frost & Sullivan’s GIL event just a few weeks ago, there was a very interesting panel of the so-called Legends of Silicon Valley. This year, our guests included Bill Davidow, Judith L. Estrin, Kim Polese, and T. J. Rogers—amazing individuals who have had a tremendous impact on Silicon Valley and technology development. One of the questions Mike Malone, the panel moderator, asked was about innovation in Silicon Valley (and beyond) and the future potential for a competitive advantage and profits.

Opinions varied and various arguments were presented in favor of each point of view. But my takeaway was that, even though it is becoming harder to experience disruptive innovation on a large scale resulting in copious profits (which could describe the state of Silicon Valley in the 1990s), there will always exist pockets of innovation and opportunities for a competitive advantage and money-making. What will drive progress is the ability to identify niches of unaddressed market needs and react nimbly and efficiently by developing products and solutions that address these needs.

Technologies may be developing at a lightning speed and information may be travelling even faster than that. But there is always the human factor. Individual investors can only process so much information without becoming overwhelmed. And even prompt access to all the information in the world does not always translate into wise investment decisions. Therefore, while I acknowledge the disruptive potential of advanced data mining and analytics tools, I would still bet my money on an imperfect market where information will never be equally accessible to everyone and hence there will always be opportunities for innovation and differentiation, and, fortunately or unfortunately, for financial speculation as well.

Login or register to make a comment on this blog post

Help Desk

Full list of offices

For more information and general enquiries, contact Frost & Sullivan near you.

North America
tel: +1.877.463.7678

Select a location near you..