Monthly Archives: October 2011

Big data – the professional and the amateur

Posted on by
Reply

In my last post on big data I hypothesised that really we are just beginning to understand some of the implications.

Within this evolving ecosystem we have an interesting tension developing between the role of the professional and a community of amateurs. Three models of engagement seem to be emerging between the two communities, all of which deliver value, regardless of any apparent conflict:

  1. Where there is an immense volume of data, how does a professional get to the core data they need as quickly as possible? Possibly by setting a simple classification process and using crowd sourcing to triage data in a similar manner to the Galaxy Zoo experiments. Effectively the professional can work on the needle once the haystack has been removed.
  2. In the second model, a community of interest is created. An example of this is Patients Like Me – a data-oriented social network bringing together people with a shared interest (in this case chronic illness). They share how they feel, their condition, the symptoms they are exhibiting and the treatments they are receiving. What’s interesting is when the community of interest can support and in some cases challenge the opinion of the professionals.
  3. Model three is concerned with gamification, in that it encourages loyalty, participation and a particular behaviour through rewards. In reality it’s rewarding people or making a game of something that most of us would consider a chore. Another post examined the concept of gamification in the workplace and in another example, the Search for Extra-Terrestrial Intelligence (SETI) is looking to add game-play mechanics to continue its quest (which I’m sure we all did ten years ago, but got bored with it when ET did not come to visit!).

These three models offer a means to harness society and resolve real problems for real people in a timely and collaborative way. The non professionals amongst us contribute through triage to allow the professionals to operate upon the nugget that makes a difference.

So what am I trying to say here is that social networks are not only a means of creating massive data sets but can also be used to extract value for the common good. The interest may be as part of a hobby, a personal interest in the subject, or participants might just be looking to earn some rewards, but engaging amateurs and professionals in a collaborative manner could lead to an exponential growth in ideas, reduced time to market for products, validation of products and services (and the list goes on), all because “we did our bit” on the big data mountain.

Big data – getting on the front foot

Posted on by
2

With cloud now part of everyday language, the next big thing is Big Data. Essentially it is the recognition that the digital world is generating increasing volumes of data (according to Cisco, humans created more data in 2009 than in all previous years combined), most of which no one is doing anything with, except storing it. The challenge articulated by the big data concept is effective mining and analysis the data to create value and wealth. By way of an example, The Big Mac Index brings together a set of data that can give an indication of the relative wealth of a country but how and when it is applied is the key.

Titling this post “Big Data – getting on the front foot” refers to a balance with human intuition; we often make a decision based on a small set of knowledge and information only to be second-guessed later with facts and figures that indicate whether our decision was correct (or not). For me the execution of big data is to put the right information, data and knowledge in to the hands of the decision maker at the point they need it, not at some point post-decision. What does this mean for you and me? Well, healthcare professionals, retailers, financial services providers, government or just about anyone that we interact with in a social or business context will have immense amounts of information about us and our relative positions in teams of health, wealth, buying habits, risk for insurance purposes etc. – let’s hope that the decisions they make, based on that data, are the correct ones!

Fujitsu’s vision of a Human Centric Intelligent Society highlights all the positive aspects of this digital society with the “Internet of Things” playing a pervasive role. But is the World going to be so different as a result or will it spin just a bit faster? If we take our health and well being as an example, there is a logical chain events that lead to a general improvement. By using a simple logical sequence of mapping the human genome, understanding the variation from what is expected, how we live and the environment we live in, we can potentially be offered very precise and evidence-based advice on how to avoid certain illnesses. Add the ability to model potential drugs in the digital world against the human gnome including demographic variances and the potential outcome has a huge value to society. The research and development costs of drugs drop considerably as potential failures are weeded out very early in the development cycle and, using big data, a doctor can map the best drug to a condition you have based on your gnome.

It all sounds great but there are some challenges along the way:

  • McKinsey indicate that big data will bring lots of new jobs; however it’s my hypothesis that these are really the same jobs carried out differently.
  • Some of the bastions of our society (particularly in the west) will need to change. For example, insurance companies will need to take a different view on their risk-based business model (otherwise we will all be uninsurable!).
  • We’ll need to take a different approach to security too – look at how the “Facebook generation” views sharing and what they care about.

In short – we will all have to behave differently in the world of Big Data. After all, it’s not just a big social network where everyone is your friend!