As seen in Global Corporate Venturing, July/August 2015 issue.
From London to Boston to Silicon Valley, the message is consistent—we are finally at the tipping point of realising the value of big data. Analytics coupled with machine learning will help both companies and individuals to derive valuable insights from ever-growing data sources, which will ultimately translate into better decision-making and greater value creation.
As costs of data infrastructure – transmission, storage and computing – dramatically drop and analytical tools proliferate, the innovations of big data will be game-changing.
We are already seeing remarkable developments. At a San Francisco event showcasing machine learning, I was amazed to learn how Orbital Insight, a Silicon Valley Bank (SVB) client founded by artificial intelligence expert Jimi Crawford, is using images collected by nano-satellites to help track the world's oil supply. As oil tanks are drained, shadows cast by their sinking lids change, allowing experts to estimate how much oil remains in storage tanks around the world. Other company projects include analysing photo imagery to measure construction in China, grain production in Russia and the number of cars parked at big-box (mega-store) retailers.
In May, my colleagues at SVB Analytics, a non-bank affiliate of SVB, presented their vision of what's ahead at the SVB Next: Big Data event in Boston. Data is growing faster than ever, and the means for storing and processing these huge quantities of data are becoming less expensive. These two trends are driving new and more powerful use cases for big data.
Finally, new analytics applications help to derive insight from the vast new quantities of data, driving real-world value for business users across many industries. These tools ultimately help companies to know their customers better, bolster security and improve operational efficiency.
Steve has created the SVB Big Data Maturation Index, which is designed to pinpoint which industries have adopted big data more quickly than others. Several factors impact the pace of adoption, including ease of data capture, level of regulatory oversight, and level of technology enablement. Interestingly, the report finds several of the industries with the largest market sizes—financial services, healthcare, and energy—are relatively underdeveloped when it comes to big data, which indicates significant potential in these markets.
In the coming iteration of big data, Steve says the highest-value input will be the data itself. As data infrastructure, management and analytics tools become commoditised and more commonly adopted, building a proprietary data set will become the most important factor in maintaining a sustainable competitive advantage.
At the Boston big data event, we heard lots of sage advice to consider as we enter this brave new world of zettabytes.
Several presenters working at corporations or with corporate partners recommended starting with a clear long-term data strategy, including identifying a hypothesis, a problem to solve or customer demand to create. And then, they advised, do not be afraid to go against preconceived notions if the data disproves that hypothesis.
Cracking the code in this environment means unlocking the key to how humans make decisions. Machines are growing in sophistication to augment and supplant current data analysis techniques and the "automated hypothesis" is now at our doorstep. The decade of big data disruption has arrived.