Big data, the term given to the incredibly large sets of data that exist all around us but which can only be analysed through complex algorithms, is being applied to everything from marketing to healthcare, sports and investing.
Computer-driven analyses of big data can produce what would be near impossible when done manually, meaning patterns can be extracted from even the most obscure sources.
In the investing world, a swathe of new sources of information are now being used to make trading decisions. This reflects a shift from analyses of structured data only, to also incorporating unstructured data.
Big data is being used to predict sales figures based on web traffic to retailers’ sites, as well as app downloads, for example. Social media websites like Twitter are being mined for investment trends, events and opportunities. While big data can reveal patterns among millions of tweets, single tweets from the right people can now result in instantaneous, automated trades – with the aim of beating manual traders to it.
The pool of data available on the internet is massive and rapidly growing. Investment banks are widely believed to be slightly behind the curve in their adoption and application of big data, meaning the potential applications are substantial.
Demand for big data specialists is growing among investment firms, which are having to compete with other sectors and industries for this relatively niche skill.
Online retailers in particular were among the frontrunners in embracing big data analytics, using algorithms to identify and leverage consumer spending habits. Elsewhere, big data is being harvested by state security departments, sports teams and healthcare providers, among numerous other sectors.