Data has been the driving force behind investment success since the Buttonwood agreement was signed in 1792 near a wall in southern Manhattan that had originally been constructed to keep out pirates (and the British). Paul Reuter, founder of the eponymous Reuters, started his data career using pigeons to transmit stock prices before switching to using early telegraph technology to increase speeds. By 1870 this technology led to the ticker tape, which ruled the world of equity price transmission for the next 100 years. In the meantime, companies began publishing more detail around their operations, and the analyst and investor communities craved more data to drive their insights and investment decisions.

Michael Bloomberg stepped into this gap in 1981 by founding Innovative Market Systems, which after finding success he renamed after himself. Bloomberg provided real-time data about market transactions, world events, and anything else that they thought might inform a trade. They along with Reuters, who followed them into this technology space having long since abandoned the pigeons, were the driving forces of information-based trading in the 1980’s and 90’s.

Fast forward to today. Powerful computing, cheap data storage and the rise of the information age has resulted in a glut of data. No longer is it enough to trade based on traditional market data. To find an information advantage and alpha edge over your competitors today, one must mine the vast ocean of ‘alternative’ data. Leveraging this landscape of data is quite difficult though as the volume of data is so great, and identifying the sources that will inform a given trading strategy is not straight forward. The ability to navigate this new and evolving world of data will be one of the key differentiators in investment success for years to come.

Although the U.S. alternative data market is expected to reach $1.7B this year and more than half of hedge fund managers are now using alt data, 77% of market leaders find that backtesting of these unstructured data sets still poses a big challenge. The universe of alternative data is expanding so quickly, with more and more marketplaces wanting to capitalize on this growing industry. However, no one is really addressing the current market challenges from a data consumer point of view. That is, creating the necessary teams and internal data capability, along with deploying the relevant infrastructure to be able to completely utilize and understand the full value of alternative data through analysis driven alpha.

With one marketplace estimating there to be 5,000 alternative data sets globally by 2024, it is therefore not surprising that sourcing quality datasets is the second biggest challenge for user groups. In particular, hedge fund managers find it difficult to know which data sets to look at, being able to focus on what matters in accordance with their proprietary systems and algorithms, whilst also keeping an open-mind for new opportunities. Even after identifying what data sets to analyse, there are a number of other considerations before using the data including, data governance checks, the breadth and coverage, how robust the data source is – in terms of completeness, consistency and delivery – the longevity of the dataset, and how flexible that data is in terms of mapping to fixed references etc. The good news for us is that investment teams are dedicating more of their time towards alt data, with nearly the majority of them allocating 10-20% of their capacity towards this. And Data Engineers and Data Scientists also remain the most sought after hires for the late adopters of alternative data.

We have learned from operating in the space, that hedge fund managers that are receiving, storing, and consuming alternative data requires a nimble mindset and often a different approach than what is required when dealing with traditional data. Particularly having to navigate the regulatory environment that governs the collection, usage and distribution of data, which becomes nuanced with alternative data. At Alqami we have seen an opportunity to help solve for this by developing the technology that will create a systematic process to better identify, evaluate and ingest only the highest quality data sources. Turning a human evaluation and selection process into an automated method. Our concept for a solution has been a big focus point for us this year, using the Covid downturn to spend more time on understanding our buy-side clients challenges and innovating to take advantage of these opportunities and support a more widespread adoption of alternative data with broader user groups within the investment management industry. Alqami is a key driving force for the maturation of the alternative data market, and in time, it will easier and more timely to utilise these unique data sources for trading insight.