A Journey Through the Evolution of Buy-Side Investment Data Management

Marc Rubenfeld, CIPM, Head of Eagle Solutions EMEA/APAC, Eagle Investment Systems 

Originally appeared on Eagle’s blog website here

At the recent TSAM London event, I had the honour of chairing the data management conference stream. This involved moderating debates, introducing speakers and facilitating conversation throughout the day. In preparation for the event, I began thinking about the history of data management and what I have seen as the stages in its evolution within buy-side investment managers.

Each new innovation or concept in data management has followed a similar evolutionary path, starting with awareness or recognition around the concept itself. Then follows a ripple of early adopters that look to build their own solutions, before vendors step in to refine and improve the innovation with commercial solutions. Over time, these vendor solutions can offer even more cost savings in the form of a managed service.

Over time, with each new innovation, the period it takes to evolve has contracted as firms have become quicker to embrace new concepts and vendors more agile in reacting to client needs. To illustrate this, I put together the diagram below, based on the design of the London Underground map. The horizontal line roughly approximates time and each evolutionary stage is represented as a station.

Evolution of the Data Warehouse

Take the first revolutionary data management concept: the introduction of the data warehouse. Data warehouses were first discussed decades ago but those who have worked in the industry for many years will recall that people who had heard of the technology were asking questions well before any physical developments emerged. This only materialised once it was recognised that there was value in owning, managing and controlling a copy of data from the administrators and data vendors.

Some of these early adopters went on to build the technology in-house, which accelerated adoption once the broader industry recognised how the technology was being leveraged. However, the approach raised a number of issues, chief among those being the “key man” risk as only certain individuals understood how that information was being stored. These early, customised solutions also tended to lack flexibility. When the need arose to add or adjust the warehouse to accommodate new products or investments, it necessitated bespoke work, which in turn created bottlenecks.

This is where Eagle spotted an opportunity early on to supply standardised data warehouses that eliminated “key man” risks, offered more flexible architectures and provided economies of scale. The added benefit for our clients was that it allowed them to introduce new products far more quickly.

The industry has now evolved to a point where data warehousing is a service. As asset managers focus on their core competencies they are increasingly looking to offload the day-to-day operational aspects of running their data warehouse to third parties. Indeed, managed services is one of the fastest growing areas of our business.

Data Mastering and Data Marts

Subsequent innovations such as data mastering and data marts followed the same progression. Data mastering emerged in response to the need to create “gold copy” records that the organisation could rely on to support business processes. Data integration and validation capabilities emerged as solutions built onto data warehouses and evolved to encompass processes that would ensure data was fit-for-purpose and available within a data hub or warehouse. Today, most organisations do not want to own the process, as it may not be a core competency, yet it remains a critical service that enables valid data for the organisation.

Data marts emerged as firms needed to have subject specific or localised copies of data to support data consumption or specific business processes.  Today, marts can be populated as a service with changes made as business needs arise.

In each of these cases, the progression from original concept to the solution as a service occurred in a shorter space of time.

Data Governance and Data Lakes

Looking at two still fledgling areas of data management today, the same pattern is starting to emerge with data governance and data lakes. With data governance, the period in which internal builds took place and process archetypes were established was very short. Asset managers didn’t really spend time building their own data governance solutions because vendors quickly recognised the need and in many cases expanded their offerings.

Today, data governance is not yet readily available as a standalone managed service, but based on the course of previous innovations, nobody should be surprised if it happens fairly soon.

Data lakes, on the other hand, are still in their infancy. These store large amounts of raw structured and unstructured data to provide new insights, particularly for the front-office. As the concept of big data emerged, firms with visions of competitive advantage and the threat of disruptive challengers started to experiment. Today’s vendors are now beginning to design solutions around data lakes and if history is a guide, will probably follow the same path of offering a product and then a managed service shortly thereafter.

To date, data management practices in the asset management industry have progressed towards managed services and outsourcing solutions. This march is being driven by the desire of firms to focus on their core competencies. If there is no competitive advantage to building or running solutions in-house, they quickly realise the efficiency gains to be had by leveraging a partner. There’s no reason to believe that this trend will change with the latest innovations; indeed, a recent research study we sponsored would seem to bear this out. The survey, which focused on business transformation initiatives, revealed that nearly half (47%) are considering the use of managed services to simplify their operations and eliminate certain manual processes related to data management, while more than one in ten were considering fully outsourcing their data management needs.

With each new innovation in data management, the evolution from concept to solution as a service is happening faster and firms today are quicker to recognise the benefits of handing off responsibilities to third parties. In turn, this will only serve to accelerate the pace of innovation, as new advances are refined and institutionalised more immediately to drive efficiencies.