As new regulations such as Dodd-Frank, Basel III and Solvency II are implemented, Big Data demands are being placed on financial firms to track the source of data, how it has changed over time, and who has changed it.

The ability to pull data in near real-time from a wide variety of sources to calculate exposure reports and feed risk algorithms is especially important as plans toward centrally cleared OTC markets continue.

Regulatory changes are forcing firms to source and report increasingly large amounts of trade data, as well as to adopt higher-quality—and usually data hungry—risk and pricing models, according to a report by Interactive Data Corp.

“The applications that we service are consuming more data of a wider variety in order to satisfy stringent reporting and regulatory requirements,” said Marc Alvarez, senior director, reference data infrastructure at Interactive Data.

The Dodd-Frank Act contains major reforms to the derivatives market, including requiring that standardized, or vanilla, OTC swaps be executed on a swap execution facility or exchange, and be cleared through a central counterparty clearing house.

Other jurisdictions, such as Europe with the Markets in Financial Instruments Directive (MiFID), Markets in Financial Instruments Regulation (MiFIR) and European Market Infrastructure Regulation (EMIR), have or plan to establish similar requirements.

The uptick of cleared OTC transactions is part of a growing trend of increasing volumes of data that are enveloping the capital markets.

“There are large volumes of OTC derivatives that will need to be cleared at a CCP,” said Geir Reigstad, head of commodities at Nasdaq OMX. “This means that the clearing and risk management systems need to be able to handle these volumes.”

Nearly 80 per cent of buy-side participants recently polled by MoneyMate said that automating their firms’ data management processes were very important, and over 60% cited regulations as a key driver.

In a similar poll a year earlier, 80% of respondents said they were still unprepared for regulatory changes. In the U.S., 75% of firms rated Dodd-Frank as a major concern.

“Last year, respondents were unsure how they were going to have to make changes and prepare to comply with regulation as they were unsure as to what exactly the requirements would be,” said Ronan Brennan, chief technology officer at MoneyMate, a provider of data management software. “It appears that this year, they are getting ready and regulation is a key driver for change.”

Dodd-Frank’s requirements for central clearing of OTC derivatives as well as for credit risk management has challenged fund managers struggling to achieve compliance.

”This is primarily due to the current technology landscape, where disparate systems across the front-to-back office make it an uphill battle to calculate exposure, automate and optimize collateral allocations, and automate position valuation and generation of variation margin,” said David Kubersky, managing director of SimCorp North America, a provider of investment management software and services.

The greatest challenge for risk management is the ability to view exposure across the firm’s entire book of business. In a recent SimCorp poll, 30% of buy-side firms stated that it would take them days or weeks to calculate their exposure across all holdings.

“What this means is that in a distressed situation such as in the collapse of Lehman Brothers, 30% of buy-side firms would not have a timely view into their exposure to the distressed party, leaving them unable to react quickly,” said Kubersky.

Regulators have become increasingly focused on data as they seek to transform OTC derivatives toward and exchange-traded model.

For example, they’re pushing for a system of legal entity identifiers (LEIs), by which counterparties to transactions would be tagged.

The Committee on Payment and Settlement Systems (CPSS) and the International Organization of Securities Commissions (IOSCO) have issued recommendations on minimum data reporting requirements, access to data by regulators, LEI development, and development of an international product classification system for OTC derivatives.

The International Swaps and Derivatives Association has advocated the use of a single “counterparty exposure repository” to provide for an aggregated risk view for regulators.

“We are seeing a number of different proposals and actual rulemaking mandates that will require the use of counterparty information for OTC derivatives reporting,” said Robin Doyle, senior vice-president at JPMorgan Chase, during a recent webinar. “There is a heightened awareness of some of the issues around viewing counterparty risk.”

The CPSS/IOSCO recommendations address the Financial Stability Board’s call for minimum data reporting requirements and standardized formats, and for a methodology and mechanism for aggregation of data on a global basis. The FSB is a working group led by representatives of CPSS and IOSCO that’s charged with implementing the G20 reforms for OTC derivatives.

The issue of gaining a view of counterparty risk is pervasive across the capital markets industry, and is central to the Big Data discussion.

Ironically, while regulations are one of the main drivers behind Big Data, the regulators themselves have shown little inclination to employ Big Data techniques in their own work.

“On the one hand, regulations require great amounts of information that requires Big Data methods,” said Ran Fuchs, head of product development at Citco Transparency Platform, during a recent webinar. “On the other hand, regulators keep working as if nothing has changed. Regulators stand to make the most use of Big Data, methodologies because Big Data is the only way to measure systemic risk. So far, they haven’t done anything in this direction.”