As market data output continues to grow exponentially, market participants are faced with the challenge of establishing and maintaining large historical databases.

“Storing and having to churn through terabytes of data in order to make a trading decision or develop a trading strategy requires significant storage, bandwidth and processing power,” said Brian Fuller, business development manager, Global Market Data, at NYSE Technologies, the technology division of stock exchange operator NYSE Euronext. “Terabytes upon terabytes of data needs to be supported.”

NYSE Technologies is addressing this challenge via its newly-launched Market Data Analytics Lab, which provides access to a hosted data warehouse comprising a portfolio of historical and reference market data.

The Market Data Analytics Lab is a centralized, hosted database granting access to NYSE Technologies’ historical TAQ (Trades and Quotes) and reference data sets, along with corresponding corporate actions data, with a complex range of analytical and quantitative capabilities.

“Customers can use the Lab to assist in research, analytics and trading decisions via a complex range of mathematical and statistical functions,” said Fuller. “Analysis can easily be carried out on large datasets at extremely fast speeds.”

Query, tabulation and cross tabulation tools enable complex analysis across multiple datasets, including statistical modeling, functional grouping, multiple regression analysis and time-series analysis, he said.

Although pre-built complex analytics are not available, these functions can easily be created using the common functions.

NYSE Technologies provides market data, including real-time, intraday, historical and reference data for all of NYSE Euronext’s U.S. and European equity, options, futures and fixed income trading venues, as well as many away markets. Prior to Market Data Analytics Lab, that data had to be downloaded in bulk.

“One of our marquee historical data products is Daily TAQ, which provides trades, quotes and NBBO [national best bid and offer] for all 13 U.S. equities markets,” said Fuller. “That’s many gigabytes of data, and upwards of hundreds of millions of rows of data, every day.”

Exchanges are repackaging the information they produce as standalone products to generate revenues.

“Exchange see derived data as a value-added service that can be monetized to its full effect both in terms of the tradable products and associated information,” said Hirander Misra, chairman of Forum Trading Solutions, which offers security matching and surveillance services..

Under Market Data Analytics Lab, all TAQ data for a region is collected and normalized in a single ticker plant and all hardware within the plant is synchronized.

“This ensures that clients work with realistic data as a production ticker plant is receiving it,” said Fuller. “This is especially useful for testing trading strategies across multiple market centers, across similar products within a market, or calendar spreading strategies across maturities within a product.”

Once customers sign up for Market Data Analytics Lab, they will have access to the TAQ and reference data sets to which they’ve subscribed or for which they have previously purchased bulk history. Customers must purchase the relevant data sets prior to purchasing access to the Market Data Analytics Lab, but can purchase additional data sets at any point during their Lab subscription.