Capital markets firms are extending their use of high-performance database technology to harness the power of ‘big data’.

“Our Wall Street customers have instances of kdb+, and its built-in time-series programming language, q, throughout their companies,” Abby Gruen, director of business development at Kx Systems, told Markets Media. “It is being used to aggregate and analyze data, perform statistical functions, and join data sets in more ways than are possible using SQL technology.”

Kx Systems is anticipating a new release of its kdb+ database later this year. Kdb+ is used widely in the financial services sector for applications such as automated trading monitoring, pre-trade analysis and algorithmic trading analytics engines.

In addition to capturing market data, firms are using kdb+ for order-book management, algorithmic trading, and risk assessment. “They are using kdb+/q for queries being performed on both streaming or historical data — the latter easily accommodating research and back-testing,” Gruen said.

DEVnet, a technology services and solutions consultancy in financial trading and risk management, has released an open source version of its proprietary suite of building blocks for kdb+, making it easier for firms to develop applications built on kdb+.

“DEVnet has extensive experience in financial services consulting,” Gruen said. “They complement Kx Systems and do sales, training and consulting in kdb+.”

DEVnet, a strategic partner of Palo Alto, Ca.-based Kx Systems, has been developing these software components over several years and has deployed them in trading environments across Europe, North America, Asia and Australia.

By helping companies build applications on kdb+, DevNET is making it easier for firms to leverage the strength of the kdb+ core technology to support their businesses, according to Robin Mess, director of sales at DEVnet.

Over the past 15 months, hardware vendors like Intel and IBM have deployed kdb+ for running the well-known STAC M-3 historical tick database benchmark at the Securities Technology Analysis Center, said Gruen.

DeltaFlow, a platform from First Derivatives that’s based on kdb+, is used by traders for high volume, low-latency algorithmic trading and by regulators for real-time detection of market abuse and unauthorized trading activity across multiple asset classes.

Other recent partners, in addition to DevNET, First Derivatives, and Bedarra Research Labs, include AquaQ Analytics and QuantumKDB.

“We released our smaller memory space version of kdb+ (32-bit) free for commercial use on April 1, 2014, and two of our partners released free tools at the same time,” said Gruen. “DevNET made its kdb+ Exxeleron framework with fast interfaces free, and AquaQ released its TorQ framework for free.”

Featured image via iStock