By Theo Hildyard , Software AG
@theohildyard

In the third of a blog series outlining the Seven Pillars of Market Surveillance, we investigate Pillar #3 which shows us how we can analyze the fire hose of market, trade and social media data to prevent fraud and market manipulation.

Called Market Surveillance 2.0, the next generation of market surveillance will act as a kind of crystal ball; one that is able to look into the future to see the early warning signs of unwanted behaviors and alert managers or trigger business rules to ward off crises. By spotting the patterns that could lead to fraud, market abuse or technical error, we may be able to prevent a repeat of the recent financial markets scandals, such as the Libor fixing and manipulation of the Foreign Exchange benchmark.  

My last blog outlining Pillar #2 in the Seven Pillars of Market Surveillance 2.0 explained how we can enable banks and regulators to identify anomalous behaviors before they impact the market by using a combination of historical, real-time and predictive analysis tools to achieve this capability.

As we continue to look into the foundations of the next-generation of market surveillance and risk systems, I’d like to explore Pillar #3 – support for fast Big Data. Fast Big Data comes at high velocity from the fire hose of market and trade data and, increasingly, from social media, emails, instant messages, news headlines, audio data from phone calls, or video data from surveillance activities.

To be able to monitor and make sense of fast Big Data you must connect to multiple, disparate, live data sources. Some of these sources may be internal to your firm; some may live in the cloud or be streaming from sensors. You must be able to ingest and digest all of these data streams, and able to handle the peaks and troughs of the velocity while detecting actionable patterns.

For example, the perpetrators of the London 4pm FX benchmark manipulation scandal might have been caught a lot sooner (it went on for at least four years) had regulators and banks been using fast Big Data analysis.

The relationship between the banks’ traders, and possibly even their corporate clients, was known as “The Cartel.” They used a private chat room, as well as industry forums, emails, instant messaging and mobile phones to communicate and trade.

By connecting to the disparate sources of information coming from their trading relationships – IMs, chat rooms, emails, mobile phones, video and audio surveillance – we could have drawn a picture of trader behavior. For example, if a trader stopped using an industry FX chat room suddenly, how was he communicating with his clients or counterparties? Was there an unusually high volume traded on a specific currency with a particular bank or client after that?

Did that currency rise after any specific global event and/or news headline? Did other markets react to the rise? Or was it an isolated event that happened only at and around the 4pm benchmark fixing time?

And by tapping into internal data such as human resources, middle and back office and entry card data – we could add more color to the picture. Was the trader working unusual hours? Was he never taking holidays? Did he buy a fishery together with another FX trader who happened to be his banks’ client?

The tangled web that fraudsters weave becomes much more predictable when we can watch the threads creating the web. Fast Big Data comprises the threads; Market Surveillance 2.0 takes them and analyzes them for patterns then kicks out actions we can take.

By combining Pillar #1, a convergent threat system, Pillar #2, a combination of historical, real-time and predictive analysis tools, and Pillar #3, support for Fast Big Data, we can go a long way toward creating our Market Surveillance 2.0 crystal ball.