Analyzing data from social media sites such as Twitter to assess market sentiment may well be the next big thing in financial technology.

This scraping of social media and news sites to give traders another tool in their bid to successfully predict the direction of markets is beginning to take off in the finance world.

One London-based technology start-up called Knowsis, which is ready to go live, will offer firms a data feed service that will be installed on traders’ desktops providing real-time analysis of market conditions, asset classes and specific securities.

Twitter users alone generate 250 million new Tweets each day, so there is a lot of ‘big data’ to be processed before traders can start to make sense of it all.

“The finance world has started to take notice of social media as a powerful platform for breaking news and views,” Oli Freeling-Wilkinson, chief executive of Knowsis, told Markets Media.

“The trouble from a finance perspective is that social media has a lot of problems. It is very noisy and unstructured—most of what is on Twitter is rubbish—so it is really hard to keep track of the multiple streams out there. It is really hard to get value out of it efficiently.

“There are a lot of generic players out there starting to look at the space but nobody was offering a finance-focused solution that was accessible and affordable to traders and the investment community. What differentiates us is that our systems were built from the ground up with input from traders and investors.”

Freeling-Wilkinson says that Knowsis, which mainly monitors a combination of Twitter, RSS feeds and news sites, will identify things that make a difference to traders.

“Our testing has had really positive results,” said Freeling-Wilkinson. “We can really help people beat the markets and at the very least we provide market color.

“We aim to quantify the mood of the crowd. We can’t predict really what is going to happen with key data points. You might get an insight to what the crowd thinks and use it to your advantage but you will never really know until that data point comes out. We tend to track the reaction to news, which is really key to understanding the underlying trends from a behavioral finance view.”

An example of this would be if the U.S. released some downbeat news on the economy.

“If there was some bad economic news coming out of the U.S.—and the market begins to sell off—should you continue to sell off or will the market buy into that bad news because it just doesn’t care,” said Freeling-Wilkinson. “That is the phenomenon we track.

“The great thing with social media is that it gives a great insight into what story matters. That is really key; it is a market color tool. For example, if you have read a story about the iPad being terrible, what you don’t realize is that everyone else may be reading a story that is positive about it and that everyone really likes it. And that is what we can tell you—that there is another story out there that everyone really cares about.”

In 2011, the first dedicated Twitter hedge fund, run by London-based Derwent Capital, shut down after just a month of trading despite returning 1.86% during the time the fund was open. It closed after deciding to offer its social media indicators to traders instead.

Other firms, such as hedge funds and high-frequency traders, are now beginning to see real value to this social media data.

A recent White Paper written by U.S. social media data provider Gnip, which supplies hedge fund clients with an aggregate data feed of public social media posts in a bid to better understand public moods and capture breaking news, extolled the virtues of this new technology.

“For the first time in history, access to the observations, wisdom and emotional reactions of millions of people globally is available in real time,” said the report, entitled ‘Social Media and Markets: The New Frontier’.

“Social media data represents a collective barometer of thoughts and ideas touching every aspect of the world. As social platforms increasingly become a primary means of communication for our age, asset managers, equity analysts and high frequency traders are incorporating leading indicator data from these platforms into investment decisions as a means to create alpha.”

The ability to understand the forces shaping the market—and allowing one to predict direction—has always been an elusive skill but Twitter may well hold the answers.

Gnip highlighted the death of Osama bin Laden in May 2011. DataMinr, a social media monitoring tool, was able to spot this with just 19 Tweets on the subject. The company then issued a signal to their clients, alerting them to this important piece of information, but it was over 20 minutes before the story appeared on traditional news sites.

“Not everyone wants or needs to use social media data as part of the investment process,” said the Gnip report. “But as social media garners greater mind share, accessing Twitter data on a company may be as common as checking a stock quote.

“The challenge of incorporating social media into traditional research and review strategies is rapidly being overcome as new products and services enter the marketplace. Whether it is aggregating high volume, low-latency new media, sifting through social data across many sites, or providing real-time data streams for trend analysis, hedge funds are driving innovation as service providers seek to give them access to leading indicator data that will help them create alpha for their investors.”