04.10.2012

Big Data Moves to the Cloud

04.10.2012
Terry Flanagan

New computing paradigm offers limitless potential.

The cloud computing paradigm is being applied to capital markets, changing the economics for storing and processing Big Data.

The goal is to lift the entire high-cost infrastructure into the cloud, where hardware and software resources, apps, and market data can be distributed, shared, and leveraged.

Cloud computing“We are seeing universal trends in terms of technological advancements that are allowing structuring and analysis of large volumes of data in real-time,” said Rutger ter Hoeven, financial segment marketing manager at Interxion.

“Solutions such as BigTable, Cassandra, and SimpleDB are playing a key role in this, especially as these solutions are typically highly scalable as they can be easily spread over a cluster of machines in the cloud,” ter Hoeven said. “The increasing use of supercomputers and the usage of software solutions such as Hadoop are also driving this development.”

On-demand business analytics employ a SaaS [software as a service] model to provide massive data volume support and distributed computing to solve problems faster in the cloud.

“This model has several big advantages, such as the ability to grow and shrink resource usage on an as-needed basis and removing the need to incur the costs and complexities of building and maintaining complex infrastructure in-house, said Matt Blakely, business intelligence technology practice lead at SWI, a software consultancy.

”As data volumes become extreme and must be combined with internal and cloud based sources, the demand and growth for BI platforms which utilize SaaS and the cloud will see a big increase,” Blakely said.

Xignite, a global market data cloud, is providing bond data on-demand, which will allow financial services professionals to adapt strategies to the volatile market on-the-fly.

“Firms will be able to access a wider universe of OTC and exchange-traded bond data without having to commit to a specific universe beforehand,” said Rajesh Razdan, director of product marketing at Xignite.

Xignite offers more than 50 financial cloud APIs and fulfills more than five billion service requests per month for real-time, historical and reference data covering global equities, commodities, currencies, options, fixed income, mutual funds, derivatives and OTC instruments.

“Investment managers, traders, risk and compliance staff all desire access to a large data universe, but their firms struggle to keep up with on-going demands of petabyte-era constant capacity expansion of IT infrastructure to process, store and disseminate this data to various applications and processes,” said Razdan.

Technology and cloud service providers are maturing at a rapid clip.

“The scalability of the cloud is its most compelling aspect,” said Steve Melanaski, chief technology officer for capital markets at Hewlett-Packard. “As volumes of unstructured data such as social media content expand, you don’t have to worry about additional hardware.”

Autonomy, an HP Company, announced this week that Autonomy’s private cloud now manages more than 50 petabytes of web content, video, email and multimedia data on 6,500 servers in 14 data centers around the world.

Fifty petabytes is equal to 665 years of HD-TV video, or 1 billion four-drawer file cabinets filled with text.
Powered by Autonomy’s Intelligent Data Operating Layer (IDOL), the private cloud automatically recognizes concepts and patterns in billions of structured and unstructured data files it ingests and indexes every day.

Autonomy provides a range of cloud-based solutions that leverage Autonomy IDOL’s ability to understand in real time massive volumes of unstructured data inside and outside an organization.

To be sure, cloud computing is a deep dive, requiring investment in skills and technology and the creation of horizontal services, which thereafter can be leveraged in a cloud environment.

“For capital markets, private clouds enable firms to scale out applications on a massive scale, similar to Amazon or Google,” said Roji Oommen, senior director, business development, financial services at Savvis.
“The biggest challenge is migrating legacy applications,” Oommen said. “You can’t just take your existing application stack and move it seamlessly to the cloud. The overwhelming majority of legacy applications are not cloud-friendly.”

Xignite’s bond data cloud offerings include bond security master product for U.S. bonds, price data for Finra TRACE reportable and exchange traded bonds, security master detail for European and Asian corporate bonds, and real-time price data for Finra TRACE reportable and exchange-traded bonds.

Key to Xignite’s offering is the ‘on-demand’ aspect of the platform. Applications can request data on an as-needed basis model, whereby only a subset of data that’s required by the application or process at a particular time can be requested and immediately made available through web services.

“For firms that do need large datasets the platform provides mass data delivery directly into cloud storage drop boxes such as Amazon S3, thus enabling firms to take advantage of cloud computing in a holistic manner,” Razdan said.

Various exchanges are building their cloud data offerings directly on the Xignite platform, thus removing latencies that are associated with tradition data redistribution architectures.

Xignite partners with various exchanges, data vendors and get certain data directly from sources. “Xignite adds value by enabling users to request data piece meal and creating custom composites from multiple data services,” said Razdan.

Pension funds, sovereign wealth funds, endowments and other institutional asset owners are sitting on vast troves of data -- but extracting value from that data is more challenging than ever.

#AssetOwners #DataQuality

Technology costs in asset management have grown disproportionately, but McKinsey research finds the increased spending hasn’t consistently translated into higher productivity.
#AI #Fiance

We're in the FINAL WEEK for the European Women in Finance Awards nominations – don't miss your chance to spotlight the incredible women driving change in finance!
#WomenInFinance #FinanceAwards #FinanceCommunity #EuropeanFinance @WomeninFinanceM

ICYMI: @marketsmedia sat down with EDXM CEO Tony Acuña-Rohter to discuss the launch of EDXM International’s perpetual futures platform in Singapore and what it means for institutional crypto trading.
Read the full interview: https://bit.ly/45xRUWh

Load More

Related articles

  1. Chainlink enables 21X to bring real-time, verifiable market data for tokenized securities onchain.

  2. The typology will help trading firms ready themselves for the pending European consolidated tape.

  3. This enables traders to anticipate volatility, minimize market impact & optimize execution in real time.

  4. This is a significant milestone towards mainstream adoption of onchain finance.

  5. From The Markets

    SIX Selects Corvil Analytics

    Corvil Analytics provides improved data transparency and helps optimize low-latency data delivery.

We're Enhancing Your Experience with Smart Technology

We've updated our Terms & Conditions and Privacy Policy to introduce AI tools that will personalize your content, improve our market analysis, and deliver more relevant insights.These changes take effect on Aug 25, 2025.
Your data remains protected—we're simply using smart technology to serve you better. [Review Full Terms] | [Review Privacy Policy] By continuing to use our services after Aug 25, 2025, you agree to these updates.

Close the CTA