Goldman Sachs Open Sources Data Modeling Platform10.19.2020
The Fintech Open Source Foundation (“FINOS”), together with platinum member Goldman Sachs, today announced the launch of Legend, Goldman’s flagship data management and data governance platform. Developed internally and used by both engineers and non-engineers alike across all divisions of the bank, the source code for five of the platforms’ modules have today been made available as open source within FINOS.
Today’s launch comes on the heels of the completion of a six-month pilot in which other leading investment banks, such as Deutsche Bank, Morgan Stanley and RBC Capital Markets, used a shared version of Legend, hosted on FINOS infrastructure in the public cloud, to prototype interbank collaborative data modeling and standardization, in particular to build extensions to the Common Domain Model (CDM), developed by the International Swaps and Derivatives Association (ISDA). This shared environment is now, starting today, generally available for industry participants to use and build models collaboratively. With the Legend code now available as open source, organizations may also launch and operate their own instances. The components open-sourced today allow any individual and organization across any industry to harness the power of Goldman Sachs’ internal data platform for their own data management and governance needs as well as contribute to the open code base.
“Legend provides both engineers and non-engineers a single platform that allows everyone at Goldman Sachs to develop data-centric applications and data-driven insights,” said Atte Lahtiranta, chief technology officer at Goldman Sachs. “The platform allows us to serve our clients better, automate some of the most difficult data governance challenges, as well as provide self-service tools to democratize data and analytics. We anticipate that the broad adoption of Legend will bring real, tangible value for our clients as well as greater standardization and efficiency across the entire financial services ecosystem.”
“Information is the lifeblood of financial services, but it is becoming increasingly more difficult to ensure you have accurate, complete and timely information,” said Neema Raphael, chief data officer and head of data engineering at Goldman Sachs. “Over the last seven years, Goldman Sachs has been developing, in the form of the Legend platform, a new way to provide fast, easy, secure access to our information—for revenue generation, better client service, operational efficiency and regulatory compliance. We believe this new data platform is so powerful and important that we are making it available to our clients and the world fully open and free of charge as an open source platform through FINOS.”
The Legend platform and language were together known as “Alloy” and “PURE” internally within the bank. “The choice of a new holistic name, Legend, reflects our vision of a singular platform to serve as a critical guide—a legend—for Goldman’s internal data strategy, whether building data services for the full lifecycle of a trade or more easily servicing client and regulatory requests,” added Pierre De Belen, head architect of the Legend platform.
“Financial services firms have much to gain from open source adoption as the potential for its use to reduce financial burdens and needless complexity is nearly unlimited,” said Gabriele Columbro, executive director of FINOS. “A leader in our foundation from day one, Goldman Sachs shares this vision. Legend provides a concrete path towards greater collaboration and data standardization which will benefit all market participants, large and small.”
“Legend is an impressive technology with great potential for improving industry efficiency,” said Stephen Goldbaum, an executive director at FINOS platinum member Morgan Stanley, which participated in the Legend Pilot. “We see tremendous potential for synergies between Legend and our own Morphir project, also open-sourced through FINOS, just last month. These contributions together validate the FINOS model of bringing industry competitors together to solve industry challenges.”
“Elements of a complex instrument, such as a currency option, might be stored in dozens, even hundreds, of systems within a modern investment bank’s infrastructure across its front, middle, and back office—Legend is a critical tool Goldman Sachs uses to maintain data consistency across those various applications and databases and now it’s available for the entire industry to use,” said Rob Underwood, FINOS’ chief development officer who led the open sourcing effort for the foundation.
“Fierce market competitors, such as those that participated in pilot, do tremendous amounts of trading business with each other daily. They operate under common regulatory frameworks across the globe, such as MiFID II, which necessitate advanced data lineage tooling such as what’s available in Legend. The power of Legend to help drive data consistency and improve interoperability will compound as it’s adopted by more institutions, both sell-side and buy-side,” added Underwood.
Along with Goldman Sachs, FINOS members Deutsche Bank, Itau Unibanco, Morgan Stanley, RBC Capital Markets, ScottLogic, Wells Fargo as well as Digital Asset, ISDA and REGnosys participated in the pilot. The pilot group chose to use Legend for modeling work in two areas—FX option extensions to the CDM, as well as nascent work on commodities reference data. The pilot group also identified ESG data as a good potential future use case for Legend. The FX option extensions, specifically the Averaging Model used in the CDM, modeled collaboratively by the pilot group financial institutions participants using Legend, were proposed into the CDM and have since been accepted, released and integrated into a recent release to the public.
“The Legend pilot proved FINOS to be a great vehicle for open source enablement of the CDM for ISDA and its members,” said Ian Sloyan, director, market infrastructure and technology at ISDA. “Community use of the shared environment to do even more collaborative modeling will generate more interest in Legend, as well as the CDM and other standards maintained by ISDA. This is a win-win for all concerned.”
“Data architecture and modeling teams need to consistently innovate in order to improve data governance and controls, increase efficiency, reduce operating costs and deliver a seamless user experience,” said Russell Green, head of group architecture, Deutsche Bank, and a FINOS board member. “This is in addition to meeting the ever increasing data modeling requirements from data sourcing needs. Deutsche Bank is keen to explore and operationalize data modeling tools to further build seamless integration between different participants, toolkits and simplify data management. From our participation in the FINOS pilot, we believe that Legend Studio holds promise to enhance collaborative and federated data architecture and modeling within the bank and the industry.”
“We see great potential for banks and financial institutions to address common challenges of the industry collaboratively, with great potential for sustainability, ESG, market data distribution and risk management. We look forward to experimenting with the platform further and working together with an amazing community,” said Vanessa Fernandes, chief technology officer of emerging technologies & open source at FINOS Silver member Itau Unibanco.
“As an open core company, we are proud to have Legend deployed using GitLab for modeling source control which is now fully available to the open source community for further innovation within the financial services industry,” said Sid Sijbrandij, CEO at GitLab, a FINOS silver member. “GitLab is committed to open source contribution and stewardship and we see an incredible potential to simplify complexity, improve compliance, reduce costs and increase efficiencies, which ultimately strengthens institutions and supports the customers they serve.”
“REGnosys was founded on the premise that to solve the regulatory compliance challenge, the industry must collaborate out of shared, open source data and processing models,” said Leo Labeis, CEO, REGnosys, another participant in the pilot and curator of the Rosetta DSL used in the ISDA CDM. “We are extremely pleased to collaborate with FINOS and Goldman Sachs to further promote the development of industry standards, such as the ISDA CDM, which has stood the test of working across platforms.”
“At Scott Logic, we’ve long been proponents of the benefits of open source in creating common industry standards and a shared ecosystem of value. We were delighted to participate in the Legend pilot and look forward to contributing to its future development—including possible integrations with some of our own FINOS contributions, such as DataHelix,” said Colin Eberhardt, Scott Logic’s chief technology officer and FINOS board member.
Raphael first announced Goldman Sachs’ intention to open source Legend at last November’s Open Source Strategy Forum (OSSF), FINOS’ annual conference, and the only conference dedicated to driving collaboration and innovation in financial services through open source. Speakers from Goldman Sachs will return this year to OSSF, to be held virtually on November 12-13, to present on the next steps in Legend’s developments and adoption, including the bank’s intentions to open source additional portions of Legend in 2021.
The five modules open sourced today are Legend Studio, Legend Engine, Legend SDLC, Legend Shared, and the Legend-PURE language itself.
To request an account on the hosted instance of Legend, go to www.finos.org/legend. Legend’s documentation is available at legend.finos.org. To view, download and contribute to the Legend source code open sourced today, go to http://github.com/finos/legend.
To find out more about how to contribute to FINOS projects and participate in the FINOS open source community, please visit www.finos.org/get-involved.
If anyone thinks this move is purely for the good of the industry, they're crazy. In the same way that bigtech embraced open source, so should financial services – for competitive edge. And no, I'm not talking only about open banking here. #fintech #opensource https://t.co/QY34aZDTFN
— Virginie O'Shea (@virginieoshea) October 19, 2020
Singletrack combines on-demand technology and analytics for a purpose-built capital markets CRM solution.
The suite of tradeable indices are based on the Bloomberg Fixed Income Indices.
Bringing Neptune onto Genesis technology makes it easy to add features, datasets and analytics.
European financial markets would benefit from a well-functioning fixed income consolidated tape.
The goal of carbon reduction has become a key focus for investors.