No other science-fiction trope like a rogue AI is closer to reality, and Wall Street is doing everything it can to prevent the situation from happening.

“‘Governance’ cannot be considered a bad word,” said Cathy Bessant, chief operations and technology officers, during the 2018 AI Summit in Manhattan. “Most of the fintech companies that work with Bank of America think the governance and approval processes are bureaucratic.”

Some of the processes may give that appearance while others are as bureaucratic as they appear, but all of the processes act a firewall for the bank.

“What appears to be a bureaucracy, I would call good governance, which is absolutely necessary to make sure we have transparency into what we are doing,” she added.

Unlike venture capital-backed fintech startups, heavily regulated organizations like the Bank of America cannot adopt the “fail fast and fail often” development and deployment mentality.

“Our stockholders would flip out and so would our regulators,” said Bessant.

A critical piece of good governance in AI is transparency all along the lifecycle of the instance of AI.

“We have to understand the models we create, the data they use, the lineage of that data and model, which can be very difficult to do in a machine-learning environment,” he said.

AI and its sub-disciplines should not be thought of as the outcome themselves, but as a tool to produce insights that can deliver results and judgments, she noted.

Such demands often create tension, especially with third-party providers who view their models their intellectual property and competitive advantage.

“I cannot just buy a model and use its outcome,” said Bessant. “I have to understand that model regardless whether I created it or bought it.

Even when Bank of America uses AI instances mechanically, the algorithm itself procedures the judgment, but only after considerable review.

This approach is far from new for the bank, which has been using data and algorithms to reach decisions for more than 40 years.

“We are not new to this space where algorithms meet humanity,” said Bessant.