Financial institutions today are facing an overwhelming surge of transient data—high-velocity, short-lived, and often mission-critical. From streaming trade data and fraud signals to real-time KYC updates and credit scoring models, the tempo of financial operations has shifted to milliseconds. Yet many firms remain bottlenecked by traditional data architectures that prioritize centralization over responsiveness.
The outcome? Data lakes that are overflowing—but GenAI and AI/ML models that are underfed, or worse, trained on stale data.
The Data Paradox: Volume vs. Value
As financial institutions amass vast quantities of structured and unstructured data, a fundamental problem emerges: not all data is equally useful, and not all of it needs to be stored. Much of it is ephemeral but essential. Acting on it too late can be as bad as not acting at all.
The challenge isn’t simply collecting data—it’s ensuring that the right data is available, trustworthy, and contextual precisely when it’s needed. Traditional data pipelines and centralized lakehouses often struggle to keep up with these demands.
Coexisting with Lakehouses: A New Data Supply Chain
AWS provides the backbone for cloud-native scale and AI/ML model training and inferencing, while Denodo brings agility and abstraction through logical data management that can tap into various AWS data repositories and other sources. Together, we offer financial firms a coexistence strategy—one that blends the enduring value of lakehouses with the flexibility of real-time, virtualized data access.
We introduce the concept of AI-Ready Data Products: governed, reusable, and discoverable units of data tailored for specific outcomes. These could include credit risk profiles, real-time ESG exposure metrics, transaction behavior patterns, or asset management insights such as portfolio risk signals and real-time performance attribution.
What It Really Means to Be Data-Driven in Financial Services
Being data-driven goes far beyond deploying tools or training AI models. It’s a fundamental shift in how decisions are made and how organizations operate. For financial services firms, this means embracing:
- Culture: A data-driven culture values curiosity, transparency, and a willingness to challenge intuition with insights. It promotes cross-functional collaboration where data is treated as a shared asset rather than a siloed commodity.
- Ethics: With great data comes great responsibility. Ethical data use requires financial institutions to prioritize fairness, transparency, and bias mitigation—especially when AI influences credit, investment, or risk decisions.
- Governance: Strong governance frameworks ensure data quality, lineage, privacy, and regulatory compliance. Data products should carry policy tags, usage constraints, and auditable trails.
- Leadership Participation: Data-driven transformation must be championed from the top. CDOs, CFOs, and business leaders need to be active in defining data priorities, sponsoring data literacy, and aligning KPIs with data outcomes.
Ultimately, a truly data-driven financial institution doesn’t just react to data—it anticipates, shapes, and leads with it.
Beyond GenAI: The Full Spectrum of AI in Financial Services
While Generative AI is grabbing headlines, a successful AI strategy in financial services must span the entire AI spectrum—from predictive analytics and classical machine learning to decision automation and generative models. Each type of AI plays a unique and essential role:
- Descriptive & Diagnostic AI: Used to understand what happened and why. For instance, financial institutions can create interactive dashboards to visualize historical transaction data, helping detect fraud patterns or explain market anomalies. AWS offers tools enabling such data exploration and visualization, allowing analysts to dive deep and uncover valuable insights.
- Predictive AI: Essential for risk modelling, credit scoring, and churn prediction. These models forecast future outcomes based on past behaviors. With AWS’s machine learning platforms, banks can rapidly develop and deploy models to assess credit risk or predict market movements, leveraging automated features and a broad array of algorithms to enhance predictive capabilities.
- Prescriptive AI: Helps optimize decision-making—such as portfolio rebalancing strategies or regulatory capital allocation—by evaluating multiple risk scenarios. Banks can use AWS machine learning capabilities and workflow tools to orchestrate complex decisions while maintaining regulatory compliance and an auditable trail.
- Generative AI: Adds value by automating knowledge work—for example, generating compliance reports, summarizing customer interactions, or powering AI copilots for financial advisors. Through Amazon Bedrock, financial institutions can securely access and fine-tune leading foundation models for their specific needs, implement Retrieval-Augmented Generation (RAG) patterns to ground model responses in proprietary documentation, and leverage Bedrock’s Agentic capabilities to automate actions.
No single form of AI is sufficient. Combined, these approaches empower financial institutions to act in real time, plan for the future, and automate intelligently—all built on a foundation of trusted, AI-ready data.
Activating the Last Mile for GenAI
It’s not enough to simply adopt or fine-tune GenAI models. To deliver real business value, these models must interact with live, trusted, and governed data in real time.
According to McKinsey, more than 75 percent of financial institutions are experimenting with or deploying generative AI. Yet fewer than 10 percent have scaled these solutions into core business processes—highlighting the gap between pilot projects and enterprise-wide value realization [1]. Similar studies echo this challenge: Celent reports that nearly 70% of banks and insurers have initiated GenAI pilots, but fewer than 15% have moved beyond proof of concept [2]. Deloitte likewise finds that only 20% of financial institutions have deployed GenAI into production environments, with data access and governance cited as primary hurdles [3].
Many financial institutions remain stuck in “pilot purgatory,” where proof-of-concept models rarely reach production. A significant bottleneck is data activation—transforming scattered, often multi-cloud data into reliable, governed, AI-ready resources. In asset management, for example, delivering AI-ready data in real time is critical for applications such as portfolio optimization, ESG factor scoring, and automated investment commentary generation.
Success with GenAI isn’t merely about choosing the right model—it’s about creating a continuous data-to-insight-to-action cycle. This demands capabilities in:
- Data Engineering and Integration: Unifying real-time data from distributed and multi-cloud systems.
- AI/ML Operations: Feeding relevant, timely, trusted data into GenAI models.
- Workflow Embedding: Using insights to support decisions within operational systems.
- Feedback Loops: Closing the loop with real-world results and model refinement.
Financial institutions often operate across hybrid and multi-cloud environments, requiring seamless data virtualization and consistent governance—a space where Denodo excels.
Centralizing to Scale
Highly centralized operating models succeed because they concentrate talent, unify architectures, and simplify governance—enabling consistent policies, lineage, and compliance across distributed environments. However, physically centralizing all data into a single platform is increasingly impractical given the volume and velocity of financial data.
Treating Data as a Product
The winners in this space are focusing on AI-ready data products—categorized as Bronze (raw), Silver (curated), and Gold (decision-grade) datasets—tailored to specific GenAI use cases. Business units provide context and requirements for these data products, while centralized data teams ensure governance, consistency, and compliance. This balance is crucial in highly regulated financial environments.
Financial institutions are facing a deluge of structured and unstructured data from transactions, logs, apps, cloud services, regulators, and third parties. Rather than physically consolidating it all, successful organizations leverage logical data management to unify and govern data virtually, maintaining the flexibility required for rapid, AI-driven insights.
Agentic AI in Action
Across financial services—banking, asset management, insurance, wealth management, and capital markets—GenAI is disrupting traditional processes.
In asset management, investment strategies are being reimagined, client reporting has become real-time and contextual, and entirely new asset classes are emerging.
Agentic AI holds transformative potential:
- Portfolio Rebalancing Bots: Monitoring markets and adjusting positions in real time, enabling asset managers to optimize strategies based on dynamic market conditions.
- Customer Service Agents: Resolving multi-step issues across channels.
- Fraud Detection Agents: Monitoring anomalies, triggering alerts, and executing interventions.
- Regulatory Surveillance Bots: Flagging and explaining compliance issues on the fly—essential in asset management for monitoring portfolio mandates and regulatory limits.
- Internal Efficiency Agents: Developer copilots assist data engineers in transforming data pipelines, while policy-checking agents help compliance teams validate regulations in real time.
A striking example comes from a trading desk prototype where a GenAI agent monitored sentiment, adjusted positions, and suggested hedges. However, without adequate throttling, the agent executed a strategy that nearly exposed the firm to regulatory reporting violations—because it wasn’t constrained by compliance boundaries embedded in other systems. This underscores the critical need for Responsible AI frameworks to ensure model outputs remain explainable, safe, and compliant.
The Future Belongs to AI-Native Institutions
The future belongs to AI-native financial institutions that can continuously learn, personalize, and adapt in real time—while ensuring safety, compliance, and ethical guardrails through robust governance frameworks. The combination of AWS and Denodo offers firms a powerful way to tame the flood, transforming raw, transient data into trusted, AI-ready products.
Because in the age of GenAI, data’s value lies not in its sheer volume—but in its velocity, veracity, context, and the increasingly critical requirement to apply it responsibly. Whether it’s driving trading strategies, optimizing asset management portfolios, or enhancing customer experiences, firms that master AI-ready data will shape the future of financial services.
References
[1] McKinsey & Company. “Generative AI: From Buzz to Business Value in Financial Services.” December 2023.
[2] Celent. “GenAI in Financial Services: From Pilot to Production.” 2023.
[3] Deloitte. “AI in Financial Services Outlook.” 2024.
LinkedIn Profile: https://www.linkedin.com/in/muammar/