AI Implementation Challenges: Pipeline Sprawl and Shadow AI with Hema Raghavan

There is a specific kind of tension that fills the air in the West Loop and throughout the Chicago business corridors when the conversation shifts from AI hype to AI implementation. For most executives, the dream is a seamless integration of predictive intelligence, but the reality is often a tangled web of what industry insiders call “pipeline sprawl” and the creeping risk of “shadow AI.” This is the messy truth of the current enterprise AI landscape—a gap between the promise of the technology and the grueling manual labor required to make it actually work. This tension is exactly why the upcoming Executive Dinner in Chicago on April 16, focused on operationalizing AI, is such a critical touchpoint for the city’s tech leadership.

Moving Beyond the Relational Data Ceiling

For years, Chicago’s massive corporate footprints—from logistics giants to financial powerhouses—have sat on mountains of relational data. This data, typically stored in environments like Snowflake or Databricks, tracks every customer transaction, every behavior, and every interaction. However, as Dr. Hema Raghavan, co-founder and Head of Engineering at Kumo.ai, points out, turning that structured data into accurate predictions has traditionally been a slog. It often requires months of manual feature engineering performed by specialized teams who have to essentially “hand-craft” the signals the AI needs to see.

The bottleneck isn’t the data itself, but the architecture used to analyze it. Most companies are stuck in a relational mindset, even as the most powerful patterns in business are actually relational graphs—connections between entities. This is where Graph Neural Networks (GNNs) approach into play. By leveraging GNNs, enterprises can move past the limitations of traditional table-based analysis to see the deeper connections in their data. Kumo.ai has focused on democratizing this by creating a system that connects directly to existing relational data warehouses, removing the demand for companies to undergo the costly and time-consuming process of converting their data into a separate graph format.

The Philosophy of Simple Interfaces

One of the most significant hurdles in the Chicago tech scene is the divide between the data scientist and the business stakeholder. When technology is too complex, it leads to “shadow AI,” where different departments deploy their own fragmented, unmonitored tools to get results quickly. To combat this, the focus has shifted toward building simple interfaces for complex technology. Kumo.ai, for example, developed a SQL-like query language that allows users to access sophisticated GNN capabilities without needing a PhD in machine learning. This approach hides the underlying complexity while preserving the power of the model, allowing for a faster modern data stack strategy that prioritizes business outcomes over technical vanity.

Infrastructure Economics and the ROI of AI

Performance is often chased at any cost in the AI race, but for a sustainable enterprise strategy, the economics must make sense. The industry has seen a trend of over-reliance on GPUs, which can be prohibitively expensive at scale. A more nuanced approach involves hybrid CPU/GPU infrastructure, which optimizes for real-world economics without sacrificing the capabilities of the model. This thoughtful design ensures that enterprise-scale AI remains accessible to a broader range of customers, rather than just the wealthiest firms with unlimited compute budgets.

This focus on ROI is central to the development of KumoRFM, the first foundation model designed specifically for structured enterprise data. Unlike general-purpose LLMs, a foundation model for structured data allows organizations to generate predictions directly from their data warehouse without requiring task-specific model training for every single use case. This drastically reduces the “time-to-value,” allowing a company to move from a hypothesis to a production-ready prediction in a fraction of the time it previously took.

The pedigree behind these innovations is significant. Dr. Raghavan’s experience leading machine learning teams at LinkedIn—contributing to the platform’s growth from 400 million to 700 million users and building systems like “People You May Know”—provides a blueprint for how to scale AI from a laboratory experiment to a global utility. This proves this transition from “experimental” to “operational” that defines the current era of AI, and it is why her recognition on Inc.’s 2026 Female Founders 500 list is a testament to the impact of simplifying enterprise AI.

Navigating the AI Mess: A Chicago Resource Guide

Given my background in analyzing the intersection of technology and local economic growth, the “messy truth” of AI requires a specific set of local expertise to solve. If you are managing a digital transformation in the Chicago area and are struggling with pipeline sprawl or the inefficiencies of your current AI strategy, you shouldn’t look for generalists. You need specialists who understand the friction between legacy relational data and modern predictive models. Here are the three types of local professionals you should prioritize when building your AI operationalization framework.

Enterprise AI Strategists

Look for consultants who specialize in “operationalization” rather than just “implementation.” The right professional should have a track record of reducing “time-to-value” and can demonstrate how they have moved a company from manual feature engineering to automated predictive pipelines. Avoid those who only offer high-level strategy; seek those who can audit your current “shadow AI” footprint.

Data Architecture Specialists (Snowflake/Databricks Certified)

Since the goal is to leverage existing relational data without costly conversions, you need architects who are experts in the specific warehouses you already use. Look for professionals who understand how to integrate Graph Neural Networks or foundation models directly into Snowflake or Databricks environments, ensuring that your data remains secure and your latency remains low.

MLOps (Machine Learning Operations) Engineers

To solve the problem of “pipeline sprawl,” you need an engineer focused on the lifecycle of the model. The ideal candidate should be proficient in creating reproducible pipelines and monitoring models for drift. Look for engineers who prioritize “infrastructure economics”—those who can balance the use of CPUs and GPUs to keep your operational costs sustainable as you scale.

Ready to find trusted professionals? Browse our complete directory of top-rated podcast,se-tech,se-stackoverflow experts in the Chicago area today.