Why Enterprise Architecture Is Becoming Key to Successful AI

Artificial intelligence is changing how enterprises operate, but a gap exists between experimentation and tangible business value.

Unify SAP and non-SAP data into trusted, multidomain views of every entity you run on

While approximately 90% of companies have embarked on their AI journeys, only six to 10% are using the technology effectively, according to figures shared by Harish Parmar, enterprise architect at SAP, during his presentation at the recent Reltio event, “Data in the AI Era: Connecting the Dots.”

The lack of effectiveness isn’t due to a lack of AI capabilities. Organizations are struggling to connect sophisticated technologies with the data, processes, and business context required to make them useful.

For enterprise architects, addressing these challenges requires a fundamental shift from connecting applications and systems to designing intelligent, increasingly autonomous enterprises.

“AI does not have a context problem, but the enterprises have representation problems,” Parmar said.

Enterprises struggle to represent their business relationships, data, and processes in ways AI systems can understand and act upon.

Why AI ambitions are outpacing enterprise readiness

Parmar identified four obstacles preventing organizations from realizing AI’s potential:

  1. Poor data quality
  2. Missing business context
  3. Inadequate governance
  4. Architectural debt

Data quality remains a persistent challenge, with Parmar citing estimates that 80% of AI project effort goes toward data preparation. But clean data alone isn’t enough and AI must understand how information relates to business operations, including relationships among customers, transactions, applications, and organizational policies.

Governance presents another challenge. Unlike consumer-facing generative AI tools, enterprise AI must operate within established security, authorization, and compliance requirements. An AI assistant, for example, should not disclose confidential employee salary information simply because someone asks.

Architectural debt further complicates adoption. Most enterprises cannot replace existing systems overnight, making incremental modernization essential.

From connected systems to intelligent enterprises

Parmar traced decades of technological evolution, from on-premises infrastructure and monolithic applications to cloud computing, SAP HANA, cloud-native environments, and AI-native platforms.

Enterprise data has undergone a similar transformation.

Initially used for recording transactions, data became the foundation for reporting, dashboards, and business intelligence. Big data, machine learning, and large language models expanded its potential. Enterprises are now entering a phase centered on business semantics, knowledge graphs, and contextual intelligence.

Traditional enterprise architectures typically connected three layers: applications, data, and user experiences. AI introduces another essential component: context.

“The context enables the AI to understand what the data is all about because without context, we don’t know how to make the meaning out of it,” Parmar explained.

Five layers of AI context

Parmar outlined five interconnected layers enterprise architects should consider when building AI-ready environments.

The first is data context, establishing how information is organized and related.

The second is semantic and knowledge context, connecting information across applications and business systems. Consider a customer whose information appears in an SAP ERP system, Salesforce, and a point-of-sale application. Each may contain different identifiers for the same individual. Establishing a unified customer record gives AI a more complete picture of transactions and interactions.

The third layer, architecture and governance context, addresses how AI agents interact with enterprise systems, including authentication, authorization, monitoring, and auditing. Parmar referenced SAP LeanIX, SAP Cloud ALM, and SAP AI Agent Hub as technologies supporting architectural visibility and governance.

The fourth layer, process and predictive context, connects intelligence to business activities such as procurement, sales, and accounting. SAP Signavio, for example, helps organizations understand business processes and operational workflows.

Finally, company-specific context incorporates organizational policies, approval limits, and business rules. This layer is important because policies and employee responsibilities continually change. An AI agent operating with outdated purchasing authorization information could make an incorrect decision. Together, these layers establish a foundation for AI systems that understand enterprise information and how it should be used.

Building AI agents that can scale

As enterprises introduce custom AI applications, Parmar emphasized three architectural principles. First, organizations should treat custom AI agents as independent products with clearly defined capabilities. Second, those products should be configurable and capable of communicating with different systems through standard protocols, including Model Context Protocol (MCP). Third, wherever practical, they should be designed as headless services operating across multiple user interfaces.

This approach laid the foundation for a more sustainable solution architecture. By prioritizing flexibility and reusability, it enabled broader adoption across scenarios while avoiding fragmented implementations that often become difficult to support and maintain over time.

It also matters for SAP customers transitioning from SAP ECC to SAP S/4HANA, who should consider whether custom AI applications can continue operating after migration.

“Whenever you think about architecture, think holistically,” Parmar advised. This means ensuring SAP and non-SAP systems, authentication requirements, and integration mechanisms are in place.

Governance cannot be an afterthought

As AI agents become more autonomous, governance must be embedded directly into enterprise architecture.

“Governance cannot be an afterthought,” Parmar said. Organizations need mechanisms documenting which agents operate, which systems they access, what models they use, and how decisions can be reviewed. However, technical oversight is only part of the equation.

Parmar compared AI agents to newly hired interns who must learn an organization’s policies, procedures, and ethical expectations. Just as employees receive training before assuming responsibility, AI agents need appropriate instructions, guardrails, and access to current company policies.

“A human is at the end of the day, accountable, and this is non-negotiable,” Parmar said. Even when AI agents execute decisions independently, people must remain responsible for their actions and outcomes.

Enterprise architect’s evolving role

AI is also transforming the skills required of enterprise architects. Technical expertise remains essential, but Parmar believes education and storytelling are becoming equally important. Architects must help business users understand AI, explain its limitations, and communicate what’s required before deploying new capabilities.

“Storytelling is a key skill which is evolving as a must-have requirement for an enterprise architect,” Parmar said.

The pace of technological change also demands greater experimentation and adaptability. Rather than relying on multiyear technology assumptions, organizations must continually reassess emerging capabilities.

AI is also lowering barriers to application development, making imagination, business understanding, and identifying valuable use cases increasingly important.

When these foundations come together, organizations can better translate AI investments into business outcomes. It doesn’t eliminate the need for people. Instead, it elevates their responsibilities toward judgment, oversight, and higher-value work.

“We are the captains of this Autonomous Enterprise,” Parmar said.

Subscribe to our newsletter to receive weekly news highlights from the SAP News Center
Previous Next
Close
Test Caption
Test Description goes like this