STATE OF THE UNION ABOUT AI AGENTS REPRESENT AND TECHNOLOGY EVOLUTION FOR LLM

The Age of AI is rising from Assistants to Autonomous Systems

Scientists recently talked about shifting focus from LLMs to agent orchestration and the rise of agentic app architectures

Artificial Intelligence (AI) continues to evolve at an unprecedented pace, influencing various domains, from software infrastructure to workforce dynamics. This position paper explores the future of AI, its integration into business processes, and the challenges that lie ahead.

Phases and Strategies

AI development can be categorized into three key phases: assistant, supervisory, and exception handling.

The assistant phase enhances productivity by serving as a copilot for human workers, offering contextualized knowledge and reasoning support without full autonomy. Solutions like Microsoft Copilot and Amazon Q exemplify this approach by feeding users relevant insights and aiding in decision-making while requiring human judgment for execution. These systems excel at synthesizing data, summarizing information, and suggesting next steps but remain dependent on human oversight to ensure accuracy, strategic alignment, and identifying the best solution to a problem.

For example, Microsoft Copilot integrates with Office applications, allowing users to draft emails, summarize reports, and generate data-driven insights without manual input. However, while it enhances efficiency, users must still verify and refine outputs to maintain accuracy and intent. Similarly, Amazon Q provides developers with intelligent coding assistance, helping to debug software, suggest optimizations, and generate documentation. Yet, it cannot make independent architectural decisions or validate security implications, reinforcing the necessity of human expertise.

These systems illustrate how AI can significantly improve productivity without replacing human reasoning, ensuring businesses leverage technology effectively while maintaining control over critical decisions.

The supervisory phase involves AI performing tasks autonomously under human oversight. In this stage, AI agents move beyond mere assistance to executing workflows with minimal human input, requiring approval only at key decision points. For instance, in financial operations, AI-driven systems can autonomously process invoices, flag anomalies, and generate financial reports, with human intervention limited to verifying flagged exceptions. Similarly, in supply chain management, AI can automate inventory restocking through RPA order information extraction, ERP connection, and data verification, and finally, by analyzing demand patterns and placing orders, requiring human approval to confirm assumptions and proceed. This shift enables businesses to scale operations efficiently while ensuring accountability and control remain with human supervisors.

The final phase — exception handling — envisions AI managing routine operations independently, requiring human intervention only in novel or unexpected scenarios. At this stage, AI systems can make complex decisions, execute predefined workflows, and dynamically adapt to common variations without human oversight. Industries such as autonomous logistics, financial fraud detection, and smart manufacturing exemplify this phase, where AI continuously refines operations based on real-time data. For example, in autonomous supply chain management, AI can predict and resolve demand fluctuations, reroute shipments, and manage procurement without human involvement unless an anomaly — such as a geopolitical crisis or supply chain disruption — requires escalation. In fraud detection, AI monitors financial transactions and autonomously blocks suspicious activities, alerting human operators only when confidence in fraud detection is low. While this phase pushes AI closer to full autonomy, human oversight remains essential for rare, unpredictable, or ethical decision-making scenarios, ensuring that AI systems remain aligned with business goals and societal values.

Cloud infrastructure remains central to AI’s development, with a strategic choice between enhancing existing technologies like GPUs and Kubernetes or pioneering new frameworks. This debate underscores the challenge of balancing innovation with stability in enterprise AI adoption.

AI’s Disruption of ERP and CRM Systems

AI is set to revolutionize enterprise software by automating and replacing traditional ERP and CRM systems. Today, these systems often do not enhance the quality of work for employees but instead add layers of friction, serving primarily as tools to provide supervisors with a perception of control. For instance, many ERP solutions require extensive manual data entry, approval chains, and rigid workflows that slow decision-making rather than empowering employees. Similarly, CRM systems are frequently structured around tracking interactions rather than facilitating meaningful customer engagement, forcing sales teams to spend excessive time logging activities rather than building relationships. AI-driven automation eliminates these inefficiencies by enabling more adaptive, intelligent workflows that prioritize user experience while ensuring necessary oversight.

The transition will occur in phases, initially focusing on integrating AI-driven automation before complete system replacement. Traditional ERP and CRM systems have long relied on standardized workflows and rigid structures, often requiring businesses to adapt their processes to fit the software rather than the other way around. However, a new generation of AI-driven enterprise solutions is emerging and characterized by adaptability, real-time learning, and personalized automation.

These next-generation systems leverage AI to provide contextualized decision-making, allowing businesses to dynamically adjust workflows based on operational needs rather than predefined templates. For example, AI-driven CRMs can automatically surface relevant customer insights and recommend actions based on real-time data, reducing manual input and improving engagement. Similarly, AI-powered ERPs can optimize supply chain operations by predicting demand fluctuations and adjusting procurement strategies without requiring manual intervention.

The benefits of this shift include increased efficiency, reduced administrative burdens, and more intuitive user experiences. Instead of employees spending time on redundant data entry and approval loops, AI-driven systems ensure that human effort is focused on strategic decision-making, collaboration, and problem-solving. Unlike their predecessors, which primarily served as control mechanisms for management, these AI-enhanced platforms prioritize productivity, agility, and user empowerment while still maintaining necessary oversight.

Workforce and Demographic Shifts

Demographic trends, particularly in Europe, present economic challenges. Countries like Germany face a workforce deficit of five million people in the next five years, and AI is a critical solution to maintain productivity. However, AI is not designed to replace human workers but to complement their skills, enabling businesses to address labor shortages while enhancing job satisfaction. By automating repetitive tasks and optimizing decision-making, AI allows human workers to focus on higher-value activities such as creativity, strategic planning, and collaboration. Ethical AI deployment ensures that technology is used to augment human potential rather than diminish it, supporting sustainable economic growth while addressing the realities of an aging workforce and shifting demographics.

Reliability and Ethical Considerations

AI’s role in exception management and decision-making continues to evolve. Although large language models (LLMs) have significantly reduced hallucination rates — from 12% to below 1% — challenges remain in ensuring AI reliability. High-profile incidents, such as Air Canada’s chatbot mistakenly granting a non-existent discount, highlight the risks of uncontrolled AI decision-making.

In the Air Canada case, a customer inquired about bereavement fares through the airline’s chatbot. The AI incorrectly informed the customer that they could retroactively apply for a bereavement discount after purchasing a ticket. The customer, equipped with such information, proceeded with the booking but was later denied the discount when they contacted customer service. A lawsuit followed, and the court ruled in favor of the customer, holding Air Canada accountable for the chatbot’s misinformation, as it was an official part of their website.

This incident underscores the necessity of responsible AI implementation and supervision. While AI agents can efficiently handle common queries, they are prone to errors when faced with ambiguous or nuanced situations. Businesses must ensure that AI systems undergo rigorous testing, are continuously updated with accurate information, and have precise escalation mechanisms for human intervention when uncertainty arises. As AI continues to take on decision-making roles, organizations must balance automation with accountability to prevent similar failures and maintain consumer trust.

Defining AI agents is critical. Experts argue that agents should operate within structured, microservice-like frameworks rather than freely accessing the Internet. This controlled approach mitigates risks and ensures that AI systems provide trustworthy outputs. AI agents should not possess a broad, generalized knowledge base but instead be specialized in narrow, well-defined domains relevant to their specific function. This approach mirrors the principles of microservices in software architecture, where each component is responsible for a distinct task, ensuring modularity, scalability, and precision.

To effectively manage these specialized agents, a higher-level routing system is required. To address this goal, a router agent, such as OpenAI’s “Triage” model, comes into play. These router agents act as intermediaries, determining which specialized AI should handle a given task based on context and domain specificity. For example, in a business environment, a customer service AI agent should focus solely on support inquiries, while a finance AI agent should handle expense approvals and financial forecasting. The router ensures that requests are accurately directed to the right agent, improving efficiency and reducing the risk of misinformation. Organizations can maximize AI’s effectiveness while maintaining control and reliability by implementing structured, domain-specific AI agents alongside an intelligent routing system.

Open-Source AI and the Shift Toward Custom Models

Open-source AI models like DeepSeek are reshaping the industry by allowing businesses to fine-tune AI for specific applications while maintaining transparency and flexibility. The rise of open-source AI initiatives, such as Meta’s Llama, Mistral AI, and Hugging Face’s ecosystem, demonstrates a growing shift toward decentralized innovation. These projects enable organizations to access, modify, and deploy AI models tailored to their unique needs without reliance on proprietary ecosystems.

Unlike closed-source models, which often restrict adaptability and require significant licensing fees, open-source AI fosters a collaborative development environment where improvements and security enhancements are shared across a global community. This movement is incredibly impactful in industries requiring compliance with strict data privacy regulations, such as healthcare and finance, where organizations need complete control over how AI processes sensitive information.

Moreover, companies adopting open-source AI can integrate multiple models, leveraging strengths from various architectures while optimizing infrastructure costs. Open-weight models also allow enterprises to build domain-specific solutions with more predictable performance, as they can fine-tune models on proprietary datasets rather than relying on generalized training data.

As the focus shifts from developing new AI models to refining their deployment and integration, businesses must decide whether to embrace the flexibility of open-source solutions or the stability and support of proprietary models. Regardless of the choice, the open-source movement sets new standards in AI accessibility, ethical transparency, and innovation.

Preparing for the AI-Driven Future

AI’s trajectory points toward an increasingly autonomous role in business and daily life; however, its effectiveness hinges on responsible implementation, robust security measures, and ongoing human oversight. Organizations must approach AI adoption strategically, ensuring it enhances productivity, creativity, and problem-solving rather than merely automating processes for efficiency gains.

The key takeaways from this discussion highlight the necessity of structured AI integration. Businesses must prioritize AI systems that support human expertise rather than replace it, fostering collaboration between automated agents and decision-makers. The shift from rigid, control-driven enterprise software to adaptive, AI-powered solutions promises greater efficiency, reduced friction, and enhanced user experience. Additionally, open-source AI models and agentic approaches allow organizations to build tailored solutions while maintaining transparency and adaptability.

To fully realize AI’s potential, enterprises must embrace a long-term vision where AI-driven systems are continuously refined, ethically guided, and aligned with business and societal values. As AI evolves, the challenge will not be whether to adopt it but how to do so responsibly — ensuring that trust, security, and human judgment remain at the core of technological progress.

As AI continues to evolve, the question remains: how can businesses harness its power while maintaining control and trust? The answer lies in strategic integration, continuous validation, and a clear understanding of AI’s capabilities and limitations.