AI Context Debate: Enterprises Seek Real-time Organizational Truth, Transcending Reliance on Frontier Models

Industry Background: Why Is the "Context" of AI Agents Insufficient?

This week, a key debate in the global AI industry revolves around "real-time organizational truth." Earlier, veteran analyst Jon Reed published an article titled "Why context isn't enough — AI agents need real-time organizational truth," directly pointing out the limitations of current AI agents' over-reliance on limited context, and calling on the industry to establish a "real-time organizational truth" mechanism. This viewpoint has sparked intense feedback from vendors, enterprise users, and research institutions.

Reed points out that most vendors are making two mistakes: first, they confine context to the limited enterprise data they possess (even data lakehouse vendors lack real-time context); second, they are unwilling to acknowledge the fragility of context, especially at enterprise scale. Reed believes that without more comprehensive real-time information input to agents, no matter how powerful the model, agents will appear "not smart enough."

This debate takes place against the backdrop of enterprise AI moving from the "demonstration stage" to the "large-scale deployment stage." However, high inference costs, complex architectures, and limited business value have led many enterprise CFOs to question "when will AI deliver real returns?" Bill Patterson, Chief Customer Officer at Salesforce, said in a recent interview: "CFOs are asking — can AI deliver returns at some point? We need to shift from 'how many tokens used' to 'whether a deal was closed.'"

Market Impact: Enterprises Reassess AI Investment Strategies

Reed's viewpoint directly challenges the over-reliance on frontier models. He cites multiple studies proving that, with proper context engineering and "intelligent control" (harness engineering), small models can reduce LLM operational costs by up to 90%. Even Google acknowledged in its latest white paper that reasonable context/intelligent control can contribute 90% of the value of an agent system, while the model itself only accounts for 10%.

This means enterprises no longer need to endlessly purchase GPUs and tokens to gain AI capabilities. Instead, they can constrain and enhance LLM agents by building effective "intelligent controls" — including verification engines, pre-packaged skills, and strict guardrails — thereby achieving cheaper and more sustainable AI deployment.For NVIDIA, AMD, and other AI infrastructure providers, this trend may bring shifts in demand structure: reliance on GPUs for inference will partially shift toward investment in data pipelines and real-time retrieval systems. For model providers like OpenAI and Anthropic, if enterprises can significantly reduce model consumption through context engineering, their token-based business models may face pressure.

Competitive Landscape: Who Benefits? Who Faces Pressure?

• Beneficiaries:
• Data integration and real-time analytics platforms (e.g., Databricks, Snowflake, Confluent)—organizing the truth in real time requires merging system-of-record data with collaborative information from Slack/Teams, and the value of these platforms becomes increasingly prominent.
• Intelligent control and AI agent orchestration tools (e.g., LangChain, CrewAI, etc.)—they help enterprises manage agent context, tools, and constraints, becoming a critical link in enterprise AI architecture.
• Vendors focused on industry-specific AI (e.g., IFS, ServiceNow)—they emphasize domain knowledge rather than general model capabilities.

• Those Under Pressure:
• AI model API providers that rely solely on token consumption models (e.g., OpenAI)—if enterprises significantly reduce token usage through context engineering, their revenue growth may slow down.
• Vendors that overemphasize "first-principles" reasoning capabilities—enterprises care more about "whether the outcome is successful" than "whether the model is smart."

• Potential Followers:
• Amazon AI and Google Cloud—they are already offering more context-related services (e.g., Amazon Bedrock's Agents, Vertex AI's Agent Builder) and may further strengthen real-time data access capabilities.
• Salesforce has already launched the "Piper" sales development representative agent, focusing on "whether sales leads are closed" rather than token consumption, representing a shift from "model-driven" to "outcome-driven."

Enterprise Implications: From "AI-First" to "Architecture Sustainability"

For enterprise decision-makers, this debate sends a clear signal: do not be held hostage by vendors' token economics.

1. Redefine AI success metrics: Stop measuring win criteria by model usage rate or token consumption; instead, focus on business outcomes—cost savings, efficiency gains, customer satisfaction, etc. Refer to the practices of three major users (IFS, etc.), which have shifted their gen AI value measurement from "adoption rate" to "business impact."2. Invest in Context Engineering: Enterprises need to build an architecture that can access key data (such as inventory, customer sentiment, production status) in real time, and restrict LLMs to specific tasks through intelligent control. This offers better cost performance than purchasing more expensive models.

3. Scrutinize Vendor Commitments: When procuring AI services, ask vendors how they help reduce token consumption and whether they support hybrid small-model deployment. Avoid falling into a "token sinkhole."

4. Focus on Compliance and Regulation: The EU AI Act is requiring companies to be accountable for AI outputs. Real-time organizational truth helps provide traceable decision-making basis and reduces regulatory risks.

Future Outlook: 12 Months, 24 Months, 3 Years

12 Months: More enterprises will begin building their own "intelligent control" systems, and the deployment share of small models in vertical scenarios will increase. Companies like OpenAI may release more context-specific fine-tuned models to counter the decline in token revenue.

24 Months: Real-time organizational truth becomes a standard for enterprise AI. Data platforms will integrate collaborative data (Slack/Teams) with business system records to provide unified context. The capability gaps of AI agents (e.g., multi-step reasoning and tool invocation) will be narrowed through better context fulfillment.

3 Years: The monopoly of frontier models is broken. Industry-specific agent platforms rise, and AI infrastructure investment shifts from GPU hype to a balance between data pipelines and real-time retrieval. AI regulation emphasizes "process controllability," and enterprises will achieve higher ROI.

As Jon Reed said: "Context progress tells us that customers don't necessarily have to rely on frontier models. They can choose a more sustainable AI path." If enterprises can seize this turning point, the next decade of the AI industry will be more pragmatic and efficient.