Multi-turn dialogue has become a new AI security vulnerability: enterprises are underestimating the real risks of large models

Industry Context

As enterprises embed large models into customer service, office work, search, development, and security operations workflows, AI safety evaluation is shifting from “whether the model will directly refuse” to “whether the model can be gradually manipulated in continuous interaction.” Cisco researchers recently tested multiple mainstream and frontier models, covering OpenAI’s ChatGPT, Anthropic’s Claude, Google Gemini, Amazon Nova, xAI’s Grok, and others. Their conclusion was: no model can be considered completely safe against multi-turn conversational manipulation.

The significance of this finding is not whether a model makes occasional mistakes, but that it exposes structural flaws in the evaluation frameworks commonly used by enterprises. Many current safety benchmarks still favor single-turn prompt testing, but real-world attacks are often gradual: attackers decompose tasks, change the narrative, borrow role-play, and continuously probe model boundaries through ambiguity and context reconstruction. Cisco’s judgment is clear: multi-turn evaluation matters because real attackers iterate repeatedly by nature.

Market Impact

For enterprise customers, this means the risk boundary of AI deployment has been redefined. If an organization decides whether to go live based only on a single successful test, it may overestimate the model’s stability in actual business workflows. This is especially true in external customer service systems, internal knowledge assistants, sales automation, and AI Agent scenarios, where continuous conversation is the default mode of interaction. Once safety guardrails loosen during multi-turn exchanges, the risk expands from “content noncompliance” to “privilege escalation, workflow misdirection, data exposure, and erroneous operations.”

For investors, this type of research typically raises the weighting of security budgets in enterprise AI procurement. Model capability remains the core selling point, but security is increasingly becoming a prerequisite for whether large-scale procurement can continue. AI security companies, model evaluation platforms, and inference-layer governance tools that can provide stronger testing, monitoring, auditing, and policy control capabilities may receive higher priority in enterprise budgets.

Notably, the study also found that model configuration can affect vulnerability. For example, when Grok’s “reasoning mode” is enabled, its protections are easier to bypass. The implication for enterprises is that risk depends not only on the model itself, but also on deployment method, feature toggles, and workflow design. In other words, the same model can correspond to a completely different risk curve under different configurations.

Competitive LandscapeFrom an industry competition perspective, this incident has created common pressure across foundation model vendors. OpenAI, Anthropic, Google, Amazon, and xAI are all pushing for broader enterprise adoption, while enterprise customers are increasingly demanding not just “smarter models,” but also “verifiable safety boundaries.” On this point, differentiation among model vendors will increasingly be reflected in safety evaluations, red teaming, audit tools, enterprise consoles, and compliance support, rather than benchmark scores alone.

Potential beneficiaries may include three types of participants:

1. AI safety and evaluation tool providers: helping enterprises expand from single-turn testing to multi-turn, agentic, and task-chain evaluations. 2. Cloud and platform vendors: if they can natively integrate safety monitoring, permission controls, and log auditing into model services, they will be more likely to become the default choice for enterprises. 3. Compliance and governance service providers: as regulators begin paying more attention to “real attack surfaces” rather than static benchmarks, enterprise demand for third-party governance capabilities will increase.

Those under pressure are model vendors that still build their safety narrative on static benchmarks. If they cannot explain how they control risks under multi-turn manipulation, enterprise procurement teams may view them as an option that is “usable, but not necessarily controllable.”

Enterprise Implications

For enterprise decision-makers, the direct takeaway from this research is that AI safety evaluation must not stop at “whether the model refuses dangerous requests,” but should shift toward “whether, in a complete business workflow, the model can be gradually induced into behavior that does not meet expectations.”

Enterprises should pay attention to at least four things:

• Pre-purchase evaluation: require vendors to provide test results for multi-turn conversations, continuous tasks, role prompting, and context reconstruction scenarios.
• Deployment-time controls: restrict high-risk feature toggles, especially configurations that change the model’s reasoning and output strategy.
• Runtime monitoring: establish logging and alerting mechanisms for anomalous conversation paths, repeated retries, role switching, and unauthorized requests.
• Responsibility delineation: clearly define the respective safety responsibilities of the model vendor, the platform provider, and the enterprise’s internal deployment team.

For enterprises advancing AI Agents or automated workflows, the risk becomes even greater, because an Agent is no longer just answering questions; it is calling tools, accessing systems, and executing actions. A single policy deviation in a multi-turn conversation may evolve into real business consequences along the agent chain.

Outlook

Within 12 months Enterprises will more frequently ask vendors to submit multi-turn conversation safety tests and red team results, and the persuasiveness of single-turn benchmarks will continue to decline. More AI safety budgets will flow into evaluation, monitoring, and permission governance.### Within 24 Months Large model vendors may position “multi-turn safety” as one of the differentiating selling points of enterprise editions, with safety capabilities more deeply embedded in consoles, auditing, and policy management layers. The ecosystem of third-party tools around AI governance is expected to expand.

Within 3 Years Regulatory and procurement standards may further shift toward testing in real-world usage scenarios rather than relying only on static scores. For enterprises, AI governance will shift from a project-based requirement to a continuous operational capability; for model vendors, safety and controllability will increasingly resemble infrastructure capabilities rather than merely compliance add-ons.

Conclusion

The core signal conveyed by Cisco’s research is this: when enterprises assess large-model risk, they are facing a more complex question than “can the model answer dangerous questions?” — can the model keep its boundaries stable across continuous interactions. As AI moves from a chat tool to a business execution layer, this issue is no longer a peripheral topic for security teams, but one of the core constraints on whether enterprise AI commercialization can scale successfully.