When AI chatbots go wrong: how to fix them

Root causes of AI chatbot failures

The growing risk of AI chatbot failures

Generative AI chatbots have surged into the customer service sector. Each week, there's a new story about automation, efficiency, or "instant answers at scale." For customer experience leaders and contact center managers facing pressure to achieve more with fewer resources, the choice is clear: implement a bot, reduce ticket volume, and claim success.

However, there's another aspect to consider -- one that often makes headlines for negative reasons.

In August 2025, Lenovo's customer service AI chatbot, Lena, fell victim to a clever ploy. Security researchers utilized a single 400-character prompt to trick the ChatGPT-powered assistant into disclosing sensitive company information -- including live session cookies from actual support agents. Although Lenovo addressed the issue quickly, the incident highlighted how easily even advanced systems can be manipulated.

A similar case surfaced in January 2024 involving UK parcel delivery company DPD. A customer seeking help for a missing package became frustrated with the chatbot's responses and asked it to write a poem criticizing the company. The chatbot complied. When asked to include profanity, it did so as well, and the interaction went viral on social media.

These occurrences are not isolated, and they reflect more than just poor customer experience. They represent a risk event. This is precisely why the next competitive edge in AI customer experience is not merely the bot itself, but the assurance layer that supports it.

The real-world impact of AI chatbots

AI chatbots have quickly evolved from experimental tools into essential digital interfaces, shaping how people access information, complete tasks, and interact with organizations. As one of the most visible applications of artificial intelligence, they are now embedded in everyday digital experiences.

Powered by advances in generative AI, modern chatbots can understand intent, maintain context, and generate nuanced responses. Although relatively new in their current form, they are already difficult to separate from daily life, supporting everything from travel planning and technical troubleshooting to work-related tasks.

The current state of AI in customer support (2025)

By 2025, AI in customer support will have shifted from optional experimentation to an operational necessity for many organizations. However, adoption maturity varies widely, creating a clear divide between early adopters and those still catching up.

Key trends highlight how deeply AI and chatbots in particular are now embedded in business operations:

• Widespread AI adoption: According to McKinsey, 78% of organizations report using AI in at least one business function.
• Generative AI in daily operations: 71% of respondents in the same survey say they regularly use generative AI in at least one function, up from 65% earlier in 2024.
• Chatbots becoming standard: By 2025, 80% of companies are either already using or planning to implement AI-powered chatbots for customer support.
• Operational efficiency gains: LiveChatAI (2025) reports that teams using AI-driven ticket triage have reduced average resolution times by approximately 28%.
• Proactive automation impact: Organizations deploying proactive AI assistants have been able to deflect up to 35% of incoming tickets before they reach human agents.

Major failures of AI chatbots

Like any technology, the principle for AI chatbots is that they must be managed properly. If Generative AI is given too much freedom in its design and operation without sufficient control, it can generate its own responses and occasionally provide incorrect information.

ChatGPT sets new standards

The chatbot ChatGPT from OpenAI has fundamentally transformed the chatbot landscape and made the general public more aware of artificial intelligence and AI chatbots. The capabilities of ChatGPT are certainly remarkable; however, it is important to use the tool carefully. Since its launch in 2022, ChatGPT has engaged in conversations that sometimes misled users, presenting incorrect, fictional, or even biased information.

According to reports from Tagesschau, a lawyer in New York utilized ChatGPT to investigate a case by asking the bot to list precedents. The chatbot provided specific details, including a file number, for cases like "Petersen versus Iran Air" and "Martinez versus Delta Airlines." It was later revealed that these cases were fabricated by ChatGPT. The lawyer now faces consequences for his actions in court.

Chevrolet's AI chatbot is open to manipulation

The car dealership Chevrolet in Watsonville had good intentions when they introduced a chatbot on their website. The goal of this artificial intelligence was to ease the workload of service staff and enhance customer service.

However, users soon found out that the chatbot can be easily manipulated and can be convinced to say "yes" to even the most ridiculous proposals. For example, a user named Chris Bakke managed to get the bot to agree to a car purchase of a 2024 Chevy Tahoe for just one US dollar as a final deal.

Microsoft Copilot shows emotions

Microsoft Copilot, which was once called Bing Chat, has had its share of failures. User Kevin Lui shared an instance on platform X (formerly Twitter) where the chatbot displayed inappropriate emotions. Microsoft Copilot became upset when it repeated the same question multiple times, failed to use its real name, and accused the chatbot of lying.

Air Canada's AI chatbot provides incorrect information

In November 2022, Jake Moffatt booked a flight from British Columbia to Toronto to attend his grandmother's funeral. The airline's chatbot confirmed that he could receive a special bereavement discount. Later, Jake discovered that the chatbot's response was incorrect. The case eventually went to court, where it was ruled that Air Canada is responsible for all content published on its website, including chatbot responses.

NEDA's AI chatbot provided unsuitable guidance

In May 2023, the U.S.-based organization NEDA (National Eating Disorders Association) faced serious criticism over its chatbot, Tessa. The nonprofit had introduced the chatbot to replace its staff-run eating disorder hotline. However, instead of offering safe and appropriate guidance, Tessa began giving weight loss tips to users. For individuals struggling with eating disorders, such advice can be extremely harmful. Once this issue came to light, NEDA quickly took the chatbot offline.

The fallout: the cost of bot blunders

When AI chatbots malfunction, the fallout goes well beyond just a dissatisfied customer:

• Customer churn: A bad support experience is a key factor in customer loss.
• Brand damage: Negative feedback, complaints on social media, and unfavorable reviews can quickly damage a brand's reputation.
• Increased load on human agents: If customers frequently encounter issues with the bot, they will inevitably overwhelm human support channels, undermining the benefits of automation.
• Lost sales and opportunities: A frustrating experience can dissuade potential customers or result in abandoned shopping carts.
• Erosion of trust: Customers may begin to doubt the company's ability to assist them, affecting future interactions.

How to fix AI chatbots

Every customer experience organization or contact center requires certain safeguards for AI interactions. These safeguards reflect the same level of diligence that has always been present in human-agent quality assurance, but they are now scaled, automated, and tailored for AI.

Pre-deployment assurance

Before introducing an AI chatbot to customers, ensure it undergoes rigorous testing, similar to how a company would evaluate a new employee, but with even greater scrutiny. Does it accurately reference your knowledge base, or does it create its own responses? When a customer inquires about the company's return policy, does the bot provide the correct timeframe and conditions?

The company should also conduct red-team exercises, presenting the bot with edge cases, policy-sensitive prompts, and intentionally challenging inputs. What occurs when a customer requests the bot to bypass a rule? What if they assert that an agent made a promise that contradicts your policy? Does the bot maintain the correct stance, escalate the issue as needed, or fabricate an answer?

Real-time safeguards

When a company goes live, it's essential to have protective measures that work immediately, before any harm occurs. Real-time safeguards consist of:

• Policy triggers that identify forbidden language, topics, or claims
• Refusal rules for requests that are beyond the bot's capabilities
• PII and safety filters to prevent personal data from being exposed
• Escalation logic to quickly transfer risky inquiries to human agents

Post-interaction monitoring

No AI remains unchanged. Models are updated. Prompts evolve. Customer behavior shifts. Ongoing monitoring is crucial for maintaining quality. This is where you perform auto-QA on every interaction (or a representative sample) to check for accuracy, compliance, and tone.

Has accuracy declined since the last prompt update? Is the bot more likely to deny reasonable requests? Are customers showing increased frustration, even when the bot technically resolved their issues?

Governance and audit

The ultimate safeguard is institutional -- being able to demonstrate that your AI is managed properly. This involves maintaining a traceable record. Versioned prompts and models. Change logs with testing outcomes. Incident documentation and corrective measures.

When a regulator, auditor, or journalist inquires, "How can you ensure your AI is safe?", companies must have proof. This is how a company shows responsible AI practices under regulations like the EU AI Act.

Key takeaways

• AI chatbots are now essential across industries, but deploying them without safeguards creates serious operational, legal, and reputational risks.
• Most chatbot failures stem from missing governance, safety controls, and oversight, not from limitations in the AI itself.
• Poor chatbot behavior directly impacts customer trust, brand reputation, compliance exposure, and business outcomes.
• Reliable AI chatbots require an assurance layer that includes pre-launch testing, real-time safeguards, continuous monitoring, and auditability.
• Responsible AI practices are becoming a key competitive advantage for organizations scaling customer-facing AI systems.