AI & ML

AI Chatbot Security: Prompt Injection, Data Leaks, OWASP 2026

In this article
  1. Key takeaways
  2. Why is AI chatbot security different from normal web security?
  3. What changed in the OWASP Top 10 for LLM Applications 2026?
  4. How does prompt injection reach a website chatbot?
  5. How do chatbots leak customer and business data?
  6. How much should a chatbot be allowed to do?
  7. Which controls fit which kind of chatbot?
  8. What should you ask a chatbot vendor or development partner?
  9. Frequently Asked Questions
  10. Sources

Short answer: AI chatbot security starts from one assumption: the model can be tricked, so it must never hold the keys. Keep secrets out of its prompt, filter what it retrieves by the visitor's own permissions, give it only narrow tools, require human approval for risky actions, and log what it does. Test against the OWASP Top 10 for LLM Applications 2026.

Key takeaways

  • Prompt injection still ranks first in the OWASP Top 10 for LLM Applications 2026, and OWASP says no reliable prevention exists today, so design to limit damage.
  • Sensitive information disclosure ranks second: most chatbot data leaks come from what the bot is allowed to retrieve, log or send to a provider, not from clever hacking.
  • Excessive agency jumped to third place in 2026. The more a chatbot can do (refunds, bookings, emails), the more an attacker can make it do.
  • Authorization belongs in your application code, never in the system prompt.
  • Be wary of any vendor that claims its product stops prompt injection completely.

Why is AI chatbot security different from normal web security?

A normal web form has a clear line between code and data. A large language model does not. Everything it reads, including your instructions, the visitor's message, a retrieved help article or a product review, arrives as one stream of text, and the model decides what to follow. The UK National Cyber Security Centre makes this point directly: unlike SQL injection, which parameterized queries fixed, prompt injection may never be fully mitigated because current models do not enforce a boundary between instructions and data, per NCSC: Prompt injection is not SQL injection (2025).

So you cannot buy a filter and call the chatbot safe. You make it safe by deciding what it may see and do, and enforcing that outside the model.

What changed in the OWASP Top 10 for LLM Applications 2026?

The OWASP GenAI Security Project published the 2026 edition in August 2026, replacing the 2025 list. Its preface says the ranking now blends a practitioner vote (75% of the weight) with incident data (25%), drawn from 7,714 collected real-world incidents of which 6,639 had enough detail to classify, according to the OWASP GenAI LLM Top 10 2026 source files (2026).

2026 rankRiskWhat it looks like on a business website LLM01Prompt InjectionA visitor, or text hidden in a page or file the bot reads, overrides its instructions LLM02Sensitive Information DisclosureThe bot reveals another customer's order, internal pricing notes or staff details LLM03Excessive AgencyA support bot with refund or email tools is talked into using them LLM04Supply ChainA plugin, model or third-party tool server you rely on is compromised LLM05Data and Model PoisoningTampered documents in the knowledge base steer answers LLM06Unbounded ConsumptionScripted abuse runs up your API bill or slows the service LLM07MisinformationThe bot states a wrong policy, price or legal position as fact LLM08Hidden Context ExposureUsers extract the system prompt, tool schemas or retrieved policy text LLM09Vector and Embedding WeaknessesA shared search index returns documents the user should not see LLM10Improper Output HandlingThe bot's reply is inserted into a page or query without escaping Two moves matter most for buyers. Excessive Agency rose from sixth to third, and System Prompt Leakage was renamed Hidden Context Exposure and widened to cover retrieved documents and tool definitions, as the Cloud Security Alliance comparison of the 2025 and 2026 lists (2026) sets out. If a vendor's security report still cites only the 2025 IDs, ask them to map findings to the 2026 numbering.

How does prompt injection reach a website chatbot?

There are two routes. Direct injection is a visitor typing instructions meant to override yours. Indirect injection is more dangerous for business sites: the instructions sit inside content the bot reads, such as a product review, an uploaded PDF, an inbound email or a web page fetched by a research feature, and the visitor may never see them. The 2026 OWASP entry also covers instructions hidden in images and audio, and invisible Unicode characters, in its LLM01:2026 Prompt Injection entry (OWASP, 2026).

That entry is blunt: there is no reliable prevention today, and defense depends on an architecture that limits what a misled model can do. Defensive layers still help:

  • Label untrusted content. Pass retrieved text to the model in a clearly marked data section, separate from instructions. NCSC notes this makes attacks harder, while warning that blocklists of phrases are easy to rephrase around.
  • Validate the output in code. If the bot should return an answer and a link, check that shape with ordinary code before anything is shown or executed.
  • Strip invisible characters from uploaded and retrieved text before the model reads it.
  • Red-team it. Test with people who assume attackers know your defenses, and repeat after every significant change.

How do chatbots leak customer and business data?

In most cases the model retrieves data it was never meant to have. The OWASP sensitive-disclosure entry lists the common paths: search indexes that ignore user permissions, logs and tracing tools that capture full prompts and replies by default, and integrations that send whole customer records to an external provider when one field would do, per LLM02:2026 Sensitive Information Disclosure (OWASP, 2026).

Controls worth insisting on

  1. Filter retrieval by permission before the model sees anything. A logged-in customer's question should only search that customer's records and public content.
  2. Send the minimum to the model provider. Strip identifiers that the task does not need, and confirm the provider's retention and training settings in the contract and the account configuration, not only in a policy page.
  3. Scrub logs. Decide who can read chat transcripts and how long they are kept, and mask card numbers, passwords and health details.
  4. Keep secrets out of the prompt. OWASP's Hidden Context Exposure entry says to assume users can see whatever the model can see, so API keys, database details and internal role rules never belong there.

Data protection rules such as GDPR and UK GDPR still apply to chatbot transcripts and to what you share with an AI provider. Check your lawful basis, processor agreements and retention periods with a qualified privacy adviser.

How much should a chatbot be allowed to do?

As little as the job needs. OWASP's 2026 Excessive Agency entry traces the risk to three root causes: tools with more functions than the task requires, tools with more permissions than required, and high-impact actions that run without independent checks, per LLM03:2026 Excessive Agency (OWASP, 2026).

A useful rule of thumb, which OWASP's 2026 prompt injection entry calls the Rule of Two: when one feature combines untrusted input, access to sensitive data, and the ability to change something or contact the outside world, each action needs human approval or a formal risk assessment. In practice:

  • A bot that answers questions from public help articles carries modest risk.
  • A bot that looks up a signed-in customer's orders should run with that customer's own scoped credentials, not an admin key.
  • A bot that issues refunds, changes bookings or sends email should prepare the action and ask a person, or the customer, to confirm the exact action.

Add rate limits and spending caps per user and per session as well. Unbounded Consumption moved up four places in the 2026 list, from tenth to sixth, and a public chatbot without limits is an open invitation to run up your bill.

Which controls fit which kind of chatbot?

Chatbot typeMain risksMinimum controls FAQ bot on public contentMisinformation, injection via content, cost abuseCurated sources, answer-with-citation, rate limits, output escaping Signed-in customer assistantData disclosure across accounts, log exposurePermission-filtered retrieval, user-scoped credentials, scrubbed logs Bot with actions (refunds, bookings, CRM updates)Excessive agency, indirect injectionNarrow tools, human confirmation, authorization in code, monitoring Internal staff assistant on company documentsOversharing from unscoped drives, poisoned documentsDocument-level access control, source tracking, review of what gets indexed If you are still planning the feature, our earlier guide to building multilingual AI chatbots for international customers covers scope and language choices. These security decisions belong in the same planning stage, not after launch.

What should you ask a chatbot vendor or development partner?

  1. Which OWASP Top 10 for LLM Applications 2026 risks did you test, and can we see the results?
  2. Where is authorization enforced, and can the model call any tool without a code-level permission check?
  3. What customer data is sent to the model provider, and what are the retention and training settings?
  4. Who can read chat logs, and are sensitive fields masked?
  5. Which actions need human confirmation, and how is that shown to the user?
  6. What rate limits and spending caps are in place?
  7. How will you detect and respond to abuse after launch, and who is on call?

NCSC advises skepticism toward products that claim to "stop" prompt injection and favors suppliers who explain how they reduce its likelihood and impact. At Alive Inc., our AI chatbot and AI agent development work starts from that design-for-failure view, and an independent security audit before go-live is a sensible check on any team's build, including ours.

Frequently Asked Questions

Can prompt injection be completely prevented?

No. OWASP's 2026 guidance says there is no reliable prevention today, and the UK NCSC says prompt injection may never be fully mitigated the way SQL injection was. The realistic goal is to limit impact: keep secrets and authorization out of the model, restrict its tools, and require human approval for consequential actions.

What is the OWASP Top 10 for LLM Applications?

It is a community-maintained list of the most critical security risks in applications built on large language models, published by the OWASP GenAI Security Project. The 2026 edition, released in August 2026, ranks prompt injection first, sensitive information disclosure second and excessive agency third.

Is it safe to put customer data into a website chatbot?

It can be, if retrieval is filtered by each user's permissions, only the needed fields are sent to the model provider, logs are masked and access-controlled, and provider retention settings are confirmed. Check the privacy and contractual side with a qualified adviser for the regions where your customers live.

Should the system prompt be kept secret?

Do not rely on it staying secret. OWASP's Hidden Context Exposure entry says to assume users can see whatever the model sees. Keep credentials, internal rules and permission logic in your application code, so a leaked prompt reveals nothing that helps an attacker.

How often should an AI chatbot be security tested?

Test before launch and again after any change to the model, the prompt, the tools or the data it can reach. Ongoing monitoring of tool calls and unusual conversations matters too, because new injection techniques appear regularly and a static test result ages quickly.

Sources

Share:
Back to Insights

Explore Related Services and Solutions