Your AI is Lying to You…

Sometimes.

By The 1938 Group | 20 May 2026 | Johannesburg

No stuttering. It doesn't stammer. There is no, 'I'm not sure about this one.' It just answers with a confident fluently, an authoritative tone. Sometimes completely wrong.

AI hallucinations are outputs that seem plausible, coherent but are factually fabricated, are not a glitch, they are a structural feature of how large language models work. Right now, your teams are almost certainly using ChatGPT, Microsoft Copilot, Gemini, or Claude without any sort of safety framework in place.

This is Module 1 of the 1938Group AI Usage & Data Safety Enablement Programme. Before your organisation relies on AI for anything that matters, such as a client proposal, a compliance brief, a legal summary. Your teams need to understand what they are actually working with.

What AI Actually Is (And Pretending to Be)

A large language model (LLM) is not a search engine. It does not retrieve facts. There is no database consulted to verify any information. It generates text by predicting, at enormous scale and speed, which words are most likely to follow which other words, based on mathematical patterns learned from billions of documents.

What emerges is text that sounds like a knowledgeable human wrote it. Sometimes it is accurate. Sometimes not so much. The model itself cannot tell the difference between fact or fiction.

This is referred to as the Confidence Problem. A wrong claim and a right claim emerge with identical tone, identical fluency, identical authority. Which is which? Nothing in the output signals will help with clarity. That job belongs to the human using the LLM, and that requires a deliberate process be taken.

The 8 Hallucination Red Flags Your Team Must Know

The following patterns signal a risk in hallucinations. They come directly from the 1938Group AI Usage Policy framework and represent commonly encountered failure modes in a corporate setting.

Red Flags in A.I. Generated Content

None of these red flags mean the output is completely wrong. These red flags confirm that verification is non-negotiable. Let’s have a look at the three that cause the most damage:

1. Invented Academic Citations

In 2023, two US lawyers submitted court filings referencing six cases that did not exist. ChatGPT had generated them (complete with case names, jurisdictions, dates, and rulings). As real as each detail looked. None of it was.

This is a remarkably common pattern. An LLM knows what a citation looks like. It knows the format, the language, the structure. So, when asked a question that warrants a citation, it generates one that fits the pattern, if the source doesn’t exist? It will do so regardless.

Rule: every citation, every paper, every ruling generated by any AI tool must be verified in the original source before use. For legal, HR, compliance, and client-facing work, this is non-negotiable.

2. Confident Claims on Obscure or Technical Topics

The less specialist knowledge you have in an area, the more dangerous AI hallucinations become. A lawyer reviewing an AI-generated legal summary can catch errors that a general manager would miss. An accountant will see the problem in a financial statement that an HR professional would accept.

When AI output enters a territory outside of your area of expertise, a technical, regulatory, sector-specific verification standard goes up, not down. At this moment, an expert review step is required.

3. Information That Seems 'Too Perfect'

This is a more elusive flag to identify. An AI output that perfectly confirms your existing hypothesis, an AI that can answer your questions exactly as you had hoped. An answer that contains no friction, ambiguity, or nuance should be the triggers that activate your scepticism, not your confidence.

LLMs are trained to be helpful. They are designed to give you what the pattern of your question suggests you want to hear. The moment to slow down comes when the output is frictionless.

What Does AI Do Well?

AI tools are genuinely powerful. The goal of the 1938Group programme is not  designed to generate fear around AI, it is designed to help you in building the judgement required to use it well. Here is the honest picture of where AI adds real value and where it needs human oversight to be safe.

AI Use Cases and Responsibility

Notice the pattern: AI accelerates. Humans verify. That division of labour is the foundation of safe, effective AI use. The minute human verification is removed from the chain; the risk profile changes dramatically.

Human Accountability: The Principle That Changes Everything

The 1938Group’s AI Usage Policy is built on six core principles. The first and most foundational is:

This is not a philosophical position. It is a legal and regulatory reality.

'The AI told me’ Is not a defence, not to a client, regulator, litigation, nor before the Information Regulator in a POPIA enforcement action.

When a compliance summary contains hallucinated regulatory requirements, the organisation that submitted it is accountable. When a client deliverable contains fabricated data, the organisation that delivered it is accountable. When an employment decision is made based on AI-generated analysis, the decision-maker is accountable.

Embedding AI into professional workflows without accountability infrastructure is a liability not efficiency.

A POPIA Note for South African Professionals

South Africa's Protection of Personal Information Act places accountability for data processing decisions directly on the responsible party, your organisation. Feeding personal data into a public AI tool like ChatGPT is a data processing decision. Feeding client information into any system without appropriate contractual data protection is a data exposure risk.

Two questions that should be asked by your team before using any AI tool for work involving personal or client data:

1.    Is this tool on our approved list, and what tier of data is it approved to process?

2.    Have I classified this information, and does it meet the threshold for this tool?

The full RED/AMBER/GREEN data classification framework and POPIA compliance guidance are covered in more detail later. But the habit of asking before acting starts from Module 1.

The VERIFY Framework: Six Steps Between You and a Costly Mistake

The 1938Group’s AI Usage Policy mandates the use of the VERIFY framework for all AI outputs. It is a practical habit, not an administrative burden. For most outputs, it takes minutes. For high-stakes work, it is the difference between a professional deliverable and a liability.

Ensuring Accuracy & Reliability

The verification standard scales with the risk or reward possible. For informal internal brainstorming, a quick, reasonable check is sufficient. For client deliverables, published content, regulatory submissions, or decisions affecting people, the full framework applies, and the verification performed should always be documented.

Building the Habit: Changes Needed

The knowledge in this article is of no use unless there is a behaviour change. Here is what safe AI use looks like in practice, starting from today:

1.    Before you use AI on any task, classify the information involved. Is it RED, AMBER, or GREEN? If it is RED, stop.

2.    Before you act on any AI output, run VERIFY. Even a 60-second external check on key claims.

3.    When an AI output is unusually perfect, treat that as a red flag, not a green light.

4.    When you use AI for anything client-facing or compliance-relevant, document it. What tool. What prompts. What verification did you perform?

5.    Remember: the accountability remains with you. The AI is a tool. The user is the professional who will be held accountable.

Follow 1938Group on LinkedIn for the weekly AI-related content, or visit 1938group.com for the full programme guide.