AI Mistakes & How to Catch Them

A lawyer cited a court case that never existed — and a judge wasn't amused

In 2023, a New York attorney named Steven Schwartz used ChatGPT to research legal precedents for a case. The AI gave him six relevant court cases with names, citations, and even short summaries of the rulings. They sounded perfect. He cited them in his filing.

There was one problem: none of those cases existed. Not a single one. The AI had invented case names, fabricated judges' names, made up rulings, and assigned fake citation numbers — all with absolute confidence.

The judge discovered the fraud. Schwartz, his partner Peter LoDuca, and their firm Levidow, Levidow & Oberman were jointly sanctioned $5,000 by Judge P. Kevin Castel — payable jointly and severally. (The case was Mata v. Avianca, May 2023.) The story made international headlines.

And here's the part that should make you pay attention: Schwartz wasn't lazy. He wasn't cutting corners. He asked AI a question, got an answer that looked completely real, and trusted it. That's exactly what most people do with AI every day.

This module teaches you to never be Steven Schwartz.

The three ways AI will fail you

AI doesn't make random errors like a typo or a math mistake. It makes systematic errors — patterns of failure that are predictable once you know what to look for.

Think of AI as a brilliant intern on their first day

You just hired someone who graduated top of their class. They're articulate, fast, and eager to help. But it's their first day. They've never seen your company before. They've never met your clients. They don't know your industry's unwritten rules.

Would you let this person:

• File a legal brief without checking it? No.
• Send an email to a client without reviewing it? No.
• Make decisions about someone's job or loan application? Absolutely not.

But would you let them draft the brief, write the first version of the email, or compile the data for the decision? Yes — and then you check their work.

That's the right relationship with AI. Use it for the first draft. Verify the final output.

Hallucination: when AI invents facts with a straight face

Hallucination isn't a bug — it's a structural feature of how language models work. The model predicts what words should come next based on patterns. Sometimes the most plausible-sounding next word is wrong. The model doesn't know the difference because it doesn't "know" anything — it generates text.

What hallucinations look like

When hallucinations are most likely

High-risk situations:

• Asking for specific statistics, studies, or research papers
• Asking about obscure topics, small companies, or recent events
• Asking for legal, medical, or financial facts
• Asking about people (especially non-famous individuals)

Lower-risk situations:

• Asking for general explanations of well-known concepts
• Brainstorming and ideation
• Editing and rewriting text you provide
• Summarizing documents you paste in (AI can't hallucinate about text it's looking at)

Common AI output failures (in rough order of frequency in practice):

There Are No Dumb Questions

"If I paste a document and ask AI to summarize it, can it still hallucinate?"

Much less likely, but yes. AI can occasionally misinterpret the document, combine two separate points into one that wasn't made, or state a conclusion more strongly than the source supports. The risk drops dramatically when AI is working from source material rather than its memory — but always spot-check the summary against the original.

"Why can't they just fix hallucination?"

Because hallucination comes from the same mechanism that makes AI useful. The model generates new text by predicting patterns — that's what lets it write, summarize, translate, and create. If you made it only output text it was 100% certain about, it would barely produce anything. The industry is making progress (newer models hallucinate less), but it's a fundamental tradeoff, not a bug to patch.

Bias: when AI reflects unfair patterns

AI learns from human text. Human text contains biases. Therefore AI outputs contain biases. It's math, not malice.

Real examples of AI bias

How to spot bias in your AI outputs

Ask yourself these questions every time AI makes a recommendation, classification, or decision about people:

1. Would this output change if I swapped the person's name, gender, or background? If yes, there's bias.
2. Does this recommendation match a stereotype? If AI is confirming every expectation, it might be reflecting bias, not reality.
3. Who is missing from the output? If AI's "list of experts" has no diversity, that's a signal.
4. Is this too convenient? If AI's analysis perfectly supports what you already believe, ask it to argue the opposite.

There Are No Dumb Questions

"Am I responsible for AI bias in my work?"

Yes, in the same way you're responsible for any tool you use. If you use a calculator and enter the wrong numbers, you can't blame the calculator. If you use AI output that contains bias and it harms someone, the fact that AI generated it doesn't shield you from accountability. You're the human in the loop. That's why you're here.

"Is there a bias-free AI?"

No. Every AI model has biases because every training dataset has biases. The question isn't "is this AI biased?" (it is) but "what biases does this AI have, and how do they affect my specific use case?" Some biases are harmless for your task. Others could be devastating. You need to know the difference.

Confidentiality: what you should NEVER paste into AI

This is the mistake that gets people fired. Not hallucination, not bias — pasting sensitive information into an AI tool.

The red list: never paste these

When it goes wrong: the Samsung incident

The difference between them and you: now you know.

The green list: safe to paste

The decision flowchart

Pro tip: Many companies now offer enterprise AI tools (like ChatGPT Enterprise, Azure OpenAI, or Claude for Enterprise) where your data isn't used for training and stays within your organization's boundaries. Check with your IT department — you might already have access.

The "trust but verify" workflow

Here's the system that protects you from all three failure types:

The 60-second verification checklist

Before you use any AI output for anything important, run through this:

• Facts check: Did I verify any specific claims, statistics, or citations?
• Source check: Can I trace each fact back to a real, primary source?
• Bias check: Would this output change if I swapped demographics?
• Confidentiality check: Did I avoid pasting anything sensitive?
• Smell test: Does anything feel "too perfect" or suspiciously convenient?

This takes 60 seconds. Skipping it can cost you your reputation.

Back to Steven Schwartz

He wasn't lazy. He wasn't cutting corners. He asked a question, got an answer that looked completely real, and trusted it — the same thing most people do with AI every day. The difference is that his mistake was filed with a federal court, where it couldn't be quietly fixed.

What he was missing wasn't intelligence or diligence. It was the 60-second habit: verify the facts, trace the sources, check the output before it goes out the door.

You now have that habit. The next time AI gives you a citation, a statistic, or a legal precedent — you'll know to check it. That's the entire point.

Key takeaways

• AI hallucinates — it invents facts, sources, and statistics that don't exist. The more obscure the topic and the more specific the claim, the higher the risk. Always verify factual claims against primary sources.
• AI reflects biases from its training data. When AI makes recommendations about people (hiring, lending, evaluating), check for bias by mentally swapping demographics.
• Never paste sensitive information into AI tools. Company secrets, personal data, passwords, and legal documents should never go into a consumer AI tool. Use enterprise tools with proper data handling, or anonymize first.
• Trust but verify is not optional — it's your job. The 60-second verification checklist protects your reputation. A brilliant intern's draft still needs your review before it goes out the door.