No Bullshit AINo Bullshit AIno-bullshit-aiCannabis Courseby Distru00 XP
Plain English

Where AI Gets It Wrong (and How to Catch It)

AI will confidently invent a Metrc rule, a tag number, or a total. Here is the short list of things to never trust without checking, and the five-second check for each.AI will confidently invent a Metrc rule, a tag number or a total; engineers call this hallucination. Here is the short list of things never to trust without checking, the cheap check that catches each one, and why long chats make it worse.A taxonomy of LLM failure modes (sampling, uncalibrated confidence, tokenized arithmetic, no clock, training cutoff, context saturation) with the deterministic validator that catches each.

7 min read1 giveaway

Questions this answersQuestions this answersfaq

why does ChatGPT make up Metrc rules?why does ChatGPT make up Metrc rules?why does the model hallucinate compliance constraints?
It picks whatever words make the sentence sound right, and there is no step where it stops to look a rule up. So out comes a tidy rule that reads like your state's, sometimes from another state, sometimes from nowhere. Paste the actual bulletin or regulation text, tell it to answer only from that, and ask it to quote the line it used.It picks whatever words make the sentence sound right, and there is no step where it stops to look a rule up. So out comes a tidy rule that reads like your state's, sometimes from another state, sometimes from nowhere. Paste the actual bulletin or regulation text, tell it to answer only from that, and ask it to quote the line it used.No retrieval over an authoritative source sits in the loop, so a plausible span wins over a correct one and the expressed confidence is uncalibrated. Ground it: put the regulation text in context, constrain the answer to provided spans, and require a citation to the span. Check the citation in code rather than trusting the prose.
how do I stop it inventing a Metrc tag number?how do I stop it inventing a Metrc tag number?how do I catch a fabricated identifier before it is used?
It has seen thousands of tags, so it produces one with the right length and the right shape that belongs to nobody. Search your own export for the tag it gave you: not there means it does not exist. Nothing with an ID in it leaves the chat until you have found that ID in a real file.It has seen thousands of tags, so it produces one with the right length and the right shape that belongs to nobody. Search your own export for the tag it gave you: not there means it does not exist. Nothing with an ID in it leaves the chat until you have found that ID in a real file.Surface form is not existence: the model reproduces the shape of a Metrc tag with no set membership behind it. Validate every identifier against the source export in code, as a set lookup, before it reaches a write. Re-asking does not help, since a resample is a new guess, not new information.
why does a long chat start getting things wrong?why does a long chat start getting things wrong?what happens when the context window fills up?
Every message re-sends the whole chat, so a long one gets slower, costs more, and eventually the earliest things you pasted are squeezed out. That is the moment it forgets the price sheet you gave it on Monday and starts guessing, in the same confident voice. Ask for a short brief of what you have established, then paste that into a fresh chat.Every message re-sends the whole chat, so a long one gets slower, costs more, and eventually the earliest things you pasted are squeezed out. That is the moment it forgets the price sheet you gave it on Monday and starts guessing, in the same confident voice. Ask for a short brief of what you have established, then paste that into a fresh chat.The full history is re-sent each turn, and once the window saturates the host truncates or summarizes the oldest turns without telling you. Recall also sags for material buried mid-context. Compact deliberately: ask for a facts-and-decisions brief of a few hundred tokens and seed a new session with it.

Read the rest for freeSign in to read the restauth required past this point

A name and an email, once. The whole course is free, nothing is sold to you, and your place is kept so you can pick it up on your phone later.A name and an email, once. The course is free; the account exists so your progress and your level on the dial follow you between devices, and so you can be issued a certificate at the end.name + email. free. account carries completion state, data-level and certificate issuance.

Sign in or sign upSign in or sign upsign in

We store your name, your email, which lessons you open and which you finish, and the reading level you use. Distru can see that. Nothing is charged and nothing is sold on.Stored: name, email, which lessons you open and complete, and your level on the dial. Distru staff can see it. No payment, no third-party advertising trackers.stored: name, email, per-lesson view + completion, data-level. visible to Distru. no payment, no ad-tech, no third-party pixels.