# sop-answers

version 1.0 · 2026-09-25 · MIT · from Distru's No Bullshit AI Course (module 11, lesson 3).

Ask your SOP binder a question in plain words and get the steps back, with the passages they came from. About 160 lines of Python, standard library only. It is a starting point to read and adapt, not a product.

## Files

- `sop-answers.py`: `index` a folder of `.md` or `.txt` files, then `ask` questions.
- `sample-binder/`: six made-up SOPs for a made-up shop, so you can watch it work before you use your own.
- `questions.csv`: ten staff questions, each with the SOP that answers it, for `eval`.

## Run

```
export OPENROUTER_API_KEY=...            # openrouter.ai/keys
python3 sop-answers.py --self-test        # offline checks, no key needed
python3 sop-answers.py index sample-binder/
python3 sop-answers.py ask "what do I do with moldy flower?"
python3 sop-answers.py ask "what time does the store open on Sundays?"   # not in the binder: it should say so
python3 sop-answers.py ask "a customer returned a leaking cart" --dry-run  # see what it found and the prompt, no answer call
python3 sop-answers.py eval questions.csv  # did search find the right SOP for each question?
```

Indexing the sample costs a small fraction of a cent. Each question costs one embedding (next to nothing) plus one answer from the model, usually well under a cent.

## How it works

1. **Index, once.** Each file is split into paragraph-sized passages. Each passage is turned into a list of numbers (an embedding) with its SOP title in front, and saved to `sop-index.json`.
2. **Ask.** Your question is embedded the same way, and the four closest passages are picked.
3. **Answer.** Only those passages go to the model, with instructions to answer from them alone, cite them like [1], and say "The binder does not cover this." when they do not.

## Before you point it at your own binder

- **Your documents leave the building.** Passages go to OpenRouter for embedding, and the ones it picks go to the answering model. Read the provider's data terms, and leave out anything with customer names, license numbers or package tags.
- **Re-index when an SOP changes.** The index is a snapshot. An old index answers from old rules.
- **Check it on questions you know the answer to.** Replace `questions.csv` with twenty or more real questions in your staff's words and the file that answers each, then run `eval`. The sample scores 10 of 10 on a six-SOP binder, which is easy; your real binder will miss some, and the misses are the useful part. If the right SOP is not in the four it found, the answer cannot be right either.
- **It is not a compliance officer.** It repeats your binder. If your binder is wrong or out of date, so is the answer.

## Settings

Environment variables: `EMBED_MODEL` (default `openai/text-embedding-3-small`), `ANSWER_MODEL` (default `anthropic/claude-sonnet-5`), `SOP_INDEX` (default `sop-index.json`). If you change `EMBED_MODEL`, rebuild the index; it refuses to mix models.