From the DeepSeek-V4-Flash-0731 interpretability project
| # | Question | Records | Status | Answer |
|---|---|---|---|---|
| Exp 1 | Does the verse/prose detector care which translation you use? | 360 | done | No. All verse translations fire near zero. The Message (prose) fires high. Format, not content. |
| Exp 4b | Does the detector fire on verse quotes embedded in prose commentary? | 351 | done | It fires on the prose, not the verse quotes. The model can tell the difference within a single document. |
| Exp 12 | Is H6 actually about digits? Is e164 actually about digits? | 222 | done | H6 doesn't care about digits. e164 does (1,453x difference). Two separate effects. |
| Exp 13 | Does knocking out H6 actually hurt the model on prose? | 480 | staged | Not yet run. The code is ready, the corpus is built, we just need GPU time. |
| Exp 11 | What specific tokens trigger the specialist experts? | — | staged | Not yet run. Needs per-token routing capture. |
| Exp 3 | Is routing religion-specific or just general English? | — | not staged | Compare religious vs. secular text at same digit density. |
The question: We found H6 fires near zero on the KJV Bible. But maybe that's just something weird about the KJV's 1611 English? To test this, we took 30 Bible passages and translated each into 12 different English versions, from the 1611 KJV to the 2002 Message. All with verse numbers stripped, all at 0% digits.
The answer: H6 fired at near-zero on all 11 verse-formatted translations (KJV through NLT). It didn't matter if the English was archaic, modern, literal, or paraphrastic. If it was laid out as verse, H6 stayed quiet. But The Message, which rewrites the Bible as conversational prose, fired at 1,033/M. Same religious content, different format, completely different routing. Full results.
The question: Christian commentary books contain both prose (the author's own writing) and verse (Bible quotes). Does H6 fire on the whole thing, just the quotes, or just the commentary?
The answer: H6 fires on the commentary prose, not on the verse quotes. The correlation between "how much of the text is verse quotes" and H6 firing was -0.13, meaning more verse quotes means slightly less H6 firing. The model can distinguish prose from verse even when they're mixed in the same document. Full results.
The question: Our pipeline stripped digits from the Bible. Was H6 secretly just detecting the absence of digits? And was e164 really a digit detector?
The answer: We took 111 pairs of text where each pair was identical except one version had digits and the other didn't. H6 fired at essentially the same rate on both (1.02x ratio, basically 1:1). e164 fired 1,453x harder on text with digits. Two completely separate things the model detects: verse/prose format (H6) and digit presence (e164). Full results.
The question: H6 correlates with verse/prose format. But does it actually cause anything? If we disable the H6 experts, does the model get worse at processing prose?
The plan: Take 4 groups of text: religious verse, religious prose, secular verse, secular prose (120 passages each). Run the model normally and record its prediction quality. Then disable the H6 experts and run it again. If the model gets significantly worse on prose but not verse, H6 is causally important for prose processing. The code is written and deployed. The corpus is built. We need GPU time to run it. Full design.
Two quick checks we did to make sure our observations weren't bugs: