Say It Four Times

I kept seeing the advice to repeat important instructions in system prompts, and I'd never seen a number for it, so I tested it. Setup: one rule the model can either follow or not (use single quotes, never double quotes), six ordinary Python function tasks, and the only variable was how many times that rule appeared in the system prompt (0, 1, 2, 4, 8, 16). Thirty trials each, 1,080 runs, Gemini 2.5 Flash. Compliance checked with Python's tokenizer, so no model-grading-a-model. Results: 0% when the rule is never stated (171/171 used double quotes), 74% at one mention, 84% at two, 97% at four, then flat (94% at eight, 95% at sixteen). Two things I found more interesting than the headline: The average hides a lot. Two of the six tasks were at 100% from one mention. One was at 20% until four repetitions took it to 97%. Repetition mostly helps where the model's default fights your instruction. Over half the task/condition cells were neither all-pass nor all-fail across thirty identical runs. Non-determinism is large enough that single-run prompt comparisons are basically noise. Note on the source: the paper is Han-yu Wang, "When More Becomes Less: Position-Dependent Repetition Effects in Language Models" (arXiv 2608.04021). It reports two regimes: stacked/adjacent copies climb and plateau, while copies displaced from the readout produce the inverted-U. I ran the adjacent case, so this result matches its prediction rather than contradicting it. The displaced case is the next test. Caveats: one model, one day, one syntactic rule repeated literally with all copies in one place, six small standalone functions. Not state of the art, and it may not survive contact with a real agent loop. Writeup with the chart: https://www.khola.blog/p/say-it-four-times submitted by /u/Fragrant_Offer8745 [link] [comments]