Every request to an AI is paid for by the token. Shortening a prompt is easy. Knowing whether the shorter one still works is the part people skip.
This is a test bench for that question. It writes the same instruction six ways, gives each to three AI models many times, marks every answer by fixed rules, and reports one number: tokens per correct answer. A wrong answer costs its tokens and counts for nothing, so cutting too far is punished as hard as waste.
All models and all tasks together. The instruction column is the length of the instruction alone. The last column is the measure: everything spent, divided by the answers that were right. Lower is better.
| Way of writing the instruction | Instruction, tokens | Answers right | Tokens per correct answer |
|---|---|---|---|
| Mixed: plain direction plus the answer's shape | 33 | 59% | 179 |
| Outline with the answer's shape | 29 | 55% | 182 |
| Short plain lines | 31 | 47% | 191 |
| Abbreviated | 34 | 42% | 232 |
| Polite prose | 92 | 49% | 304 |
| Function signature | 42 | 36% | 350 |
The top three are close. The clear results are at the bottom: polite prose costs about three times the tokens to say the same thing, and the two forms that look most efficient to a programmer, abbreviation and code, did worse than plain short English.
One model, asked to pull a date and an amount out of an invoice line and return them as JSON. The only difference is how it was told not to add anything else.
JSON only.0%usable. It found the right values every time, then wrote "Here is the response" in front of them, which breaks any program reading the answer.Please give me your answer as a JSON object ... and please do not add any other words or explanation.100%usable. Longer, and on this model every extra word was doing work.The model did not change between those two columns. What it was given did. That is the whole argument of What I put in, as a measurement.
Split by kind of task, the best form changes. That is the practical finding: choose the form to suit the job, and measure, because the right choice is not the one that looks shortest.
| Kind of task | Best way of writing | Tokens per correct answer | Worst that still worked |
|---|---|---|---|
| Classify a review | Short plain lines | 60 (99% right) | Function signature: 329 |
| Pull fields from an invoice line | Mixed: plain direction plus the answer's shape | 145 (75% right) | Abbreviated: 379 |
| Do a sum | No way of writing worked. See finding 4. | 1% at best | |
| Turn a note into a record | Outline with the answer's shape | 153 (71% right) | Polite prose: 376 |
Each was checked by reading the wrong answers themselves before it was written down here.
Given a function signature and asked for the sentiment of a review, two of the three models produced a sentiment program instead of an answer: 0% and 8% right. An instruction that looks like something to be built gets built.
"Clsfy rvw sntmnt" is shorter on the page than "Classify the review's sentiment". It cost more tokens, because a model splits unfamiliar spellings into more pieces, and it scored 42% against 47%. What looks minimal to a person is not minimal to a model.
The comparison above. A short "JSON only." was ignored by one model every time; the long polite version was obeyed every time. Cutting words is only a saving while the answer stays usable.
Asked for a sum and told "number only", the models were right 1% of the time in every form. When they answered with a bare number they were right 1% of the time. When they broke the rule and showed their working, the right figure was in the answer 55% of the time. Forbidding the working removed the tokens that were doing the job. A sum belongs in code, not in a prompt.
Three small stages. The output of each is the input of the next, and only the first one uses an AI. The thing being tested is never the judge.
| Stage | Goes in | Does the work | Comes out |
|---|---|---|---|
| Answer | A task, and one way of writing its instruction | The AI model | An answer, with the exact count of tokens used |
| Mark | The answer, and the right answer already known | Fixed rules, covered by 65 tests | Right or wrong |
| Report | Every mark and every token count | Arithmetic | Tokens per correct answer |
The marking is strict on purpose. If the instruction said "JSON only" and the answer has a sentence in front of it, that is a fail, because a program reading the output would fail too. Each task is asked five times in each form, with fixed seeds so the run can be repeated exactly.
It is a real result for these models and these tasks. It is not a law.
llama3.1:8b, llama3.2:3b, qwen2.5-coder:7b, all run locally. Larger commercial models follow instructions better and would likely narrow the gaps. The ranking may differ on them.
Enough to show the method works and to find the effects above. Not enough to claim a general rate for any way of writing.
Each way of writing was tried with one phrasing per kind of task. A different phrasing of the same form could score differently, which is itself the point.
The next version feeds the result back: shorten a prompt, measure it, keep the change only if the cost per correct answer falls, and judge on tasks it was not tuned on.
The model did not change. The input did.
People say AI produces rubbish. Sometimes it does. This page is a measurement of how much of that is decided before the model reads a word of the task.