Prompt to Reduce Token Usage Without Losing Quality

A production agent system was processing 10,000 requests per day. The system prompt was 1,800 tokens. Each response averaged 650 tokens. The output token cost per request was reasonable. The input token cost was not: 1,800 tokens multiplied by 10,000 requests is 18 million tokens per day, just for the system prompt, which is re-sent on every call.

A prompt audit found that 680 tokens of the system prompt were preamble explaining context the model already knew, restated rules, and natural-language prose that conveyed the same information as a structured list in fewer tokens. Removing those tokens cut the system prompt to 1,120 tokens. That change alone reduced daily input token usage by 6.8 million tokens — at no degradation in output quality.

Token optimization is often framed as a quality tradeoff. It is not. The tokens that drive quality are specific instructions, examples, and constraints. The tokens that do not drive quality are filler, redundancy, and natural-language restatements of structured information. Removing the second category is pure savings.

Where Tokens Are Wasted

A systematic token audit of production system prompts finds the same categories of waste repeatedly:

Output tokens have a different waste profile. The most common sources of unnecessary output tokens:

The System Prompt Optimization Prompt

Run this on your existing system prompt to identify and remove token waste:

Analyze the following system prompt for token waste. For each section:

1. Identify whether the section contains:
   a) Novel information the model needs (keep)
   b) Restatement of something already said (remove or merge)
   c) Explanation of things the model already knows (remove)
   d) Natural-language prose that conveys structured information (convert to list/table)
   e) Hedging or qualifiers on instructions that should be direct (remove qualifiers)

2. Rewrite the prompt applying these rules:
   - Every rule should appear once, stated directly, without qualification unless the qualification is material
   - Prose paragraphs describing behavior should be converted to numbered lists
   - Format specifications should use a template, not a description
   - Context the model already possesses (how JSON works, what markdown is) should be removed
   - Preamble paragraphs that describe what the prompt is about should be removed
   - Instructions should say what to do, not why the instruction exists

3. Output:
   - Original prompt (for reference)
   - Optimized prompt
   - Token count estimate for each
   - List of what was removed and why

[PASTE YOUR SYSTEM PROMPT HERE]

The Output Brevity System Prompt Addition

Add this to the top of any system prompt where output token volume is a cost concern:

## OUTPUT FORMAT RULES

Apply these formatting rules to every response:

1. Do not begin responses with affirmations ("Sure", "Great", "Absolutely", "Of course", "Happy to help").
2. Do not echo or restate the question before answering it.
3. Do not summarize the response at the end. If information is worth repeating, it was not clear enough the first time.
4. Use lists for three or more parallel items. Do not write them as a comma-separated sentence.
5. Use a table when comparing three or more things across the same attributes.
6. Do not add closing phrases ("I hope this helps", "Let me know if you need anything else", "Feel free to ask follow-up questions").
7. Filler phrases to never use: "It's worth noting", "It's important to remember", "Keep in mind that", "One thing to consider", "Moving on to", "In conclusion", "To summarize".
8. When the answer to a question is a single fact, provide that fact in the first sentence. Context and explanation may follow, but the answer should not be buried.

These rules apply to all responses. They are not a stylistic preference — they are instructions.

Why These Specific Instructions Work

The key distinction in token optimization is between tokens that carry information and tokens that perform social function. Affirmations, closings, and transition phrases are social lubricant in human conversation. They are pure overhead in a model response that will be processed programmatically or read by a professional user who wants information, not rapport.

Rule 1 (no affirmations) targets the single most reliable token waste pattern. Almost every default model response begins with a social affirmation. "Sure, I'd be happy to help" is five tokens that convey nothing. Removed from 10,000 responses per day, that is 50,000 tokens of pure savings.

Rule 3 (no closing summaries) targets the second largest waste pattern. Summaries that restate the content of the response add tokens without adding value. The reader just read the content. Restating it serves the writer, not the reader.

Rules 4 and 5 (lists and tables) are quality improvements as well as token savers. A table conveying the same information as three prose paragraphs typically uses 30-40% fewer tokens and is faster to parse.

Reducing System Prompt Size Without Reducing Effectiveness

Three structural techniques reduce system prompt token count significantly:

Technique 1: Template over description. Instead of describing the output format in prose, show a template. A 200-word description of how to format a progress report uses more tokens than a 40-token template that shows the format directly. The template also produces more consistent outputs.

# Before (verbose, ~50 tokens):
When reporting on progress, you should include the current status, what has been completed so far, and what remains to be done. Format this as a brief summary.

# After (template, ~25 tokens):
Progress format:
Status: [active/blocked/complete]
Done: [list]
Remaining: [list]

Technique 2: One rule, one line. Each behavioral rule should be one line. If a rule takes a paragraph to explain, it is either doing too much work or it is explaining reasoning that the model does not need. State the rule. Trust the model to apply it.

Technique 3: Remove knowledge the model already has. Many system prompts include explanations of concepts the model knows: what an API is, how markdown works, what a user persona is. These tokens are wasted. Only include concepts that are novel to the specific context.

Measuring the Impact

Use this process to measure before and after:

1. Count system prompt tokens using your model's tokenizer
2. Run 20 representative requests and average output token count
3. Calculate: (system_tokens + avg_output_tokens) * requests_per_day * cost_per_token
4. Apply optimizations
5. Re-run the same 20 requests and compare output quality manually
6. Calculate savings

In practice, well-executed optimizations reduce system prompt tokens by 25-45% and output tokens by 20-35%. Combined, on high-volume agents, this translates to meaningful monthly cost reductions with no quality loss if the right tokens are removed.

For related techniques on managing response length specifically, see Prompt to Get Shorter Responses From Any Model. For eliminating disclaimer overhead specifically, see Prompt to Stop AI Adding Unnecessary Disclaimers.