Why AI system prompts should run in English (even when the agent speaks German)

Sometimes the biggest lever sits exactly where you would never look for it. In one of our AI projects we tested an optimization that feels almost too simple to be true, and that still had one of the clearest effects of the entire project.

The setup

We built a voice agent that handles real phone calls in German. With voice, latency is everything: every tenth of a second between a statement and the reply decides whether a conversation feels natural or robotic. The system prompt, the instruction that tells the model how to behave, was initially written entirely in German.

The finding

We rewrote the system prompt from German to English. Only the internal instruction. The spoken language of the agent stayed German, unchanged. The result: latency dropped by more than half a second. No new model, no more expensive infrastructure, just the language of the instruction.

Why this happens

The effect is no coincidence. Large language models are overwhelmingly trained and optimized on English data. That shows up in several places:

• Token efficiency: English text is split into fewer tokens than German. Fewer tokens in the prompt mean less context to process and therefore a faster first response.
• Instruction fidelity: The fine-tuning (RLHF) of the models happens primarily in English. English instructions are followed more precisely and with less detour.
• Internal reasoning: Models tend to "think" close to English internally. A German instruction adds a translation overhead before any response is even produced.

What this means in practice

The lesson is simple and still rarely applied: separate the language of the instruction from the language of the output. The system prompt can run in English while the output stays cleanly in German. Especially with voice agents, where latency directly determines conversation quality, this is a noticeable difference. But with chatbots and automations too, you save tokens, time and cost.

The LLMs are trained and perfected in English. Ignore that, and you give away speed without noticing.

For us this was one of the aha moments of the project, because it shows how much performance hides in details that seem to have nothing to do with the result. Findings like this emerge when you do not just buy AI, but truly understand it in your core processes.