AI thinks faster — context halved, cache doubled

Leaner Context, Smarter Caching

The AI agent's internal "working memory" has been completely restructured. Previously, each request carried 40-60K tokens of context — sensor logs, event traces, redundant metadata. Now it's trimmed to ~20K without losing any decision-relevant information.

What You'll Notice

• Faster responses — Less context to process means the AI reaches decisions sooner
• Lower running costs — Fewer tokens per request translates directly to lower API bills
• More consistent latency — Cache hits skip re-processing of stable prompt sections, making response times more predictable
• Smarter background monitoring — The robot's background awareness no longer wastes cycles on empty sensor batches or duplicate speech events

Under the Hood

• Prompt prefix stabilization — The message layout is reordered so system instructions, conversation history, and summaries form a stable prefix that the LLM can cache across requests. Only dynamic sensor data varies per call.
• Event text compression — ASR events reduced 92% (795 → 60 chars), action completion events stripped of redundant results the AI already knows
• Intelligent filtering — Empty monitoring cycles are skipped entirely; duplicate speech recognition events are deduplicated before entering the queue
• Background subagent optimization — Richer context sync (10 turns instead of 2), faster decisions (1-2 rounds), no more unnecessary task updates

Real-World Impact

On a typical guided tour session:

• ~100K tokens saved from eliminated empty monitoring cycles
• ~4K chars saved per request from stripped ACP events
• Cache hit rate improved from ~30% to 60%+ thanks to stable prefix ordering
• Background decisions complete in 1-2 rounds instead of 3-5