Tags: hust-wx/llama-cpp-python

LLama: Optimize KV cache management for multi-round conversations

- Implements prefix-matching logic to truncate stale "ghost" tokens in C++ KV cache
- Prevents attention misalignment and context poisoning during multi-turn interactions
- Reduces memory overhead by reusing matched prefixes efficiently

Signed-off-by: JamePeng <jame_peng@sina.com>

LLama: Optimize KV cache management for multi-round conversations

- Implements prefix-matching logic to truncate stale "ghost" tokens in C++ KV cache
- Prevents attention misalignment and context poisoning during multi-turn interactions
- Reduces memory overhead by reusing matched prefixes efficiently

Signed-off-by: JamePeng <jame_peng@sina.com>