why prompt eval can not reuse

[callMeMakerRen]

hi, guys
from recent test,i found that longer prompt need a lot time to preprocess before it start gen new token, and if you use the same prompt again, it just start gen new token,but if we add a new word prefix the previous prompt ,then again it will process the full prompt , even if it just change a little, i wonder why we need do that,and can we just cache the old process result ,and just process the unknow token,in my example ,should't only the prefix is need process? thx guys

[bullno1]

You can reuse the prefix, just call llama_eval with the right n_past.

cache the old process result ,and just process the unknow token

That's not how LLMs work. Every token depends on all previous tokens.
You can reuse prefix only.