Skip to content

Commit 6e0642c

Browse files
a-ghorbania-ghorbani
authored
fix: fix logprobs when BOS is not present (abetlen#1471)
* Fix lobprobs when BOS is not present * Fix logprobs when bos is not available
1 parent 027f7bc commit 6e0642c

1 file changed

Lines changed: 2 additions & 2 deletions

File tree

‎llama_cpp/llama.py‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1410,8 +1410,8 @@ def logit_bias_processor(
14101410
top_logprobs: List[Optional[Dict[str, float]]] = []
14111411

14121412
if echo:
1413-
# Remove leading BOS token
1414-
all_tokens = prompt_tokens[1:] + completion_tokens
1413+
# Remove leading BOS token if exists
1414+
all_tokens = prompt_tokens[1 if prompt_tokens[0] == self.token_bos() else 0:] + completion_tokens
14151415
else:
14161416
all_tokens = completion_tokens
14171417

0 commit comments

Comments
 (0)