[pull] main from abetlen:main - #3

Open

pull[bot] wants to merge 1041 commits into

imotai:mainfrom

This pull request is big! We're only showing the most recent 250 commits

Commits on Oct 22, 2024

feat: Update llama.cpp
abetlen
committed
7403e00

Commits on Oct 31, 2024

feat: Update llama.cpp
abetlen
committed
e712cff

Commits on Nov 15, 2024

feat: Update llama.cpp
abetlen
committed
cafa33e

Commits on Nov 16, 2024

Add missing ggml dependency
abetlen
committed
d1cb50b
Add all missing ggml dependencies
abetlen
committed
2796f4e
chore: Bump version
abetlen
committed
7ecdd94

Commits on Nov 28, 2024

feat: Update llama.cpp
abetlen
committed
f3fb90b

Commits on Dec 6, 2024

feat: Update llama.cpp
abetlen
committed
7ba257e
fix(ci): Explicitly install arm64 python version
abetlen
committed
9d06e36
fix(ci): Explicitly set cmake osx architecture
abetlen
committed
fb0b8fe
fix(ci): Explicitly test on arm64 macos runner
abetlen
committed
72ed7b8
fix(ci): Use macos-14 runner
abetlen
committed
8988aaf
fix(ci): Use macos-13 runner
abetlen
committed
f11a781
fix(ci): Debug print python system architecture
abetlen
committed
9a09fc7
fix(ci): Update config
abetlen
committed
a412ba5
fix(ci): Install with regular pip
abetlen
committed
df05096
fix(ci): gg
abetlen
committed
1cd3f2c
fix(ci): Use python3
abetlen
committed
b34f200
fix(ci): Use default architecture chosen by action
abetlen
committed
d8cc231
fix(ci): Update CMakeLists.txt for macos
abetlen
committed
d5d5099
fix(ci): Remove cuda version 12.5.0 incompatibility with VS (#1838)

pabl-o-ce
and
abetlen
authored
4f17ae5
fix(ci): Remove CUDA 12.5 from index
abetlen
committed
991d9cd
chore(deps): bump pypa/cibuildwheel from 2.21.1 to 2.22.0 (#1844)
dependabot[bot]
authored
2795303
fix: Fix pickling of Llama class by setting seed from _seed member. Closes #1769
abetlen
committed
2523472
Merge branch 'main' of github.com:abetlen/llama-cpp-python into main
abetlen
committed
d553a54
chore(deps): bump conda-incubator/setup-miniconda from 3.0.4 to 3.1.0 (#1821)
dependabot[bot]
authored
ddac04c
fix logit-bias type hint (#1802)
ddh0
authored
fa04cdc
docs: Remove ref to llama_eval in llama_cpp.py docs (#1819)
richdougherty
authored
38fbd29
fix: make content not required in ChatCompletionRequestAssistantMessage (#1807)
feloy
authored
4192210
fix: Re-add suport for CUDA 12.5, add CUDA 12.6 (#1775)

Smartappli
and
abetlen
authored
77a12a3
fix: added missing exit_stack.close() to /v1/chat/completions (#1796)
Ian321
authored
073b7e4
fix: Avoid thread starvation on many concurrent requests by making use of asyncio to lock llama_proxy context (#1798)

gjpower
and
abetlen
authored
9bd0c95
fix(docs): Update development instructions (#1833)
Florents-Tselai
authored
1ea6154
fix(examples): Refactor Batching notebook to use new sampler chain API (#1793)
lukestanley
authored
d610477
fix: chat API logprobs format (#1788)
domdomegg
authored
4f0ec65
misc: Update development Makefile
abetlen
committed
df136cb
Merge branch 'main' of github.com:abetlen/llama-cpp-python into main
abetlen
committed
6889429
misc: Update run server command
abetlen
committed
b9b50e5

Commits on Dec 9, 2024

feat: Update llama.cpp
abetlen
committed
5585f8a
Add CUDA 12.5 and 12.6 to generated output wheels
abetlen
committed
61508c2
chore: Bump version
abetlen
committed
a9fe0f8
fix(ci): hotfix for wheels
abetlen
committed
ca80802
chore: Bump version
abetlen
committed
002f583
fix(ci): update macos runner image to non-deprecated version
abetlen
committed
ea4d86a
fix: add missing await statements for async exit_stack handling (#1858)
gjpower
authored
afedfc8
feat: Update llama.cpp
abetlen
committed
801a73a
chore: Bump version
abetlen
committed
803924b

Commits on Dec 19, 2024

feat: Update llama.cpp
abetlen
committed
2bc1d97

Commits on Dec 30, 2024

feat: Update llama.cpp
abetlen
committed
c9dfad4

Commits on Jan 8, 2025

feat: Update llama.cpp
abetlen
committed
1d5f534
fix: streaming resource lock (#1879)

gjpower
and
abetlen
authored
e8f14ce
chore: Bump version
abetlen
committed
0580cf2

Commits on Jan 29, 2025

feat: Update llama.cpp
abetlen
committed
80be68a
feat: Update llama.cpp
abetlen
committed
0b89fe4
fix(ci): Fix the CUDA workflow (#1894)
oobabooga
authored
14879c7
fix: error showing time spent in llama perf context print (#1898)

shakalaca
and
abetlen
authored
4442ff8
chore: Bump version
abetlen
committed
710e19a

Commits on Mar 12, 2025

feat: Update llama.cpp
abetlen
committed
344c106
feat: Update llama.cpp
abetlen
committed
e232fae
chore: Bump version
abetlen
committed
37eb5f0

Commits on Apr 11, 2025

feat: Update llama.cpp
abetlen
committed
99f2ebf

Commits on May 8, 2025

feat: Update llama.cpp
abetlen
committed
4c6514d
chore: Bump version
abetlen
committed
cb2edb9
hotfix: Disable curl support
abetlen
committed
b1d23df

Commits on Jul 1, 2025

Commits on Jul 3, 2025

Commits on Jul 5, 2025

fix: Update reference to in Llama.embed. Closes #2037
abetlen
committed
9e5a4ea
fix(ci): Update cuda build action to use ubuntu 22.04
abetlen
committed
ae54cde
fix(ci): Add git to package list
abetlen
committed
083fcf6
fix(ci): Remove macos-13 builds to fix cross compilation error
abetlen
committed
11d28df
chore: Bump version
abetlen
committed
1580839
fix(ci): update runners for cpu builds
abetlen
committed
82ad829

Commits on Jul 6, 2025

fix(ci): Update docker runner
abetlen
committed
7011bc1
feat: Update llama.cpp
abetlen
committed
b39e9d4
fix(ci): Temporarily disable windows cuda wheels
abetlen
committed
98fda8c
chore: Bump version
abetlen
committed
8866fbd
fix(ci): Fix macos cpu builds
abetlen
committed
cce4887

Commits on Jul 15, 2025

feat: Update llama.cpp
abetlen
committed
a99fd21
fix: Better chat format for Qwen2.5-VL (#2040)
alcoftTAO
authored
c8579d7
chore: Bump version
abetlen
committed
d9749cb

Commits on Jul 16, 2025

feat: Update llama.cpp
abetlen
committed
95292e3

Commits on Jul 18, 2025

chore: Bump version
abetlen
committed
e1af05f

Commits on Aug 7, 2025

feat: Update llama.cpp
abetlen
committed
4f26028
misc: Update pypi downloads badge
abetlen
committed
d12ca47
misc: Add Python 3.13 classifier tag
abetlen
committed
68e89e8
feat: Add gpt-oss chat format support through strftime_now in chat format by @iamlemec
abetlen
committed
af63792
fix: rename op_offloat to op_offload in llama.py (#2046)
sergey21000
authored
30ddd56
chore: Bump version
abetlen
committed
dfc9bf5

Commits on Aug 15, 2025

feat: Update llama.cpp
abetlen
committed
ce6fd8b
chore: Bump version
abetlen
committed
c37132b

Commits on Mar 22, 2026

Commits on Mar 23, 2026

Commits on Mar 24, 2026

Commits on Mar 25, 2026

Commits on Mar 29, 2026

Commits on Mar 30, 2026

feat(server): add model-load chat_template_kwargs (#2168)
abetlen
authored
7257ba9

Commits on Apr 3, 2026

Commits on Apr 8, 2026

feat: Update llama.cpp to ggerganov/llama.cpp@3bd9aa1f9 (#2176)
abetlen
authored
1bcc5bc

Commits on Apr 13, 2026

feat: Update llama.cpp to ggerganov/llama.cpp@227ed28e1 (#2182)
abetlen
authored
1b1a320

Commits on Apr 27, 2026

feat: Update llama.cpp to ggerganov/llama.cpp@f53577432 (#2189)
abetlen
authored
d87bf08
fix(ci): Build one arm64 py3 release wheel (#2191)
abetlen
authored
511b3f4
chore: bump version to 0.3.21 (#2192)
abetlen
authored
c8075d1
fix(ci): Repair py3 CPU release wheels (#2193)
abetlen
authored
195cc59
fix(ci): Scope CPU release wheel selectors by OS (#2194)
abetlen
authored
d2bcbac
fix(docs): update mkdocstrings inventories config (#2195)
abetlen
authored
c6dc905

Commits on May 2, 2026

Commits on May 4, 2026

fix(_internals): use n_tokens0 offset when enabling last-token logits in add_sequence (#2205)
Anai-Guo
authored
90e8df9

Commits on May 8, 2026

fix: configure n_seq_max for batched embeddings (#2206)
abetlen
authored
128c331
feat: update llama.cpp to 5d6f18a63 (#2207)
abetlen
authored
f774690

Commits on May 11, 2026

Commits on May 13, 2026

fix(embedding): set kv_unified=True when embedding=True to enable batch processing (#2217)

SanjanaB123
and
abetlen
authored
95ccb19

Commits on May 15, 2026

feat: Update llama.cpp to ggerganov/llama.cpp@91e84fed6 (#2218)
abetlen
authored
7664a3e
chore: migrate llama.cpp submodule to ggml-org (#2034)

shalinib-ibm
and
abetlen
authored
c7bea71

Commits on May 18, 2026

feat: Update llama.cpp to b9a2170fc (#2223)
abetlen
authored
5dd9b1c

Commits on May 23, 2026

feat: Update llama.cpp to c0c7e147e (#2228)
abetlen
authored
52fe54b

Commits on May 24, 2026

docs: add contributing guide (#2229)
abetlen
authored
3bda091

Commits on May 31, 2026

feat: Update llama.cpp to d749821db (#2233)
abetlen
authored
2c455a5
Fix: model fails to load when chat template uses HuggingFace generation tags (#2226)

tobocop2
and
abetlen
authored
f160bf7
feat: enable arm64 musl builds (#2221)

acon96
and
abetlen
authored
b91460b
fix: suppress stdout and stderr in Jupyter notebooks (#2181)
Anai-Guo
authored
6bdab5d
fix: avoid cleanup errors for partially initialized LlamaModel (#2173)

usernames122
and
abetlen
authored
fdf38b3

Commits on Jun 1, 2026

feat: add Jinja2 loop controls to chat templates (#2018)

handshape
and
abetlen
authored
e8ee64b
fix: use env var configured multimodal library override paths when loading shared libraries (#1782)

navratil-matej
and
abetlen
authored
5848020
fix: match Transformers `tojson` in chat template rendering (#1486)

CISC
and
abetlen
authored
84bc143
fix: correct typo in comments and settings description (#2121)
thecaptain789
authored
33bf9d2
fix(docs): remove double word typo in README (#1791)

Victoran0
and
abetlen
authored
73ee7cd
fix: clear prompt for recurrent / hybrid models when only a partial prefix matches (#2108)

avion23
and
Ralf Waldukat
authored
cdb7a75
docs: fix typo in README (#2072)
ImadSaddik
authored
e3aa6b5
docs: fix NanoLlava chat handler name in README (#2059)
anakin87
authored
8687122
docs: update llama.cpp build docs link (#2056)
SleepyYui
authored
da07e46
docs: update ROCm install instructions (#1867)
agronholm
authored
52cf747
server types: Move 'model' parameter to clarify it is used (#1786)
domdomegg
authored
c3adb35
feat: update llama.cpp to af6528e6d (#2235)
abetlen
authored
2024060
chore: bump version to 0.3.24 (#2236)
abetlen
authored
26633bd
fix(ci): add Pascal compute capability targets to CUDA wheel builds (#2237)
abetlen
authored
c7af423
feat(ci): add CUDA 11.8 wheel builds (#2238)
abetlen
authored
43c92a7
feat(ci): add CUDA 13 wheel builds (#2239)
abetlen
authored
718a1ca
docs: add Python 3.14 classifier (#2240)
abetlen
authored
927b574
feat: add Gemma 4 multimodal chat support (#2241)
abetlen
authored
a9b480f

Commits on Jun 2, 2026

feat: update llama.cpp to 210a6570c (#2242)
abetlen
authored
4b66c45
chore: bump version to 0.3.25 (#2243)
abetlen
authored
f1bfa11
fix: handle additional `from_pretrained` files in subfolders (#2085)

TNing
and
abetlen
authored
d185d64

Commits on Jun 3, 2026

chore(deps): bump pypa/cibuildwheel from 2.22.0 to 3.4.1 (#2249)
dependabot[bot]
authored
bbdc851
chore(deps): bump actions/cache from 4 to 5 (#2248)
dependabot[bot]
authored
dad5d0a
ci: cache embedding test model (#2250)
abetlen
authored
aa944e4
chore(deps): bump actions/upload-artifact from 4 to 7 (#2245)
dependabot[bot]
authored
b439a84
chore(deps): bump docker/setup-buildx-action from 3 to 4 (#2246)

dependabot[bot]
and
abetlen
authored
f8bd67d
chore(deps): bump actions/setup-python from 5 to 6 (#2247)

dependabot[bot]
and
abetlen
authored
6e6c4e6
feat(ci): add Vulkan wheel builds (#2251)
abetlen
authored
ab7a9b0
feat(ci): add ROCm wheel builds (#2252)
abetlen
authored
3754c04
feat: update llama.cpp (#2253)
abetlen
authored
ddaac10
feat: update llama.cpp (#2254)
abetlen
authored
8d2d269
feat: update llama.cpp to ggml-org/llama.cpp@94a220cd6 (#2255)
abetlen
authored
cc2efc5

Commits on Jun 4, 2026

feat: update llama.cpp to ggml-org/llama.cpp@e3ba22d6c (#2262)
abetlen
authored
e2d148a
chore(deps): bump actions/download-artifact from 4 to 8 (#2257)
dependabot[bot]
authored
d099001
chore(deps): bump actions/checkout from 4 to 6 (#2258)

dependabot[bot]
and
abetlen
authored
df45432
chore(deps): bump docker/login-action from 3 to 4 (#2259)

dependabot[bot]
and
abetlen
authored
b46bccf
chore(deps): bump actions/deploy-pages from 4 to 5 (#2260)

dependabot[bot]
and
abetlen
authored
927dde2
chore(deps): bump conda-incubator/setup-miniconda from 3.1.0 to 4.0.1 (#2261)

dependabot[bot]
and
abetlen
authored
9f6efb0
feat: Generic Multimodal Chat Handler (#2256)
abetlen
authored
2dae477
chore(deps): bump docker/build-push-action from 6 to 7 (#2263)
dependabot[bot]
authored
8edcd15
chore(deps): bump actions/upload-pages-artifact from 3 to 5 (#2264)

dependabot[bot]
and
abetlen
authored
64c0175
chore(deps): bump actions/configure-pages from 5 to 6 (#2265)

dependabot[bot]
and
abetlen
authored
6bccad5
chore(deps): bump softprops/action-gh-release from 2 to 3 (#2266)

dependabot[bot]
and
abetlen
authored
23fe09f
chore(deps): bump docker/setup-qemu-action from 3 to 4 (#2267)

dependabot[bot]
and
abetlen
authored
9013c1d

Commits on Jun 5, 2026

feat: update llama.cpp to ggml-org/llama.cpp@7c158fbb4 (#2268)
abetlen
authored
c2e22ae
chore: bump version to 0.3.26 (#2269)
abetlen
authored
5151ac7
fix(ci): repair release wheel workflows (#2270)
abetlen
authored
78ac75e
fix(ci): allow empty wheel indexes (#2271)
abetlen
authored
7c86eae
fix(ci): index all CUDA wheel variants (#2272)
abetlen
authored
6721989
fix(ci): build one riscv64 release wheel (#2273)
abetlen
authored
4684985
docs: add Gemma 4 Colab notebook (#2274)
abetlen
authored
8949066
docs: fix Gemma 4 Colab notebook (#2275)
abetlen
authored
7a2a36d

Commits on Jun 6, 2026

docs: add Gemma 4 QAT Colab notebook (#2276)
abetlen
authored
7f16fe1
feat: update llama.cpp to 5a69c9743 (#2277)
abetlen
authored
ed83366

Commits on Jun 7, 2026

Commits on Jun 8, 2026

Commits on Jun 9, 2026

Commits on Jun 10, 2026

feat: update llama.cpp to ggml-org/llama.cpp@76da2450a (#2295)
abetlen
authored
b5eefc8

Commits on Jun 11, 2026

feat: update llama.cpp to ggml-org/llama.cpp@ac4cddeb0 (#2297)
abetlen
authored
19ea70c

Commits on Jun 12, 2026

feat: update llama.cpp to ggml-org/llama.cpp@3e7bd4f39 (#2298)
abetlen
authored
65b50ca

Commits on Jun 13, 2026

Commits on Jun 14, 2026

fix(ci): use C++ compiler for Docker builds (#2304)
abetlen
authored
3850aff

Commits on Jun 15, 2026

Commits on Jun 16, 2026

feat: update llama.cpp to e3a74b299 (#2310)
abetlen
authored
822146b
feat: add Pyodide wheel support (#2309)
abetlen
authored
a804233
chore: bump version to 0.3.30 (#2311)
abetlen
authored
ddb6a05

Commits on Jun 20, 2026

feat: update llama.cpp to f449e0553 (#2312)
abetlen
authored
7440aaa
chore: bump version to 0.3.31 (#2317)
abetlen
authored
b11fe07

Commits on Jun 22, 2026

fix: preserve recurrent/hybrid model state when the full prompt is already cached (#2306)

allthatido
and
Ankur Kaul
authored
9be3cd1

Commits on Jun 23, 2026

feat: update llama.cpp to 92e854ab8 (#2318)
abetlen
authored
4bee85b

Commits on Jun 25, 2026

feat(example): support chained NextN server MTP (#2319)
abetlen
authored
4ff48f0

Commits on Jun 28, 2026

feat: update llama.cpp to b3fed31b9 (#2322)
abetlen
authored
f0bf326

Commits on Jun 29, 2026

chore: bump version to 0.3.32 (#2325)
abetlen
authored
346853c

Commits on Jul 5, 2026

feat: update llama.cpp to 78d2f5246 (#2327)
abetlen
authored
7916619
chore: bump version to 0.3.33 (#2329)
abetlen
authored
e894f0d

Commits on Jul 12, 2026

feat: update llama.cpp to e3546c794 (#2338)
abetlen
authored
7e3e889
chore: bump version to 0.3.34 (#2337)
abetlen
authored
629bd1b

Commits on Aug 10, 2026

feat: update llama.cpp to 936918514 (#2348)
abetlen
authored
67014fd

Commits on Aug 15, 2026

feat: update llama.cpp to adb55e514 (#2357)
abetlen
authored
8a0810f

Commits on Aug 16, 2026

fix(example): retain recurrent state for MTP rollback (#2358)
abetlen
authored
987e3df

Commits on Aug 17, 2026

feat: update llama.cpp to 4df29be4f (#2359)
abetlen
authored
03307d8
fix(server): show falsey defaults in CLI help (#2355)

authored
d8a4a1e
fix(llama): release a vision chat handler's mtmd context on close (#2342) (#2343)

authored
9dd3950
chore: bump version to 0.3.35 (#2360)
abetlen
authored
3691546