Skip to content

feat(low-level-api): Improve API static type-safety and performance - #1205

Merged
abetlen merged 2 commits into
mainfrom
improve-low-level-perf-and-type-safety
Feb 21, 2024
Merged

feat(low-level-api): Improve API static type-safety and performance#1205
abetlen merged 2 commits into
mainfrom
improve-low-level-perf-and-type-safety

Conversation

@abetlen

@abetlen abetlen commented Feb 21, 2024

Copy link
Copy Markdown
Owner
  • Low level api calls now call directly into the loaded dll, not wrapped by a python function first. Reduces overhead however this now requires all low level calls to be positional argument only (this is type checked also).
  • Use NewType to wrap opaque pointers like ggml_context_p and ggml_model_p, improves static type safety

@abetlen
abetlen merged commit 7f51b60 into main Feb 21, 2024
@abetlen
abetlen deleted the improve-low-level-perf-and-type-safety branch February 26, 2024 19:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant