Problem statement
When integrating Pullfrog into enterprise environments or customized AI stacks, teams frequently prefer routing their LLM requests through an internal API gateway or a unified proxy layer. Currently, Pullfrog supports fixed providers (like OpenAI, Anthropic, or OpenRouter), but lacks a generalized, plug-and-play integration for LiteLLM.
Without this, users who leverage LiteLLM to handle failovers, load-balancing, spend tracking, or custom self-hosted models (e.g., via vLLM or Ollama hosted internally) cannot easily connect Pullfrog to their infrastructure without hacky workarounds or treating it strictly as a pseudo-OpenAI endpoint (which can break provider-specific parameter schemas).
Proposed solution
Add native support for LiteLLM as a Bring Your Own Key (BYOK) provider.
This involves:
Adding litellm to the allowed list/enum of providers in Pullfrog's configuration schemas (e.g., Zod schemas).
Implementing a dedicated LiteLLMProvider that respects LiteLLM's unified routing capabilities.
Allowing the configuration of a LITELLM_BASE_URL (or proxy endpoint) alongside standard API credentials, giving users complete flexibility over their custom gateway routing and dynamic model passing.
Use case
as the rest of providers
Alternatives considered
No response
Additional context
No response
Problem statement
When integrating Pullfrog into enterprise environments or customized AI stacks, teams frequently prefer routing their LLM requests through an internal API gateway or a unified proxy layer. Currently, Pullfrog supports fixed providers (like OpenAI, Anthropic, or OpenRouter), but lacks a generalized, plug-and-play integration for LiteLLM.
Without this, users who leverage LiteLLM to handle failovers, load-balancing, spend tracking, or custom self-hosted models (e.g., via vLLM or Ollama hosted internally) cannot easily connect Pullfrog to their infrastructure without hacky workarounds or treating it strictly as a pseudo-OpenAI endpoint (which can break provider-specific parameter schemas).
Proposed solution
Add native support for LiteLLM as a Bring Your Own Key (BYOK) provider.
This involves:
Adding litellm to the allowed list/enum of providers in Pullfrog's configuration schemas (e.g., Zod schemas).
Implementing a dedicated LiteLLMProvider that respects LiteLLM's unified routing capabilities.
Allowing the configuration of a LITELLM_BASE_URL (or proxy endpoint) alongside standard API credentials, giving users complete flexibility over their custom gateway routing and dynamic model passing.
Use case
as the rest of providers
Alternatives considered
No response
Additional context
No response