Skip to content

Commit bbfd2f2

Browse files
committed
feat!: simplify model (provider) configuration
- add support for using anthropic models hosted on bedrock and vertex, just install optional dependencies and set environment variables for configuring base url, authentication and authorization etc. - add support for configuring model provider through model registry (key ending with "/") instead of having to configure each model separately for a provider - add `ASKUI__VA__MODEL_PROVIDER` environment variable for setting default model provider - add `ASKUI__VA__MODEL` for setting default model - added providers `"askui/"`, `"bedrock/"`, `"anthropic/"` and `"vertex/"` per default, so you can just pass, e.g., `"askui/claude-sonnet-4-20250514"`, or `"bedrock/anthropic.claude-sonnet-4-5-20250929-v1:0"` BREAKING CHANGES: - Removed `model` from `MessageSettings` --> Use `model` parameter of `act()`, `get()`, `locate()` or a class extending `AgentBase` instead - upgraded min. version of `anthropic` dependency to 0.72.0 --> change from `anthropic.NotGiven` to `anthropic.Omit` and `anthropic.NOT_GIVEN` to `anthropic.omit` - renamed parameter `model_choice` to `model` for `ActModel.act()`, `GetModel.get()` and `LocateModel.locate()` - Removed `"ASKUI__MESSAGES__*"` and `"ANTHROPIC__MESSAGES__*"` environment variables --> use `settings` and `model` parameters when calling `<agent>.act()` instead - remove `AgentBase.model` property
1 parent 90e26b9 commit bbfd2f2

37 files changed

Lines changed: 1594 additions & 1002 deletions

docs/using-models.md

Lines changed: 99 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -229,7 +229,7 @@ class MyActModel(ActModel):
229229
def act(
230230
self,
231231
messages: list[MessageParam],
232-
model_choice: str,
232+
model: str,
233233
on_message: OnMessageCb | None = None,
234234
tools: list[Tool] | None = None,
235235
settings: ActSettings | None = None,
@@ -254,7 +254,7 @@ class MyGetAndLocateModel(GetModel, LocateModel):
254254
query: str,
255255
source: Source,
256256
response_schema: Type[ResponseSchema] | None,
257-
model_choice: str,
257+
model: str,
258258
) -> ResponseSchema | str:
259259
# Implement custom get logic, e.g.:
260260
# - Use a different OCR service
@@ -267,7 +267,7 @@ class MyGetAndLocateModel(GetModel, LocateModel):
267267
self,
268268
locator: str | Locator,
269269
image: ImageSource,
270-
model_choice: ModelComposition | str,
270+
model: ModelComposition | str,
271271
) -> PointList:
272272
# Implement custom locate logic, e.g.:
273273
# - Use a different object detection model
@@ -316,7 +316,7 @@ class DynamicActModel(ActModel):
316316
def act(
317317
self,
318318
messages: list[MessageParam],
319-
model_choice: str,
319+
model: str,
320320
on_message: OnMessageCb | None = None,
321321
tools: list[Tool] | None = None,
322322
settings: ActSettings | None = None,
@@ -362,7 +362,7 @@ class ExternalAIModel(ActModel):
362362
def act(
363363
self,
364364
messages: list[MessageParam],
365-
model_choice: str,
365+
model: str,
366366
on_message: OnMessageCb | None = None,
367367
tools: list[Tool] | None = None,
368368
settings: ActSettings | None = None,
@@ -386,7 +386,7 @@ class BusinessLogicModel(GetModel):
386386
query: str,
387387
source: Source,
388388
response_schema: Type[ResponseSchema] | None,
389-
model_choice: str,
389+
model: str,
390390
) -> ResponseSchema | str:
391391
# Apply business rules to the query
392392
if "price" in query.lower() and "discount" in self.business_rules:
@@ -408,7 +408,7 @@ class HybridModel(GetModel, LocateModel):
408408
query: str,
409409
source: Source,
410410
response_schema: Type[ResponseSchema] | None,
411-
model_choice: str,
411+
model: str,
412412
) -> ResponseSchema | str:
413413
try:
414414
# Try primary model first
@@ -440,7 +440,7 @@ class RobustModel(GetModel):
440440
query: str,
441441
source: Source,
442442
response_schema: Type[ResponseSchema] | None,
443-
model_choice: str,
443+
model: str,
444444
) -> ResponseSchema | str:
445445
try:
446446
# Your model logic here
@@ -464,7 +464,7 @@ class LoggedModel(ActModel):
464464
def act(
465465
self,
466466
messages: list[MessageParam],
467-
model_choice: str,
467+
model: str,
468468
on_message: OnMessageCb | None = None,
469469
tools: list[Tool] | None = None,
470470
settings: ActSettings | None = None,
@@ -491,8 +491,97 @@ class ConfigurableModel(GetModel):
491491
query: str,
492492
source: Source,
493493
response_schema: Type[ResponseSchema] | None,
494-
model_choice: str,
494+
model: str,
495495
) -> ResponseSchema | str:
496496
# Use configuration in your implementation
497497
return self._process_with_config(query, source)
498498
```
499+
500+
## Model providers
501+
502+
### Using a model provider
503+
504+
You can configure the model provider by setting the `ASKUI__VA__MODEL_PROVIDER` environment variable, e.g., `"bedrock"` to use Bedrock models. All the models you pass via `model` parameter of `act()`, `get()`, `locate()` will be prefixed with `"bedrock/"` in this case, e.g., if you call `agent.act("do something", model="anthropic.claude-sonnet-4-20250514-v1:0")`, it will be called as `agent.act("do something", model="bedrock/anthropic.claude-sonnet-4-20250514-v1:0")` under the hood. Alternatively, just prefix the model name(s) you pass via `model` parameter, e.g., `agent.act("do something", model="bedrock/anthropic.claude-sonnet-4-20250514-v1:0")` or `agent.act("do something", model="vertex/claude-sonnet-4@20250514")`.
505+
506+
At the time of writing, the following model providers are available:
507+
- `"bedrock"`: Use models hosted on AWS Bedrock.
508+
- `"vertex"`: Use models hosted on Google Vertex AI.
509+
- `"anthropic"`: Use models hosted behind Anthropic API.
510+
- `"askui"`: Use models hosted behind AskUI API.
511+
512+
**IMPORTANT:** If you pass a `model` argument at construction, this is not going to be prefixed with the model provider, e.g., if you call `VisionAgent(model="claude-sonnet-4-20250514", model_provider="askui")`, and later call `agent.act("do something")`, it will be called as `agent.act("do something", model="claude-sonnet-4-20250514")` and not as `agent.act("do something", model="askui/claude-sonnet-4-20250514")` under the hood. If you want to use a provider per default, just prefix the model name(s) you pass via `model` parameter, e.g., `VisionAgent(model="askui/claude-sonnet-4-20250514")` or `VisionAgent(model={"act": "askui/claude-sonnet-4-20250514"})`.
513+
514+
You can also set the `model` parameter of an agent via environment variable, e.g., `ASKUI__VA__MODEL`. For complex values, just use json, e.g., `ASKUI__VA__MODEL={"act":"askui/claude-sonnet-4-20250514"}`.
515+
516+
**IMPORTANT:** Keep in mind that the model name may differ between providers and not all providers may support a model (see https://docs.claude.com/en/docs/about-claude/models/overview). `askui` uses the same model names as `anthropic`. For `vertex` see https://docs.claude.com/en/api/claude-on-vertex-ai and for `bedrock` see https://docs.claude.com/en/api/claude-on-amazon-bedrock.
517+
518+
**IMPORTANT:** Keep in mind that when using a custom model that you may have to pass different `settings` to act as the settings support, e.g., tool or betas, differs between models, e.g., `agent.act("do something", model="askui/<a-special-model>", settings=ActSettings(tools=[ASpecialTool()]))`.
519+
520+
### Configure provider
521+
522+
The following environment variables control authentication and behavior per provider. Variables marked as required must be set for that provider, unless your environment provides credentials through instance/role bindings or local SDK configuration.
523+
524+
#### Common
525+
- `ASKUI__VA__MODEL_PROVIDER` (str, optional): Provider prefix to apply automatically (e.g., `bedrock`, `vertex`, `anthropic`, `askui`). Per default, no provider prefix is applied, e.g., `claude-sonnet-4-20250514` is going to be called as `claude-sonnet-4-20250514` and not as `bedrock/claude-sonnet-4-20250514` under the hood.
526+
- `ASKUI__VA__MODEL` (str | json): Default model or per-capability map. Example: `{"act":"bedrock/claude-sonnet-4-20250514"}`.
527+
528+
#### `askui` provider
529+
- `ASKUI_WORKSPACE_ID` (UUID, required): Workspace to route requests to.
530+
- `ASKUI_TOKEN` (str) or `ASKUI__AUTHORIZATION` (str): Exactly one required. If `ASKUI__AUTHORIZATION` is set, it is used verbatim as the `Authorization` header. Takes precedence over `ASKUI_TOKEN`.
531+
- `ASKUI_INFERENCE_ENDPOINT` (url, optional): Override base endpoint. Default: `https://inference.askui.com`.
532+
533+
#### `anthropic` provider (see https://docs.claude.com/en/docs/get-started#python)
534+
- `ANTHROPIC_API_KEY` (str, required): Anthropic API key.
535+
- `ANTHROPIC_AUTH_TOKEN` (str, optional): Anthropic auth token.
536+
- `ANTHROPIC_BASE_URL` (url, optional): Base URL to use for Anthropic API.
537+
538+
#### `bedrock` provider (via Anthropic Bedrock client, see https://docs.claude.com/en/api/claude-on-bedrock)
539+
- Uses standard AWS credential resolution. Set one of:
540+
- `AWS_PROFILE` (str) or static credentials `AWS_ACCESS_KEY_ID` (str), `AWS_SECRET_ACCESS_KEY` (str), optional `AWS_SESSION_TOKEN` (str).
541+
- Region (required): `AWS_REGION` (str) or `AWS_DEFAULT_REGION` (str).
542+
- Any other AWS SDK configuration is respected (env, shared config/credentials files, instance role, SSO, etc.).
543+
- `ANTHROPIC_BEDROCK_BASE_URL` (url, optional): Base URL to use for Bedrock API.
544+
545+
#### `vertex` provider (via Anthropic Vertex client, see https://docs.claude.com/en/api/claude-on-vertex-ai)
546+
- Uses Google Application Default Credentials (ADC). Common setups:
547+
- `GOOGLE_APPLICATION_CREDENTIALS` (path): Service account JSON key file.
548+
- Or gcloud-authenticated user with `gcloud auth application-default login`.
549+
- Project and location are resolved from ADC and/or environment; typical envs if needed in your setup: `GOOGLE_CLOUD_PROJECT` (str), `GOOGLE_CLOUD_LOCATION` (str). Consult your org’s Vertex configuration if these are required.
550+
- `CLOUD_ML_REGION` (str): Region to use for Vertex AI.
551+
- `ANTHROPIC_VERTEX_BASE_URL` (url): Base URL to use for Vertex AI.
552+
- `ANTHROPIC_VERTEX_PROJECT_ID` (str): Project ID to use for Vertex AI.
553+
554+
```python
555+
import os
556+
557+
from askui import VisionAgent
558+
559+
os.environ["ANTHROPIC_VERTEX_PROJECT_ID"] = "test-project"
560+
os.environ["CLOUD_ML_REGION"] = "europe-west1"
561+
562+
with VisionAgent() as agent:
563+
agent.act("do something", model="vertex/claude-sonnet-4@20250514")
564+
565+
# or
566+
567+
with VisionAgent(model_provider="vertex") as agent:
568+
agent.act("do something", model="claude-sonnet-4@20250514")
569+
570+
# or
571+
572+
with VisionAgent(model="vertex/claude-sonnet-4@20250514") as agent:
573+
agent.act("do something")
574+
575+
# or
576+
577+
os.environ["ASKUI__VA__MODEL"] = '{"act":"vertex/claude-sonnet-4@20250514"}'
578+
with VisionAgent() as agent:
579+
agent.act("do something")
580+
```
581+
582+
### Configure your own model provider
583+
584+
If you would like to configure you own model provider, e.g., let's say `"openai"`, just use provider string as key in the model registry as described in [Your own custom models](#your-own-custom-models) section but suffix it with a `"/"`, e.g., `"openai/"`, in order to differentiate it from regular model names. If you then, later call a model of that provider, e.g.,
585+
`agent.act("do something", model="openai/gpt-4o")`, the request will be routed to the model (api client) implementation that is the value of `"openai"` key in the model registry and the model passed to the underlying model (api client) implementation will be `"gpt-4o"`.
586+
587+
**IMPORTANT:** If you configure both a provider as well as a model prefixed with the provider prefix in the model registry, e.g., both `"openai/"` and `"openai/gpt-4o"`, the model has is going to be used and not the provider, e.g., `agent.act("do something", model="openai/gpt-4o")` is going to use `"openai/gpt-4o"` and not `"openai/"`, and `"openai/gpt-4o"` is going to be passed to the model (api client) implementation as `model` parameter instead of `"gpt-4o"`.

0 commit comments

Comments
 (0)