You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
- add support for using anthropic models hosted on bedrock and vertex, just install optional dependencies and set environment variables for configuring base url, authentication and authorization etc.
- add support for configuring model provider through model registry (key ending with "/") instead of having to configure each model separately for a provider
- add `ASKUI__VA__MODEL_PROVIDER` environment variable for setting default model provider
- add `ASKUI__VA__MODEL`
for setting default model
- added providers `"askui/"`, `"bedrock/"`, `"anthropic/"` and `"vertex/"` per default, so you can just pass, e.g., `"askui/claude-sonnet-4-20250514"`, or `"bedrock/anthropic.claude-sonnet-4-5-20250929-v1:0"`
BREAKING CHANGES:
- Removed `model` from `MessageSettings` --> Use `model` parameter of `act()`, `get()`, `locate()` or a class extending `AgentBase` instead
- upgraded min. version of `anthropic` dependency to 0.72.0 --> change from `anthropic.NotGiven` to `anthropic.Omit` and `anthropic.NOT_GIVEN` to `anthropic.omit`
- renamed parameter `model_choice` to `model` for `ActModel.act()`, `GetModel.get()` and `LocateModel.locate()`
- Removed `"ASKUI__MESSAGES__*"` and `"ANTHROPIC__MESSAGES__*"` environment variables --> use `settings` and `model` parameters when calling `<agent>.act()` instead
- remove `AgentBase.model` property
@@ -408,7 +408,7 @@ class HybridModel(GetModel, LocateModel):
408
408
query: str,
409
409
source: Source,
410
410
response_schema: Type[ResponseSchema] |None,
411
-
model_choice: str,
411
+
model: str,
412
412
) -> ResponseSchema |str:
413
413
try:
414
414
# Try primary model first
@@ -440,7 +440,7 @@ class RobustModel(GetModel):
440
440
query: str,
441
441
source: Source,
442
442
response_schema: Type[ResponseSchema] |None,
443
-
model_choice: str,
443
+
model: str,
444
444
) -> ResponseSchema |str:
445
445
try:
446
446
# Your model logic here
@@ -464,7 +464,7 @@ class LoggedModel(ActModel):
464
464
defact(
465
465
self,
466
466
messages: list[MessageParam],
467
-
model_choice: str,
467
+
model: str,
468
468
on_message: OnMessageCb |None=None,
469
469
tools: list[Tool] |None=None,
470
470
settings: ActSettings |None=None,
@@ -491,8 +491,97 @@ class ConfigurableModel(GetModel):
491
491
query: str,
492
492
source: Source,
493
493
response_schema: Type[ResponseSchema] |None,
494
-
model_choice: str,
494
+
model: str,
495
495
) -> ResponseSchema |str:
496
496
# Use configuration in your implementation
497
497
returnself._process_with_config(query, source)
498
498
```
499
+
500
+
## Model providers
501
+
502
+
### Using a model provider
503
+
504
+
You can configure the model provider by setting the `ASKUI__VA__MODEL_PROVIDER` environment variable, e.g., `"bedrock"` to use Bedrock models. All the models you pass via `model` parameter of `act()`, `get()`, `locate()` will be prefixed with `"bedrock/"` in this case, e.g., if you call `agent.act("do something", model="anthropic.claude-sonnet-4-20250514-v1:0")`, it will be called as `agent.act("do something", model="bedrock/anthropic.claude-sonnet-4-20250514-v1:0")` under the hood. Alternatively, just prefix the model name(s) you pass via `model` parameter, e.g., `agent.act("do something", model="bedrock/anthropic.claude-sonnet-4-20250514-v1:0")` or `agent.act("do something", model="vertex/claude-sonnet-4@20250514")`.
505
+
506
+
At the time of writing, the following model providers are available:
507
+
-`"bedrock"`: Use models hosted on AWS Bedrock.
508
+
-`"vertex"`: Use models hosted on Google Vertex AI.
509
+
-`"anthropic"`: Use models hosted behind Anthropic API.
510
+
-`"askui"`: Use models hosted behind AskUI API.
511
+
512
+
**IMPORTANT:** If you pass a `model` argument at construction, this is not going to be prefixed with the model provider, e.g., if you call `VisionAgent(model="claude-sonnet-4-20250514", model_provider="askui")`, and later call `agent.act("do something")`, it will be called as `agent.act("do something", model="claude-sonnet-4-20250514")` and not as `agent.act("do something", model="askui/claude-sonnet-4-20250514")` under the hood. If you want to use a provider per default, just prefix the model name(s) you pass via `model` parameter, e.g., `VisionAgent(model="askui/claude-sonnet-4-20250514")` or `VisionAgent(model={"act": "askui/claude-sonnet-4-20250514"})`.
513
+
514
+
You can also set the `model` parameter of an agent via environment variable, e.g., `ASKUI__VA__MODEL`. For complex values, just use json, e.g., `ASKUI__VA__MODEL={"act":"askui/claude-sonnet-4-20250514"}`.
515
+
516
+
**IMPORTANT:** Keep in mind that the model name may differ between providers and not all providers may support a model (see https://docs.claude.com/en/docs/about-claude/models/overview). `askui` uses the same model names as `anthropic`. For `vertex` see https://docs.claude.com/en/api/claude-on-vertex-ai and for `bedrock` see https://docs.claude.com/en/api/claude-on-amazon-bedrock.
517
+
518
+
**IMPORTANT:** Keep in mind that when using a custom model that you may have to pass different `settings` to act as the settings support, e.g., tool or betas, differs between models, e.g., `agent.act("do something", model="askui/<a-special-model>", settings=ActSettings(tools=[ASpecialTool()]))`.
519
+
520
+
### Configure provider
521
+
522
+
The following environment variables control authentication and behavior per provider. Variables marked as required must be set for that provider, unless your environment provides credentials through instance/role bindings or local SDK configuration.
523
+
524
+
#### Common
525
+
-`ASKUI__VA__MODEL_PROVIDER` (str, optional): Provider prefix to apply automatically (e.g., `bedrock`, `vertex`, `anthropic`, `askui`). Per default, no provider prefix is applied, e.g., `claude-sonnet-4-20250514` is going to be called as `claude-sonnet-4-20250514` and not as `bedrock/claude-sonnet-4-20250514` under the hood.
526
+
-`ASKUI__VA__MODEL` (str | json): Default model or per-capability map. Example: `{"act":"bedrock/claude-sonnet-4-20250514"}`.
527
+
528
+
#### `askui` provider
529
+
-`ASKUI_WORKSPACE_ID` (UUID, required): Workspace to route requests to.
530
+
-`ASKUI_TOKEN` (str) or `ASKUI__AUTHORIZATION` (str): Exactly one required. If `ASKUI__AUTHORIZATION` is set, it is used verbatim as the `Authorization` header. Takes precedence over `ASKUI_TOKEN`.
531
+
-`ASKUI_INFERENCE_ENDPOINT` (url, optional): Override base endpoint. Default: `https://inference.askui.com`.
532
+
533
+
#### `anthropic` provider (see https://docs.claude.com/en/docs/get-started#python)
534
+
-`ANTHROPIC_API_KEY` (str, required): Anthropic API key.
- Region (required): `AWS_REGION` (str) or `AWS_DEFAULT_REGION` (str).
542
+
- Any other AWS SDK configuration is respected (env, shared config/credentials files, instance role, SSO, etc.).
543
+
-`ANTHROPIC_BEDROCK_BASE_URL` (url, optional): Base URL to use for Bedrock API.
544
+
545
+
#### `vertex` provider (via Anthropic Vertex client, see https://docs.claude.com/en/api/claude-on-vertex-ai)
546
+
- Uses Google Application Default Credentials (ADC). Common setups:
547
+
-`GOOGLE_APPLICATION_CREDENTIALS` (path): Service account JSON key file.
548
+
- Or gcloud-authenticated user with `gcloud auth application-default login`.
549
+
- Project and location are resolved from ADC and/or environment; typical envs if needed in your setup: `GOOGLE_CLOUD_PROJECT` (str), `GOOGLE_CLOUD_LOCATION` (str). Consult your org’s Vertex configuration if these are required.
550
+
-`CLOUD_ML_REGION` (str): Region to use for Vertex AI.
551
+
-`ANTHROPIC_VERTEX_BASE_URL` (url): Base URL to use for Vertex AI.
552
+
-`ANTHROPIC_VERTEX_PROJECT_ID` (str): Project ID to use for Vertex AI.
If you would like to configure you own model provider, e.g., let's say `"openai"`, just use provider string as key in the model registry as described in [Your own custom models](#your-own-custom-models) section but suffix it with a `"/"`, e.g., `"openai/"`, in order to differentiate it from regular model names. If you then, later call a model of that provider, e.g.,
585
+
`agent.act("do something", model="openai/gpt-4o")`, the request will be routed to the model (api client) implementation that is the value of `"openai"` key in the model registry and the model passed to the underlying model (api client) implementation will be `"gpt-4o"`.
586
+
587
+
**IMPORTANT:** If you configure both a provider as well as a model prefixed with the provider prefix in the model registry, e.g., both `"openai/"` and `"openai/gpt-4o"`, the model has is going to be used and not the provider, e.g., `agent.act("do something", model="openai/gpt-4o")` is going to use `"openai/gpt-4o"` and not `"openai/"`, and `"openai/gpt-4o"` is going to be passed to the model (api client) implementation as `model` parameter instead of `"gpt-4o"`.
0 commit comments