Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 4 additions & 2 deletions coderd/x/chatd/ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -852,7 +852,7 @@ Model configs may carry a `reasoning_effort` config (`{default, max}`) inside `c

Subagent spawning is a second source of both values. `spawn_agent` accepts optional `model_config_id` and `reasoning_effort` args (discoverable via the `list_subagent_models` tool): an explicit model selection becomes the child chat's `last_model_config_id` and wins over personal and deployment subagent overrides and over parent inheritance, and an explicit effort is stored on the child's initial message and wins over effort carried by those overrides. Both are validated at spawn time (enabled config, enabled provider, usable credentials, effort on the global scale) and rejected with tool errors before the child chat is created; `computer_use` spawns reject both args because their model routing is specialized. Generation-time resolution and clamping below apply to the child unchanged.

During generation preparation, the effective effort is resolved as the chat's `last_reasoning_effort` if set, else the config's `default`; clamped to the config's `max` on the global scale `none < minimal < low < medium < high < xhigh < max`; and passed through to the provider. The provider verifies whether the configured value is valid for that model at runtime. If the model config has no `reasoning_effort`, any user-selected value is ignored. The resolved value is injected into the provider-native options with `chatprovider.ApplyReasoningEffort` after provider option conversion. For Anthropic, the fantasy provider converts effort into enabled budget thinking on models older than Claude 4.6, which reject adaptive thinking.
During generation preparation, the effective effort is resolved as the chat's `last_reasoning_effort` if set, else the config's `default`; clamped to the config's `max` on the global scale `none < minimal < low < medium < high < xhigh < max`; and passed through to the provider. The provider verifies whether the configured value is valid for that model at runtime. If the model config has no `reasoning_effort`, any user-selected value is ignored. The resolved value is injected into the provider-native options by `chatprovider.ProviderOptionsForCall`, which converts the model config and applies the effort in one step. For Anthropic, the fantasy provider converts effort into enabled budget thinking on models older than Claude 4.6, which reject adaptive thinking.

##### OpenAI transport selection

Expand All @@ -866,7 +866,9 @@ Request preparation reads the transport from the model instead of recomputing it
- Reasoning effort injection creates those option structs when a config has no OpenAI options of its own.
- File part conversion (`Model.AcceptsFilePartMediaType`) gates attachments, because the Responses API natively accepts only images and PDFs. A mismatch here drops text attachments.

Paths that build their own clients get a `Model` from the same constructor, including the compaction override, quick generation (used by turn status labels and debug models), and the advisor runtime. Debug recording replaces the wrapped client and preserves the resolved transport. Computer-use turns substitute a hardcoded default model that has no config of its own; it carries its own transport, so the chat model's `openai_config` does not follow it.
The first two happen together in `chatprovider.ProviderOptionsForCall`, the only entry point in `chatprovider` that builds provider options for a call; it delegates transport-aware OpenAI conversion to `chatopenai.ProviderOptionsFromChatConfig`. Config conversion and effort injection cannot pick different option types because one function owns both.

Paths that build their own clients get a `Model` from the same constructor, including the compaction override, quick generation (used by turn status labels and debug models), and the advisor runtime. Within quick generation, only title generation converts the model config through `ProviderOptionsForCall`; the turn status label and chat summary paths deliberately send no provider options, because they are short structured calls that set their own output bounds. Debug recording replaces the wrapped client and preserves the resolved transport. Computer-use turns substitute a hardcoded default model that has no config of its own; it carries its own transport, so the chat model's `openai_config` does not follow it.

Azure is deliberately exempt: its provider always enables the Responses API for known models and exposes no equivalent per-model hook, so the transport keeps following the known-model list for Azure. Ignoring the override there is what keeps the decisions above in agreement with the Azure client. The exemption is narrower than it appears, because chatd never builds an azure-typed provider as a fantasy azure client: `fantasyConfigForAIBridge` folds every provider type other than anthropic, bedrock, and openai into openai-compat, which always speaks Chat Completions.

Expand Down
14 changes: 1 addition & 13 deletions coderd/x/chatd/chatd.go
Original file line number Diff line number Diff line change
Expand Up @@ -424,19 +424,7 @@ func (p *Server) newAdvisorRuntime(
advisorCallConfig.MaxOutputTokens = ptr.Ref(maxOutputTokens)
// The override resolver pins an explicit advisor effort into the model
// config. Fallback models keep their configured default effort.
advisorReasoningEffort := chatprovider.ResolveReasoningEffort(
nil,
advisorCallConfig.ReasoningEffort,
)
providerOptions := chatprovider.ProviderOptionsFromChatModelConfig(
advisorModel,
advisorCallConfig.ProviderOptions,
)
providerOptions = chatprovider.ApplyReasoningEffort(
advisorModel,
providerOptions,
advisorReasoningEffort,
)
providerOptions := chatprovider.ProviderOptionsForCall(advisorModel, advisorCallConfig, nil)

rt, err := chatadvisor.NewRuntime(chatadvisor.RuntimeConfig{
Model: advisorModel.LanguageModel(),
Expand Down
19 changes: 16 additions & 3 deletions coderd/x/chatd/chatprovider/chatprovider.go
Original file line number Diff line number Diff line change
Expand Up @@ -1109,9 +1109,22 @@ func missingProviderAPIKeyError(provider string) error {
}
}

// ProviderOptionsFromChatModelConfig converts chat model provider options to
// fantasy provider options used for inference calls.
func ProviderOptionsFromChatModelConfig(
// ProviderOptionsForCall builds the provider options for one inference call.
// Config conversion and reasoning effort both create OpenAI option structs, so
// owning them together is what keeps their type aligned with the model's
// transport. requestedEffort is the caller's per-turn choice, which the
// config's bounds clamp.
func ProviderOptionsForCall(
model Model,
config codersdk.ChatModelCallConfig,
requestedEffort *string,
) fantasy.ProviderOptions {
options := providerOptionsFromChatModelConfig(model, config.ProviderOptions)
effort := ResolveReasoningEffort(requestedEffort, config.ReasoningEffort)
return applyReasoningEffort(model, options, effort)
}

func providerOptionsFromChatModelConfig(
model Model,
options *codersdk.ChatModelProviderOptions,
) fantasy.ProviderOptions {
Expand Down
12 changes: 7 additions & 5 deletions coderd/x/chatd/chatprovider/chatprovider_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -387,14 +387,16 @@ func TestAnthropicThinkingDisplayFromChat(t *testing.T) {
}
}

func TestProviderOptionsFromChatModelConfig_AnthropicThinkingDisplay(t *testing.T) {
func TestProviderOptionsForCall_AnthropicThinkingDisplay(t *testing.T) {
t.Parallel()

providerOptions := chatprovider.ProviderOptionsFromChatModelConfig(chatprovider.Model{}, &codersdk.ChatModelProviderOptions{
Anthropic: &codersdk.ChatModelAnthropicProviderOptions{
ThinkingDisplay: ptr.Ref(" SUMMARIZED "),
providerOptions := chatprovider.ProviderOptionsForCall(chatprovider.Model{}, codersdk.ChatModelCallConfig{
ProviderOptions: &codersdk.ChatModelProviderOptions{
Anthropic: &codersdk.ChatModelAnthropicProviderOptions{
ThinkingDisplay: ptr.Ref(" SUMMARIZED "),
},
},
})
}, nil)

require.NotNil(t, providerOptions)
anthropicOptions, ok := providerOptions[fantasyanthropic.Name].(*fantasyanthropic.ProviderOptions)
Expand Down
2 changes: 1 addition & 1 deletion coderd/x/chatd/chatprovider/reasoningeffort.go
Original file line number Diff line number Diff line change
Expand Up @@ -87,7 +87,7 @@ func SelectableReasoningEfforts(
return values[:maxRank+1]
}

func ApplyReasoningEffort(
func applyReasoningEffort(
model Model,
options fantasy.ProviderOptions,
effort *string,
Expand Down
170 changes: 170 additions & 0 deletions coderd/x/chatd/chatprovider/reasoningeffort_internal_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,170 @@
//nolint:testpackage // These tests cover the unexported applyReasoningEffort.
package chatprovider
Comment thread
ibetitsmike marked this conversation as resolved.

import (
"testing"

"charm.land/fantasy"
fantasyanthropic "charm.land/fantasy/providers/anthropic"
fantasyopenai "charm.land/fantasy/providers/openai"
fantasyopenaicompat "charm.land/fantasy/providers/openaicompat"
fantasyopenrouter "charm.land/fantasy/providers/openrouter"
fantasyvercel "charm.land/fantasy/providers/vercel"
"github.com/stretchr/testify/require"

"github.com/coder/coder/v2/coderd/util/ptr"
"github.com/coder/coder/v2/coderd/x/chatd/chattest"
"github.com/coder/coder/v2/codersdk"
)

func TestApplyReasoningEffort(t *testing.T) {
t.Parallel()

t.Run("CreatesOpenAIResponsesEntry", func(t *testing.T) {
t.Parallel()

got := applyReasoningEffort(NewModel(&chattest.FakeModel{ProviderName: fantasyopenai.Name, ModelName: "gpt-5"}, nil), nil, new(codersdk.ChatModelReasoningEffortHigh))
providerOptions, ok := got[fantasyopenai.Name].(*fantasyopenai.ResponsesProviderOptions)
require.True(t, ok, "%T", got[fantasyopenai.Name])
require.NotNil(t, providerOptions.ReasoningEffort)
require.Equal(t, fantasyopenai.ReasoningEffortHigh, *providerOptions.ReasoningEffort)
})

t.Run("PreservesOpenAIResponsesEntry", func(t *testing.T) {
t.Parallel()

options := fantasy.ProviderOptions{
fantasyopenai.Name: &fantasyopenai.ResponsesProviderOptions{
Instructions: ptr.Ref("answer briefly"),
Store: ptr.Ref(true),
},
}
got := applyReasoningEffort(NewModel(&chattest.FakeModel{ProviderName: fantasyopenai.Name, ModelName: "gpt-5"}, nil), options, new(codersdk.ChatModelReasoningEffortHigh))
providerOptions, ok := got[fantasyopenai.Name].(*fantasyopenai.ResponsesProviderOptions)
require.True(t, ok, "%T", got[fantasyopenai.Name])
require.Same(t, options[fantasyopenai.Name], providerOptions)
require.Equal(t, "answer briefly", *providerOptions.Instructions)
require.True(t, *providerOptions.Store)
require.Equal(t, fantasyopenai.ReasoningEffortHigh, *providerOptions.ReasoningEffort)
})

t.Run("PreservesOpenAILegacyEntry", func(t *testing.T) {
t.Parallel()

options := fantasy.ProviderOptions{
fantasyopenai.Name: &fantasyopenai.ProviderOptions{
User: ptr.Ref("user"),
ParallelToolCalls: ptr.Ref(true),
},
}
got := applyReasoningEffort(NewModel(&chattest.FakeModel{ProviderName: fantasyopenai.Name, ModelName: "gpt-4"}, nil), options, new(codersdk.ChatModelReasoningEffortHigh))
providerOptions, ok := got[fantasyopenai.Name].(*fantasyopenai.ProviderOptions)
require.True(t, ok, "%T", got[fantasyopenai.Name])
require.Same(t, options[fantasyopenai.Name], providerOptions)
require.Equal(t, "user", *providerOptions.User)
require.True(t, *providerOptions.ParallelToolCalls)
require.Equal(t, fantasyopenai.ReasoningEffortHigh, *providerOptions.ReasoningEffort)
})

tests := []struct {
name string
provider string
options fantasy.ProviderOptions
assert func(*testing.T, fantasy.ProviderOptions)
}{
{
name: "CreatesAnthropicEntry",
provider: fantasyanthropic.Name,
assert: func(t *testing.T, got fantasy.ProviderOptions) {
providerOptions, ok := got[fantasyanthropic.Name].(*fantasyanthropic.ProviderOptions)
require.True(t, ok, "%T", got[fantasyanthropic.Name])
require.NotNil(t, providerOptions.Effort)
require.Equal(t, fantasyanthropic.EffortHigh, *providerOptions.Effort)
},
},
{
name: "PreservesAnthropicEntry",
provider: fantasyanthropic.Name,
options: fantasy.ProviderOptions{fantasyanthropic.Name: &fantasyanthropic.ProviderOptions{SendReasoning: ptr.Ref(true)}},
assert: func(t *testing.T, got fantasy.ProviderOptions) {
providerOptions := got[fantasyanthropic.Name].(*fantasyanthropic.ProviderOptions)
require.True(t, *providerOptions.SendReasoning)
require.Equal(t, fantasyanthropic.EffortHigh, *providerOptions.Effort)
},
},
{
name: "CreatesOpenAICompatEntry",
provider: fantasyopenaicompat.Name,
assert: func(t *testing.T, got fantasy.ProviderOptions) {
providerOptions, ok := got[fantasyopenaicompat.Name].(*fantasyopenaicompat.ProviderOptions)
require.True(t, ok, "%T", got[fantasyopenaicompat.Name])
require.NotNil(t, providerOptions.ReasoningEffort)
require.Equal(t, fantasyopenai.ReasoningEffortHigh, *providerOptions.ReasoningEffort)
},
},
{
name: "PreservesOpenAICompatEntry",
provider: fantasyopenaicompat.Name,
options: fantasy.ProviderOptions{fantasyopenaicompat.Name: &fantasyopenaicompat.ProviderOptions{User: ptr.Ref("user")}},
assert: func(t *testing.T, got fantasy.ProviderOptions) {
providerOptions := got[fantasyopenaicompat.Name].(*fantasyopenaicompat.ProviderOptions)
require.Equal(t, "user", *providerOptions.User)
require.Equal(t, fantasyopenai.ReasoningEffortHigh, *providerOptions.ReasoningEffort)
},
},
{
name: "CreatesVercelEntry",
provider: fantasyvercel.Name,
assert: func(t *testing.T, got fantasy.ProviderOptions) {
providerOptions, ok := got[fantasyvercel.Name].(*fantasyvercel.ProviderOptions)
require.True(t, ok, "%T", got[fantasyvercel.Name])
require.NotNil(t, providerOptions.Reasoning)
require.NotNil(t, providerOptions.Reasoning.Effort)
require.Equal(t, fantasyvercel.ReasoningEffortHigh, *providerOptions.Reasoning.Effort)
},
},
{
name: "PreservesVercelNestedEntry",
provider: fantasyvercel.Name,
options: fantasy.ProviderOptions{fantasyvercel.Name: &fantasyvercel.ProviderOptions{Reasoning: &fantasyvercel.ReasoningOptions{Enabled: ptr.Ref(true), MaxTokens: ptr.Ref(int64(1024))}}},
assert: func(t *testing.T, got fantasy.ProviderOptions) {
providerOptions := got[fantasyvercel.Name].(*fantasyvercel.ProviderOptions)
require.True(t, *providerOptions.Reasoning.Enabled)
require.Equal(t, int64(1024), *providerOptions.Reasoning.MaxTokens)
require.Equal(t, fantasyvercel.ReasoningEffortHigh, *providerOptions.Reasoning.Effort)
},
},
{
name: "CreatesOpenRouterEntry",
provider: fantasyopenrouter.Name,
assert: func(t *testing.T, got fantasy.ProviderOptions) {
providerOptions, ok := got[fantasyopenrouter.Name].(*fantasyopenrouter.ProviderOptions)
require.True(t, ok, "%T", got[fantasyopenrouter.Name])
require.NotNil(t, providerOptions.Reasoning)
require.NotNil(t, providerOptions.Reasoning.Effort)
require.Equal(t, fantasyopenrouter.ReasoningEffortHigh, *providerOptions.Reasoning.Effort)
},
},
{
name: "PreservesOpenRouterNestedEntry",
provider: fantasyopenrouter.Name,
options: fantasy.ProviderOptions{fantasyopenrouter.Name: &fantasyopenrouter.ProviderOptions{Reasoning: &fantasyopenrouter.ReasoningOptions{Enabled: ptr.Ref(true), MaxTokens: ptr.Ref(int64(1024))}}},
assert: func(t *testing.T, got fantasy.ProviderOptions) {
providerOptions, ok := got[fantasyopenrouter.Name].(*fantasyopenrouter.ProviderOptions)
require.True(t, ok, "%T", got[fantasyopenrouter.Name])
require.True(t, *providerOptions.Reasoning.Enabled)
require.Equal(t, int64(1024), *providerOptions.Reasoning.MaxTokens)
require.NotNil(t, providerOptions.Reasoning.Effort)
require.Equal(t, fantasyopenrouter.ReasoningEffortHigh, *providerOptions.Reasoning.Effort)
},
},
}

for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
t.Parallel()
got := applyReasoningEffort(NewModel(&chattest.FakeModel{ProviderName: tt.provider}, nil), tt.options, new(codersdk.ChatModelReasoningEffortHigh))
tt.assert(t, got)
})
}
}
Loading
Loading