feat: add HELMFILE_DISABLE_VALS env var to skip vals processing#2471
feat: add HELMFILE_DISABLE_VALS env var to skip vals processing#2471jimmyR wants to merge 1 commit into
Conversation
There was a problem hiding this comment.
Pull request overview
Adds environment-variable switches to disable Helmfile’s built-in vals secret resolution, enabling either pass-through of ref+ expressions (for external processing) or strict failure when unresolved ref+ expressions are present.
Changes:
- Introduces
HELMFILE_DISABLE_VALS(pass-through) andHELMFILE_DISABLE_VALS_STRICT(error onref+) gating inpkg/plugins/vals.go. - Updates
ValsInstance()to return avals.Evaluatorinterface and wires it into template secret expansion. - Adds unit tests and documents the new env vars in the docs.
Reviewed changes
Copilot reviewed 6 out of 6 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| pkg/plugins/vals.go | Adds pass-through/strict evaluators and env-var gating in ValsInstance(). |
| pkg/plugins/vals_test.go | Adds tests for disable/strict behavior and resets singleton between tests. |
| pkg/envvar/const.go | Defines the new env var names. |
| pkg/tmpl/expand_secret_ref.go | Adjusts secret expansion to use the updated ValsInstance() return type. |
| docs/remote-secrets.md | Documents the new env vars and conftest validation workflow. |
| docs/index.md | Adds the env vars to the “Useful internal Helmfile environment variables” list. |
| t.Fatal("strict mode should detect ref+ in arrays") | ||
| } | ||
| } | ||
|
|
There was a problem hiding this comment.
The new disable/strict tests don’t cover the []string input shape that existing callers use (e.g. renderValsSecrets passes map[string]any{"values": []string{...}}). As implemented, pass-through/strict evaluators will return []string unchanged and strict won’t detect ref+ inside it. Adding a test that calls evaluator.Eval(map[string]any{"values": []string{"ref+..."}}) (and asserts strict errors / pass-through returns []any) would prevent regressions.
| func TestDisableValsStrictArrayRefStringSlice(t *testing.T) { | |
| resetInstance() | |
| defer resetInstance() | |
| os.Setenv(envvar.DisableValsStrict, "true") | |
| defer os.Unsetenv(envvar.DisableValsStrict) | |
| evaluator, err := ValsInstance() | |
| if err != nil { | |
| t.Fatalf("unexpected error: %v", err) | |
| } | |
| // Should detect ref+ in arrays when the underlying type is []string, | |
| // which matches how renderValsSecrets passes values. | |
| input := map[string]any{ | |
| "values": []string{"normal", "ref+awssecrets://db/password"}, | |
| } | |
| _, err = evaluator.Eval(input) | |
| if err == nil { | |
| t.Fatal("strict mode should detect ref+ in []string arrays") | |
| } | |
| } | |
| func TestDisableValsPassThroughArrayStringSlice(t *testing.T) { | |
| resetInstance() | |
| defer resetInstance() | |
| // Enable pass-through mode and ensure strict mode is disabled. | |
| os.Setenv(envvar.DisableVals, "true") | |
| defer os.Unsetenv(envvar.DisableVals) | |
| os.Unsetenv(envvar.DisableValsStrict) | |
| evaluator, err := ValsInstance() | |
| if err != nil { | |
| t.Fatalf("unexpected error: %v", err) | |
| } | |
| input := map[string]any{ | |
| "values": []string{"normal", "ref+awssecrets://db/password"}, | |
| } | |
| out, err := evaluator.Eval(input) | |
| if err != nil { | |
| t.Fatalf("pass-through vals should not error: %v", err) | |
| } | |
| values, ok := out["values"].([]any) | |
| if !ok { | |
| t.Fatalf("expected out[\"values\"] to be []any, got %T", out["values"]) | |
| } | |
| if len(values) != 2 { | |
| t.Fatalf("expected 2 values, got %d", len(values)) | |
| } | |
| if values[0] != "normal" { | |
| t.Errorf("expected first element to be 'normal', got %v", values[0]) | |
| } | |
| if values[1] != "ref+awssecrets://db/password" { | |
| t.Errorf("expected second element to be the raw ref+, got %v", values[1]) | |
| } | |
| } |
| // passthroughEvaluator passes values through unchanged (for external vals) | ||
| type passthroughEvaluator struct{} | ||
|
|
||
| func (p *passthroughEvaluator) Eval(m map[string]any) (map[string]any, error) { | ||
| return m, nil | ||
| } | ||
|
|
||
| // strictEvaluator passes through values but errors if ref+ is detected | ||
| type strictEvaluator struct{} | ||
|
|
||
| func (s *strictEvaluator) Eval(m map[string]any) (map[string]any, error) { | ||
| if containsRefPlus(m) { | ||
| return nil, ErrValsDisabled | ||
| } | ||
| return m, nil | ||
| } |
There was a problem hiding this comment.
passthroughEvaluator.Eval/strictEvaluator.Eval currently return the input map unchanged. Some callers rely on vals' Eval normalizing slice types: e.g. pkg/state/state.go:4324 passes []string under {"values": input} and then asserts mapRendered["values"].([]any). With the new pass-through/strict evaluators, that value stays []string, so Helmfile will error even when vals is disabled (and strict mode will also miss ref+ inside []string). Consider updating the evaluators (and/or containsRefPlus) to handle []string (and potentially other typed slices) by converting to []any during traversal/return so existing callers keep working and strict detection is effective.
fc579b3 to
6f3a35a
Compare
| func containsRefPlus(v any) bool { | ||
| switch val := v.(type) { | ||
| case string: | ||
| return strings.Contains(val, "ref+") | ||
| case map[string]any: | ||
| for _, v := range val { | ||
| if containsRefPlus(v) { | ||
| return true | ||
| } | ||
| } | ||
| case []any: | ||
| for _, v := range val { | ||
| if containsRefPlus(v) { | ||
| return true | ||
| } | ||
| } | ||
| case []string: | ||
| for _, s := range val { | ||
| if strings.Contains(s, "ref+") { | ||
| return true | ||
| } | ||
| } | ||
| } |
There was a problem hiding this comment.
containsRefPlus doesn't traverse map[any]any values. Helmfile values often flow through YAML decoding as map[any]any (e.g., state code handles both map[any]any and map[string]any), so in HELMFILE_DISABLE_VALS_STRICT mode a ref+... nested inside a map[any]any can slip through without triggering ErrValsDisabled. Consider handling map[any]any (and/or map[string]interface{}) by iterating over the map values recursively so strict mode reliably blocks unresolved refs regardless of map key type.
| func TestDisableValsStrictNestedRef(t *testing.T) { | ||
| resetInstance() | ||
| defer resetInstance() | ||
|
|
||
| os.Setenv(envvar.DisableValsStrict, "true") | ||
| defer os.Unsetenv(envvar.DisableValsStrict) | ||
|
|
||
| evaluator, err := ValsInstance() | ||
| if err != nil { | ||
| t.Fatalf("unexpected error: %v", err) | ||
| } | ||
|
|
||
| // Should detect nested ref+ | ||
| input := map[string]any{ | ||
| "outer": map[string]any{ | ||
| "inner": "ref+vault://secret", | ||
| }, | ||
| } | ||
| _, err = evaluator.Eval(input) | ||
| if err == nil { | ||
| t.Fatal("strict mode should detect nested ref+") | ||
| } | ||
| } |
There was a problem hiding this comment.
There isn't a test that exercises strict-mode detection when the input contains map[any]any (a common shape from YAML unmarshalling). Adding a regression test that passes a structure containing map[any]any{"k": "ref+..."} would help ensure HELMFILE_DISABLE_VALS_STRICT can’t be bypassed by map key type.
|
I'll try and address copilots issues. |
Signed-off-by: Jim Robinson <1643772+jimmyR@users.noreply.github.com>
6f3a35a to
6bae70b
Compare
| func containsRefPlus(v any) bool { | ||
| switch val := v.(type) { | ||
| case string: | ||
| return strings.Contains(val, "ref+") | ||
| case map[string]any: |
There was a problem hiding this comment.
containsRefPlus uses strings.Contains(val, "ref+"), which can produce false positives (e.g., a non-vals string containing "ref+" as part of other text/URLs). Since vals references are expected to be expressions starting with ref+, consider checking strings.HasPrefix(strings.TrimSpace(val), "ref+") (and applying the same logic in the []string branch) to avoid incorrectly erroring in strict mode.
There was a problem hiding this comment.
I'm not sure that copilot is correct here. I use the containsRefPlus to scan for ref+ when in STRICT mode. I'm searching the values not the keys for any ref+ mentions.
Testing vals with this:
prefixedValue: "foo - ref+echo://prefixed-test"
This works and renders to:
prefixedKey: "foo - prefixed-test"
So surely strings.HasPrefix() would be wrong?
@yxxhero thoughts?
There was a problem hiding this comment.
@yxxhero I also tested with:
prefixedValue: "fooref+echo://prefixed-test"
Which renders to:
prefixedKey: "fooprefixed-test"
So, I'm feeling like strings.Contains() is still OK? But I'm new to helmfile and vals, so I might be missing something.
|
@jimmyR ping |
|
@yxxhero Hi, sorry for delay, I'll resolve this weekend. |
|
@jimmyR thanks so much. |
|
@jimmyR fix review comments and conflcts. |
|
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions. |
|
@copilot resolve the merge conflicts in this pull request |
|
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions. |
|
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions. |
|
@jimmyR ping |
|
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions. |
|
@copilot resolve the merge conflicts in this pull request |
Summary
This PR adds two new environment variables to allow helmfile's built-in vals processing to be disabled. This enables a more secure pattern where
ref+expression resolution is separated from helmfile execution — allowing credential access to be tightly scoped, audited, and validated before manifests are applied.HELMFILE_DISABLE_VALS=true: Passesref+expressions through unchanged, allowing an external vals processing (with appropriately scoped credentials) to resolve them, or additional security validation.HELMFILE_DISABLE_VALS_STRICT=true: Returns an error if anyref+expressions are encountered, enforcing that all secret references have been resolved prior to helmfile executionUse Case
In some CI environments, vals resolution requires AWS credentials (e.g.
ref+tfstates3withaws_profile) that are not available during the helmfile execution phase. The vals processing can be run as a separate step with the appropriate credentials, and helmfile can then consume the pre-rendered values withHELMFILE_DISABLE_VALS=true. This is the case where one AWS profile is required to pull a chart from S3, but another is required to readref+tfstates3values.HELMFILE_DISABLE_VALS=truealso enables security validation ofref+expressions before they are resolved — using a policy tool such as conftest to enforce that only approved secret references are permitted in your helmfile values.HELMFILE_DISABLE_VALS_STRICT=trueis useful for environments whereref+expressions should never appear at helmfile execution time — it provides a safety net to catch accidental unresolved references.Changes
Core Implementation
HELMFILE_DISABLE_VALSandHELMFILE_DISABLE_VALS_STRICTconstants inpkg/envvar/const.gopkg/plugins/vals.go, controlled by the new env varsTests
pkg/plugins/vals_test.goDocumentation
docs/remote-secrets.mdBehaviour
ref+expressionHELMFILE_DISABLE_VALS=trueHELMFILE_DISABLE_VALS_STRICT=trueHELMFILE_DISABLE_VALS=trueTesting
All tests pass:
go test ./pkg/plugins/...passesValidation with conftest
HELMFILE_DISABLE_VALS=truecan be combined with conftest to validate thatref+expressions only reference approved tfstates3 URIs before processing:Example rego policy: