Skip to content

feat(config): enable pipe operations on source function output#6790

Open
VaismanLior wants to merge 1 commit intoupdatecli:mainfrom
VaismanLior:source-pipe-operations
Open

feat(config): enable pipe operations on source function output#6790
VaismanLior wants to merge 1 commit intoupdatecli:mainfrom
VaismanLior:source-pipe-operations

Conversation

@VaismanLior
Copy link
Copy Markdown

Fix #6789

Enable pipe operations on {{ source }} function output to allow transformation of source values at runtime using sprig template functions.

Changes

  • Added sprig functions to runtime template processing in pkg/core/config/main.go
  • Updated source function documentation with pipe examples
  • Added test coverage for pipe operations with source function

Example

sourceid: '{{ source "version" | replace "." "_" }}'

Test

To test this pull request, you can run the following commands:

cd pkg/core/config
go test -v -run TestUpdate/10

Additional Information

Checklist

  • I have updated the documentation via pull request in website repository.

Tradeoff

No breaking changes. All existing {{ source }} usage continues to work. Adding sprig functions to runtime templates provides flexibility with minimal performance impact.

Potential improvement

Comment thread pkg/core/config/main.go

tmpl, err := template.New("cfg").Funcs(updatecliRuntimeFuncMap(data)).Parse(string(content))
tmpl, err := template.New("cfg").
Funcs(sprig.FuncMap()).
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

While we are here, I am wondering if you shouldn't also add the custom Helm function

Funcs(helmFuncMap()). // add helm funcMap

Suggested change
Funcs(sprig.FuncMap()).
Funcs(sprig.FuncMap()).
Funcs(helmFuncMap()). // add helm funcMap

@olblak
Copy link
Copy Markdown
Member

olblak commented Nov 18, 2025

Thank you for the PR, it looks great especially with the tests.
When I first implemented this, I hesitated between adding those to the source function or not, but I am happy to see this evolving.

I just have a minor feedback to only this templating with the one happening at the beginning

@olblak olblak added enhancement New feature or request core All things related to Updatecli core engine labels Nov 18, 2025
@olblak
Copy link
Copy Markdown
Member

olblak commented Nov 18, 2025

I don't know what happening but I just gave it a test using
And it doesn't seem to work

sources:
  1:
    name: echo 1.2.3
    kind: shell
    dependson:
      - 2
    spec:
      command: echo 1.2.3
  2:
    name: echo 2.3.4
    kind: shell
    spec:
      command: echo 2.3.4

conditions:
  3:
    name: 'echo {{ source "1" }}.x'
    kind: shell
    disablesourceinput: true
    spec:
      command: 'echo {{ source "1" }}'

  4:
    name: 'echo {{ source "1" | replace "." "_" }}'
    kind: shell
    disablesourceinput: true
    spec:
      command: 'echo {{ source "1" | replace "." "_" }}'

@olblak
Copy link
Copy Markdown
Member

olblak commented Nov 19, 2025

I just took another look and unfortunately I am not even sure it's technically possible to achieve what you need.

Updatecli has two kind of templating, one at start time and a second one after each resource execution to leverage dynamic information like source result.

Because the templating engine doesn't know when a resource is executed, we are using this trick
https://github.com/updatecli/updatecli/blame/7e7aaa2e622dc26f9fdfe8b58e228879afcdfda7/pkg/core/config/funcsUpdatecli.go#L86

If a source hasn't been executed yet, then the template returns '{{ source "sourceID"}}', so in the current implementation it would drop any additional filters like replace "." "_"
I don't know if it's possible to return the initial expression instead.

@olblak
Copy link
Copy Markdown
Member

olblak commented Nov 19, 2025

Worth mentioning your tests are passing because it's executed at start time but with source result which never happen in reality

@VaismanLior
Copy link
Copy Markdown
Author

Thanks for the quick review and for looking into this!
I see what you're talking about. I'll try to find a fix when I have more time to tackle this.
Hopefully there's a way around the issue without major changes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

core All things related to Updatecli core engine enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Feature Request: Support pipe operations on {{ source }} function output

2 participants