Is it possible to use envsubst without removing missing variables?

[jpmorin]

Hi,

I use envsubst to replace dozens of variable in a ci yaml file. However, the file also contains variables that i do not wish to replace nor remove as they are used at runtime.

sample.yml

example: ${foo}
message: |
  Build $BUILD_ID failed
  Link: $URL/$BUILD_NAME/$BUILD_ID

command

foo=bar yq '(.. | select(tag == "!!str")) |= envsubst' sample.yml

expected

example: bar
message: |
  Build $BUILD_ID failed
  Link: $URL/$BUILD_NAME/$BUILD_ID

actual

example: bar
message: |
  Build  failed
  Link: //

Is there a way to use envsubst so that it does not remove missing variables?

Thanks.

[mikefarah]

Yes - the expression you are using (.. | select(tag == "!!str")) finds all the strings and attempts substitution.

If you had a way of only selecting the 'build' time subs you wanted to do, then that would work.

e..g

foo=bar yq '.example |= envsubst' sample.yml

[mikefarah]

Just had a thought that may also help, if you're env variables have a naming convention (e.g. prefix with build for build time), you could do something like:

foo=bar yq '(.. | select(tag == "!!str" and test("\bBUILD\b"))) |= envsubst' sample.yml

[jpmorin]

Hi Mike,

I cannot test against a standard naming convention since the environnements variables present in the multiple pipeline files come form differents teams and/or open-source resources.

However, I worked around this problem by selecting only the env vars with brackets ({}) that have a default value set by testing for the presence of a dash (-) in the env vars name.

yq 'with(.. | select(. == "*${*-*}*"); . |= envsubst)' sample.yml

Thanks for the input, it set me on the right track.

[bukowa]

Thank you that works cool

[jmpokar]

@jpmorin your solution helped me. thanks