Allow aws.* configuration inside remoteWrite

[5nafu]

As a user of the operator
I would like to configure IAM based authentication inside my remoteWrite configuration
so that I don't need to (ab-)use extraArgs to use it.

The vmagent supports using IAM authentication by setting (for example) -remoteWrite.aws.roleARN et.al but the operator does not accept it as parameter yet. While it would be possible to use extraArgs to set these parameters, if used together with (for example) inlineUrlRelabelConfig one would need to generate a configmap outside of the general operator (helm) configuration and attach it to the agent. It would be better to have everything in one place.

[Haleygo]

Hello @5nafu .

It would be better to have everything in one place.

What do you mean by "have everything in one place"? Do you have any perferred proposal?

[5nafu]

Hi @Haleygo,

Currently, if you have one or multiple remote-writes with at lease one IAM auth, your vmagent object might look like (only the relevant parts):

remoteWrite:
  - url: http://vmsingle-victoria-stack.victoria.svc:8429/api/v1/write
  - url: https://vmingest.basic.auth.url/api/v1/write
    basicAuth:
      password:
        key: password
        name: victoria-credentials
      username:
        key: username
        name: victoria-credentials
    inlineUrlRelabelConfig:
    - action: labeldrop
      regex: source_.*|destination_service_.*|destination_can.*|destination_principal
    # ... 
  - url: https://victoria.i.am.auth.url/insert/123/prometheus/api/v1/write
    inlineUrlRelabelConfig:
    - action: drop
      regex: ^kube_.*;kubecost-cost-analyzer$
      source_labels:
      - __name__
      - job
    # ...
extraArgs:
    # ...
    remoteWrite.aws.region: ',,eu-central-1'
    remoteWrite.aws.roleARN: ',,arn:aws:iam::AccountID:role/Role'
    remoteWrite.aws.service: ',,execute-api'
    remoteWrite.aws.useSigv4: false,false,true

Notice the need to add the appropriate amount of comma (and boolean values) for each remote write. It is even worse when using the helm chart, as there will be an additional "local" write that the user does not configure.

It would be better for the user if one could use a configuration similar to the basicAuth like:

remoteWrite:
  - url: http://vmsingle-victoria-stack.victoria.svc:8429/api/v1/write
  - url: https://vmingest.basic.auth.url/api/v1/write
    basicAuth:
      password:
        key: password
        name: victoria-credentials
      username:
        key: username
        name: victoria-credentials
    inlineUrlRelabelConfig:
    - action: labeldrop
      regex: source_.*|destination_service_.*|destination_can.*|destination_principal
    # ... 
  - url: https://victoria.i.am.auth.url/insert/123/prometheus/api/v1/write
    aws:
      region: 'eu-central-1'
      roleARN: 'arn:aws:iam::AccountID:role/Role'
      service: 'execute-api'
      useSigv4: true
    inlineUrlRelabelConfig:
    - action: drop
      regex: ^kube_.*;kubecost-cost-analyzer$
      source_labels:
      - __name__
      - job
    # ...

[f41gh7]

I agree, your example with separate aws spec looks better.
I think we should add more configuration params to the remoteWrite defintion. It makes configuration easy to understand without extraArgs comma-madness syntax.