Provide control over pod labels and annotations

[SaschaSchwarze0]

There are certain capabilities in Kubernetes that require annotations to be present on the pod, for example the kubernetes.io/ingress-bandwidth and kubernetes.io/egress-bandwidth to support Network Traffic Shaping. At the moment, they cannot be controlled.

Similarly, labels on pods can be used for filtering or monitoring.

We need to provide some control to users who define the build strategies as well as to those who define build runs. Some scenarios:

• For the traffic shaping, it makes sense to define the annotations in the build strategy as this is the place where the resources for the steps are also defined.
• It also makes sense if users can defined labels on a build run and use the same label to search for taskruns and pods, which means that those labels need to be copied as well.

We will also need to think about / consider:

• Do we want to propagate labels and annotations on builds to the taskrun as well ?
• What about name clashes (same label / annotation defined with different values on build strategy / build / build run), what value wins. In the network traffic scenario, the person who defines the values in the build strategy does not want those to be overwritten in the build / buildrun. In other scenarios, this might be fine.
• Instead of providing annotations for the pod in the metadata of the build strategy / build / build run, one could also introduce something like spec.podAnnotations and spec.podLabels to allow to explicitly define values that are set on the task run / pod, while other values can be defined in the metadata that do not get propagated.
• Which kind of configuration on the build controller level is needed to restrict setting annotations and labels?

[adambkaplan]

Definitely something to work out via an enhancement proposal. There are other annotations out there a build strategy would want to take advantage of, such as user namespaces on CRI-O clusters.

Not sure if we've officially laid this out as an API convention, but it feels like we have precedent for inherited spec attributes that flow from BuildStrategy -> Build -> BuildRun. BuildRun can override and augment specs from Build, Build can override and augment specs from BuildStrategy.

[SaschaSchwarze0]

Definitely something to work out via an enhancement proposal. There are other annotations out there a build strategy would want to take advantage of, such as user namespaces on CRI-O clusters.

Not sure if we've officially laid this out as an API convention, but it feels like we have precedent for inherited spec attributes that flow from BuildStrategy -> Build -> BuildRun. BuildRun can override and augment specs from Build, Build can override and augment specs from BuildStrategy.

Thank you for the feedback, Adam. Yeah, EP is planned. The inheritance as you lay it out is the obvious model I also agree with in general. Nevertheless, with the possible persona differentiation with admins maintaining build strategies and users that create builds and build runs, we also require a mechanism for certain annotations defined by the admin in the build strategy not to be over-writable.

We'll think through it.

[zhangtbj]

If possible, I suggest we can do this enhancement step by step.

• If we all agree annotations should also be propagated from build strategy like what we are doing for label:

  build/pkg/controller/buildrun/generate_taskrun.go

  Lines 257 to 259 in ae7a362

  	for label, value := range strategy.GetResourceLabels() {
  	expectedTaskRun.Labels[label] = value
  	}

  , I suggest we add the annotations capability as first step so that we complete this missing part first.
• If there is some end-user requirements in future that want to override the label or annotation from build or buildrun, I think it could be the second part of the enhancement.

But I think the first part is more important because we miss that.

[SaschaSchwarze0]

If possible, I suggest we can do this enhancement step by step.

* If we all agree `annotations` should also be propagated from build strategy like what we are doing for label: https://github.com/shipwright-io/build/blob/ae7a36292ad165203eaa043f85703842c3f3b0ed/pkg/controller/buildrun/generate_taskrun.go#L257-L259
  , I suggest we add the annotations capability as first step so that we complete this missing part first.

* If there is some end-user requirements in future that want to override the label or annotation from build or buildrun, I think it could be the second part of the enhancement.

But I think the first part is more important because we miss that.

This matches exactly what I agreed with Enrique yesterday afternoon. EP will come soon.

[SaschaSchwarze0]

EP is open and it focuses on annotations first to not over-complicate things: EP: Propagating annotations from the build strategy to the pod #523. Labels cover other scenarios that we can handle separately.

[SaschaSchwarze0]

Annotation propagation has been implemented here: Propagate annotations from BuildStrategy to TaskRun #539.

Putting the remaining label work in a separate issue: Provide user control over pod label #625