Add Jenkins job for Docker Patch jenkins shared library #3937

Divyaasm · 2023-08-30T00:18:32Z

Description

Add a Jenkins job in build library to call the docker patch functionality

Issues Resolved

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Signed-off-by: Divya Madala <[email protected]>

codecov · 2023-08-30T00:22:24Z

Codecov Report

Merging #3937 (7233582) into main (6a2287e) will increase coverage by 0.00%.
Report is 17 commits behind head on main.
The diff coverage is n/a.

❗ Current head 7233582 differs from pull request most recent head ab1c2ca. Consider uploading reports for the commit ab1c2ca to get more accurate results

@@           Coverage Diff           @@
##             main    #3937   +/-   ##
=======================================
  Coverage   92.05%   92.06%           
=======================================
  Files         187      187           
  Lines        5667     5669    +2     
=======================================
+ Hits         5217     5219    +2     
  Misses        450      450

see 1 file with indirect coverage changes

gaiksaya · 2023-08-30T00:57:59Z

jenkins/docker/docker-re-release.jenkinsfile

+
+pipeline {
+    options {
+        timeout(time: 7, unit: 'HOURS')


Does it need to be 7 hours?

Had some confusion on this. Should 5 hrs be sufficient.

Why does it needs to be 5 hours?
Max this activity should be 2 hours isnt it?

The most time you take on this is rebuild, and the rebuild does not need more than 1 hour, if not 30min.

I am confused of how this 5hr number coming from.

Sure Peter, I'll set that to 2 hrs. A bit confused to see that the docker-build job alone has 5 hrs timeout though the build finishes in less than 5 minutes.

Yes @Divyaasm that is particularly set for the release ci image, as that one takes at least 4 hours.

gaiksaya · 2023-08-30T00:58:44Z

jenkins/docker/docker-re-release.jenkinsfile

+        )
+        string(
+            name: 'TAG',
+            description: "Enter tag of the Product",


Wondering if version is the right naming here? Also a small nit:

Suggested change

description: "Enter tag of the Product",

description: "Enter version tag of the Product",

It's just a tag Sayali, The input would be 1 or 2 for example

Can you add an example in the description to be more clear? Also Does this take care to update latest and stuff? Also how many tags can you pass?

Yes @gaiksaya, the latest is updated only when the corresponding version is latest, currently it's for 2.x. And it's completely taken care by the shared library. Currently TAG and PRODUCT are strings. I can change them to list. So we can pass multiple values at a time.

@Divyaasm I would suggest use a choice param so people can choose the major version.

You did not give any example in the description and it will naturally confuse people.

gaiksaya · 2023-08-30T01:00:10Z

jenkins/docker/docker-re-release.jenkinsfile

+        booleanParam(
+            name: 'RE_RELEASE',
+            defaultValue: true,
+            description: "Forces to create a Docker-Copy if the Docker Build is successful"


This description is not clear, can you please elaborate or be more clear?
Thanks!

gaiksaya · 2023-08-30T01:01:42Z

tests/jenkins/TestDockerReRelease.groovy

+        assertThat(getCommandExecutions('build', ''), hasItem('{job=docker-build, propagate=true, wait=true, parameters=[null, null, null]}'))
+        assertThat(getCommandExecutions('build', ''), hasItem('{job=docker-copy, propagate=true, wait=true, parameters=[null, null, null, null]}'))
+        assertThat(getCommandExecutions('build', ''), hasItem('{job=docker-scan, propagate=true, wait=true, parameters=[null]}'))
+        assertThat(getCommandExecutions('build', ''), hasItem('{job=docker-promotion, propagate=true, wait=true, parameters=[null, null, null]}'))


Why are the parameters going null here? Shouldn't they be actual parameters?

Not sure why the parameters are taken as null though the above lines shows the exact parameter values that are passed. This is the same case when the jenkins job is triggered inside the library checkout previous implementation here

Can you try addParam instead of binding variable? Binding variables are used for mocking env variables mostly. https://github.com/jenkinsci/JenkinsPipelineUnit#mocking-jenkins-variables

peterzhuamazon

A lot of issues observed in this design.
Add some comments.

Thanks.

peterzhuamazon · 2023-08-30T17:04:35Z

jenkins/docker/docker-re-release.jenkinsfile

+
+pipeline {
+    options {
+        timeout(time: 7, unit: 'HOURS')


Why does it needs to be 5 hours?
Max this activity should be 2 hours isnt it?

The most time you take on this is rebuild, and the rebuild does not need more than 1 hour, if not 30min.

I am confused of how this 5hr number coming from.

peterzhuamazon · 2023-08-30T17:05:49Z

jenkins/docker/docker-re-release.jenkinsfile

+        )
+        string(
+            name: 'TAG',
+            description: "Enter tag of the Product",


@Divyaasm I would suggest use a choice param so people can choose the major version.

You did not give any example in the description and it will naturally confuse people.

peterzhuamazon · 2023-08-30T17:05:59Z

jenkins/docker/docker-re-release.jenkinsfile

+        booleanParam(
+            name: 'RE_RELEASE',
+            defaultValue: true,
+            description: "Forces to create a Docker-Copy if the Docker Build is successful"


jenkins/docker/docker-re-release.jenkinsfile

peterzhuamazon · 2023-08-30T17:07:39Z

jenkins/docker/docker-re-release.jenkinsfile

+            trim: true
+        )
+        booleanParam(
+            name: 'RE_RELEASE',


Not sure the point of this param if the workflow itself is aimed for re-release already?
What is the point of rebuild but not re-release?
What does it force in this case?

This parameter acts as a buildOption to create a docker-copy after the successful build inside the buildDockerImage()
I will update the description.

So in which case you do not want to docker-copy?
If not for docker-copy after we can just use docker-build without this workflow tho.
I am still not really sure why we need this at all.

peterzhuamazon · 2023-08-31T17:14:31Z

jenkins/docker/docker-re-release.jenkinsfile

+        )
+        choice(
+            name: 'TAG',
+            choices: ['1', 'latest'],


This should be 1 and 2, not latest.

@peterzhuamazon Can't this value be 2.9.0, 1.3.11?

Also I believe this should not be a choice but a string parameter?

I think this workflow is intended to concentrate only on the latest versions. So tag should do the work and update the major tag and latest images. Or providing the latest version number in each branch be it 1.x or 2.x should do the same some job as per library implementation.

You dont need to be 2.9.0/1.3.11 as we only patch the latest supported version.
And major 1/2 tag already tag to the latest version.

peterzhuamazon · 2023-08-31T17:15:31Z

jenkins/docker/docker-re-release.jenkinsfile

+
+pipeline {
+    options {
+        timeout(time: 7, unit: 'HOURS')


Yes @Divyaasm that is particularly set for the release ci image, as that one takes at least 4 hours.

peterzhuamazon · 2023-08-31T17:17:16Z

jenkins/docker/docker-re-release.jenkinsfile

+            trim: true
+        )
+        booleanParam(
+            name: 'RE_RELEASE',


So in which case you do not want to docker-copy?
If not for docker-copy after we can just use docker-build without this workflow tho.
I am still not really sure why we need this at all.

peterzhuamazon · 2023-09-05T17:09:47Z

jenkins/docker/docker-re-release.jenkinsfile

@@ -16,14 +16,9 @@ pipeline {
        )
        choice(
            name: 'TAG',
-            choices: ['1', 'latest'],
+            choices: ['1', '2'],


This should be going from latest to earliest, as if 2 1 instead of 1 2.

peterzhuamazon

Thanks @Divyaasm ,

One more nitpik and we are good to go.

Thanks.

Signed-off-by: Divya Madala <[email protected]> Update changes Signed-off-by: Divya Madala <[email protected]> Remove re-release option Signed-off-by: Divya Madala <[email protected]> Change tag options Signed-off-by: Divya Madala <[email protected]>

Add Jenkins job for docker patch library

149ce98

Signed-off-by: Divya Madala <[email protected]>

Divyaasm requested review from dblock, peterzhuamazon, bbarani, gaiksaya, rishabh6788, zelinh, jordarlu, prudhvigodithi and tianleh as code owners August 30, 2023 00:18

github-actions bot added the all-star-contributor label Aug 30, 2023

gaiksaya reviewed Aug 30, 2023

View reviewed changes

peterzhuamazon requested changes Aug 30, 2023

View reviewed changes

peterzhuamazon requested changes Aug 31, 2023

View reviewed changes

Divyaasm requested review from gaiksaya and peterzhuamazon August 31, 2023 20:46

peterzhuamazon reviewed Sep 5, 2023

View reviewed changes

peterzhuamazon approved these changes Sep 5, 2023

View reviewed changes

gaiksaya approved these changes Sep 5, 2023

View reviewed changes

Divyaasm force-pushed the patchjob branch from 7233582 to ab1c2ca Compare September 5, 2023 23:01

Divyaasm merged commit 577af1a into opensearch-project:main Sep 5, 2023
12 checks passed

gaiksaya mentioned this pull request Sep 13, 2023

Docker re-release Jenkins Job for the Docker Patch library #3930

Closed

Divyaasm deleted the patchjob branch February 27, 2024 19:42

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add Jenkins job for Docker Patch jenkins shared library #3937

Add Jenkins job for Docker Patch jenkins shared library #3937

Divyaasm commented Aug 30, 2023 •

edited

Loading

codecov bot commented Aug 30, 2023 •

edited

Loading

gaiksaya Aug 30, 2023

Divyaasm Aug 30, 2023

peterzhuamazon Aug 30, 2023

Divyaasm Aug 30, 2023

peterzhuamazon Aug 31, 2023

gaiksaya Aug 30, 2023

Divyaasm Aug 30, 2023

gaiksaya Aug 30, 2023

Divyaasm Aug 30, 2023 •

edited

Loading

peterzhuamazon Aug 30, 2023

gaiksaya Aug 30, 2023

peterzhuamazon Aug 30, 2023

gaiksaya Aug 30, 2023

Divyaasm Aug 30, 2023

gaiksaya Aug 30, 2023

peterzhuamazon left a comment

peterzhuamazon Aug 30, 2023

peterzhuamazon Aug 30, 2023

peterzhuamazon Aug 30, 2023

peterzhuamazon Aug 30, 2023

Divyaasm Aug 30, 2023

peterzhuamazon Aug 31, 2023

peterzhuamazon Aug 31, 2023

gaiksaya Aug 31, 2023

gaiksaya Aug 31, 2023

Divyaasm Aug 31, 2023

peterzhuamazon Aug 31, 2023

peterzhuamazon Aug 31, 2023

peterzhuamazon Aug 31, 2023

peterzhuamazon Sep 5, 2023

peterzhuamazon left a comment

	description: "Enter tag of the Product",
	description: "Enter version tag of the Product",

Add Jenkins job for Docker Patch jenkins shared library #3937

Add Jenkins job for Docker Patch jenkins shared library #3937

Conversation

Divyaasm commented Aug 30, 2023 • edited Loading

Description

Issues Resolved

codecov bot commented Aug 30, 2023 • edited Loading

Codecov Report

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Divyaasm Aug 30, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

peterzhuamazon left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

peterzhuamazon left a comment

Choose a reason for hiding this comment

Divyaasm commented Aug 30, 2023 •

edited

Loading

codecov bot commented Aug 30, 2023 •

edited

Loading

Divyaasm Aug 30, 2023 •

edited

Loading