Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Hosting request for CloudAEye Plugin #4131

Open
bhavyalatha26 opened this issue Oct 14, 2024 · 22 comments
Open

Hosting request for CloudAEye Plugin #4131

bhavyalatha26 opened this issue Oct 14, 2024 · 22 comments
Labels
bot-check-complete Automated hosting checks passed hosting-request Request to host a component in jenkinsci security-audit-todo The security team needs to audit the hosting request code

Comments

@bhavyalatha26
Copy link

Repository URL

https://github.com/CloudAEye/cloudaeye-plugin

New Repository Name

cloudaeye-plugin

Description

The CloudAEye Plugin is designed to enhance CI/CD pipelines by seamlessly exporting job details and build logs from Jenkins to the CloudAEye platform. CloudAEye analyzes test failure logs to identify root causes and suggests actionable code fixes, helping developers resolve issues more efficiently. By providing real-time insights into build failures, the plugin enables faster troubleshooting and minimizes downtime in software delivery pipelines.

Key Features:

  • Exports detailed logs and job metadata for comprehensive analysis.
  • Automates root cause analysis and propose fix suggestions for test failures.
  • Integrates effortlessly into existing Jenkins pipelines.

GitHub users to have commit permission

@bhavyalatha26

Jenkins project users to have release permission

cloudaeye

Issue tracker

GitHub issues

@bhavyalatha26 bhavyalatha26 added the hosting-request Request to host a component in jenkinsci label Oct 14, 2024
@jenkins-cert-app
Copy link
Collaborator

Security audit, information and commands

The security team is auditing all the hosting requests, to ensure a better security by default.

This message informs you that a Jenkins Security Scan was triggered on your repository.
It takes ~10 minutes to complete.

Commands

The bot will parse all comments, and it will check if any line start with a command.

Security team only:

  • /audit-ok => the audit is complete, the hosting can continue 🎉.
  • /audit-skip => the audit is not necessary, the hosting can continue 🎉.
  • /audit-findings => the audit reveals some issues that require corrections ✏️.

Anyone:

  • /request-security-scan => the findings from the Jenkins Security Scan were corrected, this command will re-scan your repository 🔍.
  • /audit-review => the findings from the audit were corrected, this command will ping the security team to review the findings 👀. It's only applicable when the previous audit required changes.

Only one command can be requested per comment.

(automatically generated message, version: 1.29.20)

@jenkins-cert-app jenkins-cert-app added the security-audit-todo The security team needs to audit the hosting request code label Oct 14, 2024
Copy link

Hello from your friendly Jenkins Hosting Checker

It appears you have some issues with your hosting request. Please see the list below and correct all issues marked Required. Your hosting request will not be approved until these issues are corrected. Issues marked with Warning or Info are just recommendations and will not stall the hosting process.

  • ⛔ Required: The following usernames in 'Jenkins project users to have release permission' need to log into Jira: cloudaeye (reports are re-synced hourly, wait to re-check for a bit after logging in)
  • ⛔ Required: The following usernames in 'Jenkins project users to have release permission' need to log into Artifactory: cloudaeye (reports are re-synced hourly, wait to re-check for a bit after logging in)
  • ⛔ Required: The <groupId> from the pom.xml should be io.jenkins.plugins instead of com.cloudaeye.jenkins

You can re-trigger a check by editing your hosting request or by commenting /hosting re-check

@jenkins-cert-app
Copy link
Collaborator

The Jenkins Security Scan discovered 5 finding(s) 🔍.

For every identified issue, please do one of the following:

  • Implement the recommended fix to address the issue.
  • If you think it's a false positive, suppress the warning directly within the code.
  • Alternative, you write an explanation here about why you think it's irrelevant. That will require a manual review, leading to a slower process.

After addressing the findings through one of the above methods:

  • If all modifications have been made to the code, please initiate a new security scan by triggering the /request-security-scan command.
  • If there are any unresolved findings (those not corrected or suppressed), request a review from the Jenkins security team by using the /audit-review command.

Stapler: Missing permission check

You can find detailed information about this finding here.

GlobalKeyConfiguration.java#91
Potential missing permission check in GlobalKeyConfiguration#doTestConnection

Jenkins: Plaintext password storage

You can find detailed information about this finding here.

CloudAEyeNotifications.java#36
Field should be reviewed whether it stores a password and is serialized to disk: token
CloudAEyeNotifications.java#34
Field should be reviewed whether it stores a password and is serialized to disk: tenantKey
GlobalKeyConfiguration.java#35
Field should be reviewed whether it stores a password and is serialized to disk: token
GlobalKeyConfiguration.java#34
Field should be reviewed whether it stores a password and is serialized to disk: tenantKey

@jenkins-cert-app jenkins-cert-app added security-audit-needs-correction The security audit revealed issues that must be corrected from the hosting request and removed security-audit-todo The security team needs to audit the hosting request code labels Oct 14, 2024
@bhavyalatha26
Copy link
Author

/hosting re-check

Copy link

Hello from your friendly Jenkins Hosting Checker

It appears you have some issues with your hosting request. Please see the list below and correct all issues marked Required. Your hosting request will not be approved until these issues are corrected. Issues marked with Warning or Info are just recommendations and will not stall the hosting process.

  • ⛔ Required: The following usernames in 'Jenkins project users to have release permission' need to log into Jira: cloudaeye (reports are re-synced hourly, wait to re-check for a bit after logging in)
  • ⛔ Required: The following usernames in 'Jenkins project users to have release permission' need to log into Artifactory: cloudaeye (reports are re-synced hourly, wait to re-check for a bit after logging in)

You can re-trigger a check by editing your hosting request or by commenting /hosting re-check

@bhavyalatha26
Copy link
Author

/request-security-scan

@jenkins-cert-app jenkins-cert-app added security-audit-todo The security team needs to audit the hosting request code and removed security-audit-needs-correction The security audit revealed issues that must be corrected from the hosting request labels Oct 14, 2024
@jenkins-cert-app
Copy link
Collaborator

The Jenkins Security Scan did not find anything dangerous with your plugin, congratulations! 🎉


💡 The Security team recommends that you are setting up the scan in your repository by following our guide.

@jenkins-cert-app jenkins-cert-app added security-audit-done The hosting request code passed the security audit with success and removed security-audit-todo The security team needs to audit the hosting request code labels Oct 14, 2024
@bhavyalatha26
Copy link
Author

/hosting re-check

Copy link

Hello from your friendly Jenkins Hosting Checker

It looks like you have everything in order for your hosting request. A member of the Jenkins hosting team will check over things that I am not able to check(code review, README content, etc) and process the request as quickly as possible. Thank you for your patience.

Hosting team members can host this request with /hosting host

@github-actions github-actions bot added bot-check-complete Automated hosting checks passed and removed needs-fix labels Oct 14, 2024
@mawinter69
Copy link
Contributor

@mawinter69
Copy link
Contributor

mawinter69 commented Oct 14, 2024

@mawinter69
Copy link
Contributor

Please provide a meaningful description here: https://github.com/CloudAEye/cloudaeye-plugin/blob/a26b6638f8375a81ab4a6d12dbe46b77c302f92b/src/main/resources/index.jelly#L3
This is what is shown to users in the Jenkins UI when they look at the pluginmanager.

@bhavyalatha26
Copy link
Author

No our plugin for both free-style and pipeline, you can find it in implementation ( https://github.com/CloudAEye/cloudaeye-plugin/blob/a26b6638f8375a81ab4a6d12dbe46b77c302f92b/src/main/java/io/jenkins/plugins/cloudaeye/CloudAEyeNotifications.java#L198) and also in README instructions on how to use it pipeline

@bhavyalatha26
Copy link
Author

Please provide a meaningful description here: https://github.com/CloudAEye/cloudaeye-plugin/blob/a26b6638f8375a81ab4a6d12dbe46b77c302f92b/src/main/resources/index.jelly#L3 This is what is shown to users in the Jenkins UI when they look at the pluginmanager.

Sure will update it.

@bhavyalatha26
Copy link
Author

@mawinter69 are there any other suggestions that we need to improve on so that we can finish all of them together at once and resubmit for review ?

@mawinter69
Copy link
Contributor

No our plugin for both free-style and pipeline, you can find it in implementation ( https://github.com/CloudAEye/cloudaeye-plugin/blob/a26b6638f8375a81ab4a6d12dbe46b77c302f92b/src/main/java/io/jenkins/plugins/cloudaeye/CloudAEyeNotifications.java#L198) and also in README instructions on how to use it pipeline

If you support pipeline you should annotate the descriptor with @Symbol('cloudAEyeNotification') (singular as you send only one request) at https://github.com/CloudAEye/cloudaeye-plugin/blob/a26b6638f8375a81ab4a6d12dbe46b77c302f92b/src/main/java/io/jenkins/plugins/cloudaeye/CloudAEyeNotifications.java#L321
This allows you to write in a pipeline
cloudAEyeNotification() instead of step([$class: 'CloudAEyeNotifications'])
Also change line 327 so that it works with Job instead of AbstractProject. Otherwise I think the step will not appear in the Snippet Generator.

The plugin seems to be focused on git especially github. There are many other SCMs around like gerrit, gitlab, bitbucket, perforce and many more

@bhavyalatha26
Copy link
Author

Please provide a meaningful description here: https://github.com/CloudAEye/cloudaeye-plugin/blob/a26b6638f8375a81ab4a6d12dbe46b77c302f92b/src/main/resources/index.jelly#L3 This is what is shown to users in the Jenkins UI when they look at the pluginmanager.

Sure will update it.

Done

@bhavyalatha26
Copy link
Author

Committed most of the suggested changes. Here are few that we did not / failed to:

  1. [In Progress] Migrating to secrets for token

  2. Remove redundant check box: We want to keep this because in upcoming updates, we need this feature to help user select some preferences based on this

  3. Also change line 327 so that it works with Job instead of AbstractProject. Otherwise I think the step will not appear in the Snippet Generator.

    Changing it to Job was throwing error:

    Class 'DescriptorImpl' must either be declared abstract or implement abstract method 'isApplicable(Class<? extends AbstractProject>)' in 'BuildStepDescriptor'
    
  4. https://github.com/CloudAEye/cloudaeye-plugin/blob/a26b6638f8375a81ab4a6d12dbe46b77c302f92b/src/main/java/io/jenkins/plugins/cloudaeye/CloudAEyeNotifications.java#L99 is there a missing return statement?
    https://github.com/CloudAEye/cloudaeye-plugin/blob/a26b6638f8375a81ab4a6d12dbe46b77c302f92b/src/main/java/io/jenkins/plugins/cloudaeye/CloudAEyeNotifications.java#L132 same here missing return?

    Since it is a void function, we are returning nothing here.

  5. https://github.com/CloudAEye/cloudaeye-plugin/blob/a26b6638f8375a81ab4a6d12dbe46b77c302f92b/src/main/java/io/jenkins/plugins/cloudaeye/CloudAEyeNotifications.java#L155 Are you sure you want to use run.getLogText() this will return annotated text with html. It is meant for use in the UI. You might want to use run.getLog which already returns an array with lines and has annotations removed.

    Actually we did not want to use getLogText(), but getLog(int n) always asks for number of lines as input which we are not sure how to get the total lines. Is there a way to know and pass number of lines to that function or a way to tell just give all lines generated ?

  6. The plugin seems to be focused on git especially github. There are many other SCMs around like gerrit, gitlab, bitbucket, perforce and many more

    Yes this was intentional because on CloudAEye we right now support only github but not other providers

Regards,
Bhavya

@mawinter69
Copy link
Contributor

  1. https://github.com/CloudAEye/cloudaeye-plugin/blob/a26b6638f8375a81ab4a6d12dbe46b77c302f92b/src/main/java/io/jenkins/plugins/cloudaeye/CloudAEyeNotifications.java#L99 is there a missing return statement?

but you don't want to execute anything afterwards so if you don't add a return statement the rest of the method will still be executed so it should look like this I think

        if(!this.getEnableExport()){
            LOGGER.info(MessageFormat.format("[#{0}] Exporting to CloudAEye is not enabled. Skipping export", run.getNumber()));
            return;
        }
  1. https://github.com/CloudAEye/cloudaeye-plugin/blob/a26b6638f8375a81ab4a6d12dbe46b77c302f92b/src/main/java/io/jenkins/plugins/cloudaeye/CloudAEyeNotifications.java#L155 Are you sure you want to use run.getLogText() this will return annotated text with html. It is meant for use in the UI. You might want to use run.getLog which already returns an array with lines and has annotations removed.

Actually we did not want to use getLogText(), but getLog(int n) always asks for number of lines as input which we are not sure how to get the total lines. Is there a way to know and pass number of lines to that function or a way to tell just give all lines generated ?

You could use run.getLog(Integer.MAX_VALUE), so you get the complete log, if there are less lines than you requested it will not fail. Beware that logs can be huge (I have builds that produce logs that are several hundred MiB). You might want to make this configurable, I could imagine that for build error analysis you usually don't need the complete log but only the last few lines.

@bhavyalatha26
Copy link
Author

bhavyalatha26 commented Oct 15, 2024

  1. https://github.com/CloudAEye/cloudaeye-plugin/blob/a26b6638f8375a81ab4a6d12dbe46b77c302f92b/src/main/java/io/jenkins/plugins/cloudaeye/CloudAEyeNotifications.java#L99 is there a missing return statement?

but you don't want to execute anything afterwards so if you don't add a return statement the rest of the method will still be executed so it should look like this I think

        if(!this.getEnableExport()){
            LOGGER.info(MessageFormat.format("[#{0}] Exporting to CloudAEye is not enabled. Skipping export", run.getNumber()));
            return;
        }
5. > https://github.com/CloudAEye/cloudaeye-plugin/blob/a26b6638f8375a81ab4a6d12dbe46b77c302f92b/src/main/java/io/jenkins/plugins/cloudaeye/CloudAEyeNotifications.java#L155 Are you sure you want to use run.getLogText() this will return annotated text with html. It is meant for use in the UI. You might want to use run.getLog which already returns an array with lines and has annotations removed.

Actually we did not want to use getLogText(), but getLog(int n) always asks for number of lines as input which we are not sure how to get the total lines. Is there a way to know and pass number of lines to that function or a way to tell just give all lines generated ?

You could use run.getLog(Integer.MAX_VALUE), so you get the complete log, if there are less lines than you requested it will not fail. Beware that logs can be huge (I have builds that produce logs that are several hundred MiB). You might want to make this configurable, I could imagine that for build error analysis you usually don't need the complete log but only the last few lines.

Yes thanks for pointing it out, I also was thinking in same lines that the logs can be huge, is there a way to identify the failed step and only extract logs of that failed step ? I could achieve this with github workflows and AWS codebuild, not sure how to do it with jenkins

@bhavyalatha26
Copy link
Author

New updates:

  • Replaced string with secrets wherever necessary
  • Shifted to using getLogs(Integer.MAX_VALUE) to get all logs instead of annotated logs

@jenkins-cert-app jenkins-cert-app added security-audit-todo The security team needs to audit the hosting request code and removed security-audit-done The hosting request code passed the security audit with success labels Oct 15, 2024
@bhavyalatha26
Copy link
Author

@mawinter69 Can you please review the updated code and let us know any changes required

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bot-check-complete Automated hosting checks passed hosting-request Request to host a component in jenkinsci security-audit-todo The security team needs to audit the hosting request code
Projects
None yet
Development

No branches or pull requests

3 participants