Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Instructions for DEB/RPM build, doing the temporary workaround until the scripts are fully working. #1594

Open
wants to merge 6 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
70 changes: 69 additions & 1 deletion RELEASING.md
Original file line number Diff line number Diff line change
Expand Up @@ -112,4 +112,72 @@ git push
are properly added to the `main` branch.

6. Create a GitHub release from the tag. See [GitHub documentation](https://docs.github.com/en/repositories/releasing-projects-on-github/managing-releases-in-a-repository#creating-a-release)
for instructions.
for instructions.

### DEB/RPM Packaging Workflow

#### Pre-requisites

Before building the packages:

* ensure that the release itself, either the alpha or prod versions, is published on PyPI.
* VPN is enabled to access the build page.

#### Build Process

To build the alpha DEB/RPM packages after the alpha PyPI is released, simply put
the tag name ie. ``v2.29.0a0`` or ``v2.29.0`` into the input textbox of the
[build page here](https://builds.globus.org/jenkins/job/BuildGlobusComputeAgentPackages/build?delay=0sec).
then clicking the green **Build** button.

For production builds, check the ``BUILD_FOR_STABLE`` box.

1. Notes
Our alpha builds will go to the ``unstable`` repo, and production packages goes
to both the ``testing`` and ``stable`` repos.

After this build process for production, the testing and stable packages will
reside in an internal globus 'holding' repo. GCS manages the infrastructure
so we need to run another Jenkins build to push it to live if GCS is not having
a release the same week. TBD details

* Example of unstable repo:
* https://downloads.globus.org/globus-connect-server/unstable/rpm/el/9/x86_64/
* Directory of testing images:
* https://builds.globus.org/downloads.globus.org/globus-connect-server/testing/rpm/el/8/x86_64/
* Stable repo:
* The images will be in the below build directory after we finish our build process, but not public:
* https://builds.globus.org/downloads.globus.org/globus-connect-server/stable/rpm/el/8/x86_64/
* After GCS push during deploy day (or if we ping them to do so), the public images will be located at:
* https://downloads.globus.org/globus-connect-server/stable/rpm/el/9/x86_64/
<<<<<<< HEAD
[publishResults.groovy line 85](https://github.com/globusonline/gcs-build-scripts/blob/168617a0ccbb0aee7b3bee04ee67940bbe2a80f6/vars/publishResults.groovy#L85)
2. (Access on VPN) Click the [build button here](https://builds.globus.org/jenkins/job/BuildGlobusComputeAgentPackages/build?delay=0sec)
3. Wait 20-30 minutes and confirm that the [build is green](https://builds.globus.org/jenkins/job/BuildGlobusComputeAgentPackages/)
4. For production release, we will have finished the build before the GCS team has
=======

1. Wait 20-30 minutes and confirm that the [build is green](https://builds.globus.org/jenkins/job/BuildGlobusComputeAgentPackages/)
>>>>>>> dd04032c (Document package release procedure)

#### Old Build Instructions

As a temporary workaround, we needed to add a few lines to manually set some
env variables in our [JenkinsFile](https://github.com/globus/globus-compute/blob/743fa1e398fd40a00efb5880c55e3fa6e47392fc/compute_endpoint/packaging/JenkinsFile#L24) before triggering the build, as detailed below.

1. Notes
* Example of unstable repo:
* https://downloads.globus.org/globus-connect-server/unstable/rpm/el/9/x86_64/
* Directory of testing images:
* https://builds.globus.org/downloads.globus.org/globus-connect-server/testing/rpm/el/8/x86_64/
* Stable repo:
* The images will be in the below build directory after we finish our build process, but not public:
* https://builds.globus.org/downloads.globus.org/globus-connect-server/stable/rpm/el/8/x86_64/
* After GCS push during deploy day (or if we ping them to do so), the public images will be located at:
* https://downloads.globus.org/globus-connect-server/stable/rpm/el/9/x86_64/
[publishResults.groovy line 85](https://github.com/globusonline/gcs-build-scripts/blob/168617a0ccbb0aee7b3bee04ee67940bbe2a80f6/vars/publishResults.groovy#L85)
2. (Access on VPN) Click the [build button here](https://builds.globus.org/jenkins/job/BuildGlobusComputeAgentPackages/build?delay=0sec)
3. Wait 20-30 minutes and confirm that the [build is green](https://builds.globus.org/jenkins/job/BuildGlobusComputeAgentPackages/)
4. For production release, we will have finished the build before the GCS
team, and can notify them that our build is complete. They then will
publish all packages when they finish their builds, which includes ours.
2 changes: 1 addition & 1 deletion compute_endpoint/globus_compute_endpoint/version.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# single source of truth for package version,
# see https://packaging.python.org/en/latest/single_source_version/
__version__ = "2.29.0"
__version__ = "2.30.0a0"

# TODO: remove after a `globus-compute-sdk` release
# this is needed because it's imported by `globus-compute-sdk` to do the version check
Expand Down
8 changes: 8 additions & 0 deletions compute_endpoint/packaging/JenkinsFile
Original file line number Diff line number Diff line change
Expand Up @@ -22,10 +22,18 @@ pipeline {
script {
def scmVars = checkout scm
env.BRANCH_NAME = scmVars.GIT_BRANCH.replaceFirst(/^.*origin\//, "")
env.TAG_NAME = sh(returnStdout: true, script: "git tag --contains | head -1").trim()
env.SOURCE_STASH_NAME = "${UUID.randomUUID()}"
echo "env.BRANCH_NAME = ${env.BRANCH_NAME}"
echo "env.DEFAULT_BRANCH = ${env.DEFAULT_BRANCH}"
sh "git clean -fdx"


// temporary hack to build for stable
sh "git checkout build_for_stable"
env.TAG_NAME = "2.30.0"
env.DEFAULT_BRANCH = "build_for_stable"

dir("compute_endpoint/packaging/") {
sh(script: "make show_vars")
sh(script: "make setup_dist_for_deb setup_dist_for_rpm")
Expand Down
2 changes: 1 addition & 1 deletion compute_endpoint/setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
REQUIRES = [
"requests>=2.31.0,<3",
"globus-sdk", # version will be bounded by `globus-compute-sdk`
"globus-compute-sdk==2.29.0",
"globus-compute-sdk==2.30.0a0",
"globus-compute-common==0.4.1",
"globus-identity-mapping==0.3.0",
# table printing used in list-endpoints
Expand Down
2 changes: 1 addition & 1 deletion compute_sdk/globus_compute_sdk/version.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@

# single source of truth for package version,
# see https://packaging.python.org/en/latest/single_source_version/
__version__ = "2.29.0"
__version__ = "2.30.0a0"


def compare_versions(
Expand Down
Loading