feat: Move release note generation to a sub module #3299

JoeWang1127 · 2024-10-17T14:48:22Z

In this PR:

Separate projects to three modules: common, library_generation and release_note_generation
Add generate_release_note.py to separate PR description generation from entrypoint.py.
Only install common and library_generation module in image.
Remove PR description comparison in integration test.

Example generation workflow in downstream library: https://github.com/googleapis/java-bigtable/actions/runs/11442581326/job/31833887512

Note that release_note_generation module still depends on library_generation module because we didn't separate config change functions into common module. This will be in a follow up PR.

JoeWang1127 · 2024-10-22T18:49:50Z

.github/workflows/verify_library_generation.yaml

+        pip install --upgrade pip
+        pip install hermetic_build/common
+        pip install hermetic_build/library_generation
+        pip install hermetic_build/release_note_generation


How about we keep the dependencies together.

Created requirements.in and requirements.txt in each module.

blakeli0 · 2024-10-22T22:33:23Z

.github/scripts/action.yaml

+    shell: bash
+    run: |
+      cd "${GITHUB_WORKSPACE}"
+      pip install --upgrade pip


Do we have to upgrade pip? Can this be done in previous steps like setup-python?

According to this comment, pip is not updated in this action.

As for pip, we don't need it to be updated. I'll remove this line.

blakeli0 · 2024-10-22T22:34:43Z

.github/scripts/action.yaml

+      pip install --require-hashes -r hermetic_build/common/requirements.txt
+      pip install hermetic_build/common
+      pip install --require-hashes -r hermetic_build/library_generation/requirements.txt
+      pip install hermetic_build/library_generation


I don't think we need to install library_generation anymore once we move ConfigChange out of this module?

Yes. We need to install library_generation because config change is still part of it. Once it moves to common, we don't need to install it.

diegomarquezp

Thanks for this PR. LGTM so far - just some minor comments.

diegomarquezp · 2024-10-23T13:10:24Z

.github/workflows/verify_library_generation.yaml

-        echo "PATH=${PATH}" >> $GITHUB_ENV
-
-        set +ex
+        python-version: 3.12
    - name: install python dependencies


Suggested change

- name: install python dependencies

- name: install python modules and dependencies

diegomarquezp · 2024-10-23T13:12:28Z

.github/workflows/verify_library_generation.yaml

@@ -80,23 +69,26 @@ jobs:
    - uses: actions/checkout@v4
    - uses: actions/setup-python@v5
      with:
-        python-version: 3.11
+        python-version: 3.12
    - name: install python dependencies


Suggested change

- name: install python dependencies

- name: install python modules and dependencies

diegomarquezp · 2024-10-23T13:14:10Z

hermetic_build/common/requirements.txt

@@ -0,0 +1,109 @@
+#
+# This file is autogenerated by pip-compile with Python 3.11


What are the implications of generating a reqs file with a different python version? Maybe we can create one with 3.12 instead?

pip-tools (which contains pip-compile) is built with python 3.11.

You can confirm this by running which pip-compile.

Ah, looks like pip-compile is available only for 3.11, but it's installable via pip install pip-tools from 3.12 onwards. I managed to locally generate the hashes with python 3.12. Can you confirm this works?

pyenv install 3.12 pyenv local 3.12 python --version pip install pip-tools pip-compile --generate-hashes

Thanks.

I changed the requirements.txt using python 3.12.

diegomarquezp · 2024-10-23T13:20:15Z

hermetic_build/library_generation/DEVELOPMENT.md

@@ -21,9 +21,9 @@ code declared in a "golden branch" of the repo.
 It requires docker and python 3.x to be installed.

 ```
+python -m pip install --require-hashes -r requirements.txt
 python -m pip install .


The development guide shows commands to be run from the sdk-platform-java/library_generation folder. However this command now must be run from sdk-platform-java/hermetic_build/library_generation (or change pip install . to pip install library_generation and run it from //hermetic_build. We can modify this note to point to hermetic_build.

With this, then we could correct this to

Suggested change

python -m pip install .

python -m pip install library_generation

diegomarquezp · 2024-10-23T13:28:12Z

hermetic_build/library_generation/README.md

@@ -89,14 +89,6 @@ will be created/modified:
 | pom.xml (repo root dir)             | Always generated from inputs                                             |
 | versions.txt                        | New entries will be added if they don’t exist                            |

-### Change history


IIUC this feature is not being actually removed. Should we keep it?

Change history is not generated by entry_point.py so I think we don't need to include this part.

I plan to raise another PR to consolidate library generation and release note generation into a README in hermetic_build directory.

Ah gotcha, I was missing the fact that this readme is in the library_generation folder.

diegomarquezp · 2024-10-23T13:30:14Z

hermetic_build/library_generation/requirements.txt

@@ -0,0 +1,417 @@
+#
+# This file is autogenerated by pip-compile with Python 3.11


same concern about python versions

sonarcloud · 2024-10-23T18:40:17Z

Quality Gate passed for 'gapic-generator-java-root'

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarCloud

sonarcloud · 2024-10-23T18:43:26Z

Quality Gate passed for 'java_showcase_integration_tests'

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarCloud

🤖 I have created a release *beep* *boop* --- <details><summary>2.49.0</summary> ## [2.49.0](v2.48.0...v2.49.0) (2024-10-25) ### Features * Move release note generation to a sub module ([#3299](#3299)) ([7d6d66a](7d6d66a)) ### Bug Fixes * add additional potential exceptions when retrieving protobuf manifest file to get version ([#3315](#3315)) ([ef9e518](ef9e518)) ### Dependencies * update dependency com.google.errorprone:error_prone_annotations to v2.35.1 ([#3316](#3316)) ([d7c290f](d7c290f)) </details> --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please). Co-authored-by: release-please[bot] <55107282+release-please[bot]@users.noreply.github.com>

JoeWang1127 added 2 commits October 17, 2024 10:17

feat: separate pr description generation from entrypoint

0cb1ee4

add unit test

7b9e493

product-auto-label bot added the size: m Pull request size is medium. label Oct 17, 2024

JoeWang1127 added 6 commits October 17, 2024 10:58

fix unit test

f74d1c5

fix integration test

99eb34a

fix integration test

c87d6fb

change shell script

c5dfb84

change entrypoint

758f0bc

add env

8fa281e

JoeWang1127 marked this pull request as ready for review October 17, 2024 17:03

JoeWang1127 requested a review from blakeli0 October 17, 2024 17:03

change doc

8ae1360

product-auto-label bot added size: l Pull request size is large. and removed size: m Pull request size is medium. labels Oct 17, 2024

JoeWang1127 added 2 commits October 17, 2024 13:20

change ci

19de9cd

change ci

ff85b71

JoeWang1127 changed the title ~~feat: separate pr description generation from entry point~~ feat: Move release note generation to a sub module Oct 17, 2024

JoeWang1127 added 10 commits October 18, 2024 08:27

create separate packages

72b3acb

setup common project

3c7e8f0

fix package

82c1642

add gapic_inputs.py to common

54baaf9

remove library_generation dir

40c12af

add library_generation module

1e2473b

add release_note_generation module

5578047

add init

deadfb5

lint

fcebe5c

change ci

0b400e9

product-auto-label bot added size: m Pull request size is medium. and removed size: l Pull request size is large. labels Oct 18, 2024

change integration in ci

b323412

JoeWang1127 marked this pull request as ready for review October 21, 2024 16:05

JoeWang1127 requested a review from diegomarquezp October 21, 2024 16:06

Merge branch 'main' into feat/separate-pr-des-gen

91f7096

JoeWang1127 commented Oct 22, 2024

View reviewed changes

create requirements.in for each module

1c9a9a7

product-auto-label bot added size: l Pull request size is large. and removed size: m Pull request size is medium. labels Oct 22, 2024

JoeWang1127 added 7 commits October 22, 2024 15:58

lint

394733f

change dockerfile

3cbe316

change dockerfile

f7559dc

change dockerfile

93ffa8a

Merge branch 'main' into feat/separate-pr-des-gen

939d8b1

Merge branch 'main' into feat/separate-pr-des-gen

2925d89

change doc

d8d1e8e

blakeli0 reviewed Oct 22, 2024

View reviewed changes

do not update pip in action

208f51f

JoeWang1127 requested a review from blakeli0 October 22, 2024 23:07

diegomarquezp reviewed Oct 23, 2024

View reviewed changes

change comment

a4beff9

JoeWang1127 requested a review from diegomarquezp October 23, 2024 14:39

update requirements.txt

80623c0

diegomarquezp approved these changes Oct 23, 2024

View reviewed changes

blakeli0 approved these changes Oct 23, 2024

View reviewed changes

JoeWang1127 merged commit 7d6d66a into main Oct 23, 2024
50 of 52 checks passed

JoeWang1127 deleted the feat/separate-pr-des-gen branch October 23, 2024 23:18

release-please bot mentioned this pull request Oct 24, 2024

chore(main): release 2.49.0 #3312

Merged

JoeWang1127 mentioned this pull request Oct 24, 2024

chore: change file path in checks googleapis/google-cloud-java#11269

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: Move release note generation to a sub module #3299

feat: Move release note generation to a sub module #3299

JoeWang1127 commented Oct 17, 2024 •

edited

Loading

JoeWang1127 Oct 22, 2024

JoeWang1127 Oct 22, 2024

blakeli0 Oct 22, 2024

JoeWang1127 Oct 22, 2024

JoeWang1127 Oct 22, 2024

blakeli0 Oct 22, 2024

JoeWang1127 Oct 22, 2024

diegomarquezp left a comment •

edited

Loading

diegomarquezp Oct 23, 2024

JoeWang1127 Oct 23, 2024

diegomarquezp Oct 23, 2024

JoeWang1127 Oct 23, 2024

diegomarquezp Oct 23, 2024

JoeWang1127 Oct 23, 2024

diegomarquezp Oct 23, 2024

JoeWang1127 Oct 23, 2024

diegomarquezp Oct 23, 2024

JoeWang1127 Oct 23, 2024

diegomarquezp Oct 23, 2024

JoeWang1127 Oct 23, 2024

diegomarquezp Oct 23, 2024

diegomarquezp Oct 23, 2024

sonarcloud bot commented Oct 23, 2024

sonarcloud bot commented Oct 23, 2024

	- name: install python dependencies
	- name: install python modules and dependencies

		@@ -0,0 +1,109 @@
		#
		# This file is autogenerated by pip-compile with Python 3.11

	python -m pip install .
	python -m pip install library_generation

		@@ -0,0 +1,417 @@
		#
		# This file is autogenerated by pip-compile with Python 3.11

feat: Move release note generation to a sub module #3299

feat: Move release note generation to a sub module #3299

Conversation

JoeWang1127 commented Oct 17, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

diegomarquezp left a comment • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sonarcloud bot commented Oct 23, 2024

Quality Gate passed for 'gapic-generator-java-root'

sonarcloud bot commented Oct 23, 2024

Quality Gate passed for 'java_showcase_integration_tests'

JoeWang1127 commented Oct 17, 2024 •

edited

Loading

diegomarquezp left a comment •

edited

Loading