Modernize CI/CD workflow

[jakob-keller]

Description of Change

Modernize CI/CD workflow based upon recommendations from https://docs.github.com/en/actions/use-cases-and-examples/building-and-testing/building-and-testing-python#publishing-to-pypi and @webknjaz

• Replace python setup.py by build
• Build unconditionally and validate metadata (closes Perform metadata validation in CI/CD workflow #1164)
• Use deployment environment pypi
• much more - your reviews are gold, @webknjaz!

Assumptions

None

Checklist for All Submissions

• I have added change info to CHANGES.rst
• If this is resolving an issue (needed so future developers can determine if change is still necessary and under what conditions) (can be provided via link to issue with these details):
  • Detailed description of issue
  • Alternative methods considered (if any)
  • How issue is being resolved
  • How issue can be reproduced
• If this is providing a new feature (can be provided via link to issue with these details):
  • Detailed description of new feature
  • Why needed
  • Alternatives methods considered (if any)

Checklist when updating botocore and/or aiohttp versions

• I have read and followed CONTRIBUTING.rst
• I have updated test_patches.py where/if appropriate (also check if no changes necessary)
• I have ensured that the awscli/boto3 versions match the updated botocore version

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 86.93%. Comparing base (1fcc8b9) to head (0e6dd62).
Report is 3 commits behind head on master.

Additional details and impacted files

@@           Coverage Diff           @@
##           master    #1163   +/-   ##
=======================================
  Coverage   86.93%   86.93%           
=======================================
  Files          64       64           
  Lines        5978     5978           
=======================================
  Hits         5197     5197           
  Misses        781      781           

Flag	Coverage Δ
unittests	86.93% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[jakob-keller]

@thehesiod: From my point of view, now would be a good time to cut a release. What do you think?

[jakob-keller]

With regards to the previously used PyPI credentials: we should both revoke them over at PyPI as well as remove them from this repo's settings.

[thehesiod]

@thehesiod: From my point of view, now would be a good time to cut a release. What do you think?

are there any substantial changes for users that would be included?

[jakob-keller]

@thehesiod: From my point of view, now would be a good time to cut a release. What do you think?

are there any substantial changes for users that would be included?

Anything strictly user-facing is included in CHANGES.rst. Beyond that, there are plenty of contributor-facing changes which are addressed by updated documentation. Not sure if those need to go into CHANGES.rst, too.

[jakob-keller]

It's quite a lot of commits and sizeable diff: 2.13.2...master

[thehesiod]

It's quite a lot of commits and sizeable diff: 2.13.2...master

ya we have code changes, sounds good!

[webknjaz]

@jakob-keller in other projects, we sometimes make an alpha-release to safely test out the pipeline. I think you could do that here as well, to get familiar with the updates. You can do that from a separate branch if you don't want a commit on main.

[jakob-keller]

2.13.3-alpha it is: https://github.com/aio-libs/aiobotocore/releases/tag/2.13.3-alpha

[jakob-keller]

Guess it didn't trigger due to the restrictive on push tags filter. Will do some experiments...

[jakob-keller]

Actually, our release process is pretty coarse: It does not involve any commits at all, just a tag. I would skip the alpha and go straight to 2.13.3. The only thing that could go wrong IMO is the publishing to PyPI, most likely due to permissions. In that case, we can fix it and try again, YOLO!

[webknjaz]

I deleted the repository secret, however the bot account on PyPI did not have a token. So it must be under somebody else's human account.

[jakob-keller]

There we go: https://github.com/aio-libs/aiobotocore/releases/tag/2.13.3

@thehesiod: Feel free to edit the What's Changed section, if it looks too verbose.

[jakob-keller]

I deleted the repository secret, however the bot account on PyPI did not have a token. So it must be under somebody else's human account.

I believe I saw @thehesiod being mentioned in #1116

[jakob-keller]

Nice, I didn't know there was a manual review step involved. LGTM!

[webknjaz]

2.13.3-alpha

That's not compliant with PEP 440. It would be 2.13.3.a0 in Python ecosystem.

[webknjaz]

Actually, our release process is pretty coarse: It does not involve any commits at all, just a tag.

Well, since this isn't using setuptools-scm (maybe it should be integrated — I usually default to having it), I would imagine that the version number bump would have to be committed and then that commit would need to be tagged. Otherwise, they'd be out of sync.

[webknjaz]

I would imagine that the version number bump would have to be committed and then that commit would need to be tagged. Otherwise, they'd be out of sync.

Additionally, that commit might make the change log consistent. In projects with Towncrier, it'd be generated from fragments.

[jakob-keller]

2.13.3-alpha

That's not compliant with PEP 440. It would be 2.13.3.a0 in Python ecosystem.

Good to know, I simply went with GitHub UI's suggestion.

Actually, our release process is pretty coarse: It does not involve any commits at all, just a tag.

Well, since this isn't using setuptools-scm (maybe it should be integrated — I usually default to having it), I would imagine that the version number bump would have to be committed and then that commit would need to be tagged. Otherwise, they'd be out of sync.

We still do that manually, prior to release.

[jakob-keller]

I would imagine that the version number bump would have to be committed and then that commit would need to be tagged. Otherwise, they'd be out of sync.

Additionally, that commit might make the change log consistent. In projects with Towncrier, it'd be generated from fragments.

I agree, but that's probably #1167.

[webknjaz]

In projects, where the version is hardcoded, it's usually bumped to the next .dev0 after each release, and then, turned to stable before the next one.

[jakob-keller]

It worked like a charm: https://pypi.org/project/aiobotocore/

[jakob-keller]

Thank you @thehesiod and @webknjaz for your efforts!

[webknjaz]

2.13.3-alpha

That's not compliant with PEP 440. It would be 2.13.3.a0 in Python ecosystem

Good to know, I simply went with GitHub UI's suggestion.

If you didn't update the version in code, that what would've ended up being used. And if you'd have burned that version number, and you'd need to bump to the next stable because the release artifacts are immutable on PyPI. Good thing it didn't trigger.

[webknjaz]

Nice, I didn't know there was a manual review step involved. LGTM!

That's how I set up GitHub Environments for publishing. And that's the reason I insisted on having them. The old process didn't have this since this feature is specific to having Environments integrated.

[thehesiod]

There we go: https://github.com/aio-libs/aiobotocore/releases/tag/2.13.3

@thehesiod: Feel free to edit the What's Changed section, if it looks too verbose.

I typically don't really include internal related changes as they're not relevant for the consumer of the library, just add noise to end-users. Perhaps we can add internal changes in a collapsible section

[thehesiod]

2.13.3-alpha

That's not compliant with PEP 440. It would be 2.13.3.a0 in Python ecosystem.

@jakob-keller we should have a test to ensure the CI fails if the version is invalid (can parse with disttools or something similar). Can do this next to our existing version checks

[jakob-keller]

There we go: https://github.com/aio-libs/aiobotocore/releases/tag/2.13.3
@thehesiod: Feel free to edit the What's Changed section, if it looks too verbose.

I typically don't really include internal related changes as they're not relevant for the consumer of the library, just add noise to end-users. Perhaps we can add internal changes in a collapsible section

I split the release notes as suggested.

[jakob-keller]

2.13.3-alpha

That's not compliant with PEP 440. It would be 2.13.3.a0 in Python ecosystem.

@jakob-keller we should have a test to ensure the CI fails if the version is invalid (can parse with disttools or something similar). Can do this next to our existing version checks

Opened as #1177

[webknjaz]

Here, I came up with some extra granular sections that may target some changelog reading audiences that would otherwise be treated as internal: https://yarl.aio-libs.org/en/latest/changes/#id7.