Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ci: compliance: fetch only the last revision and skip tags #64457

Merged

Conversation

fabiobaltieri
Copy link
Member

@fabiobaltieri fabiobaltieri commented Oct 26, 2023

Add some west update flags to do a shallow fetch of the modules and skip the tags. That data is not needed anyway, should make the compliance check initialization a bit faster.


Speeds up the "west setup" step from ~3:30m to ~1m40.

@zephyrbot zephyrbot added Trivial Changes that can be reviewed by anyone, i.e. doc changes, minor build system tweaks, etc. area: Continuous Integration labels Oct 26, 2023
@nashif nashif changed the title ci: compliance: fech only the last revision and skip tags ci: compliance: fetch only the last revision and skip tags Oct 26, 2023
Add some west update flags to do a shallow fetch of the modules and
skip the tags. That data is not needed anyway, should make the
compliance check initialization a bit faster.

Signed-off-by: Fabio Baltieri <[email protected]>
Copy link
Collaborator

@marc-hb marc-hb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This would work only for PRs that have a single commit

./scripts/ci/check_compliance.py -h

options:
  -h, --help            show this help message and exit
  -c COMMITS, --commits COMMITS
                        Commit range in the form: a..[b], default is HEAD~1..HEAD

Try west update --fetch-opt=--filter=tree:0

We've been using that for a long time and it worked great in CI

@marc-hb marc-hb dismissed their stale review October 26, 2023 22:25

Just realized this is only for modules sorry

@fabiobaltieri
Copy link
Member Author

@marc-hb thanks for the comment - yes it's only for the modules so it should work fine, I actualy never used the filter option, will look into that, thanks for the pointers. :-)

I did look into optimizing the main checkout as well but it's tricky to do a shallow fetch and not mess up the rebase step, maybe there's some clever filter that could be use to expand the history on top of a shallow fetch to just the right amount.

@marc-hb
Copy link
Collaborator

marc-hb commented Oct 26, 2023

maybe there's some clever filter that could be use to expand the history on top of a shallow fetch to just the right amount.

It is when trying exactly that that I ended up with Pull Requests with a million commits, see picture in

Granted: this was in a repo with a lot of git merges and other git "backflips". It wouldn't happen with a linear history like Zephyr's.

I thought there would be a Github Action ready to "Just give me all the commits in the PR" but I later realized it is actually quite tricky in the general case.

https://github.com/thesofproject/sof/blob/main/.github/workflows/shallowfetchPRcommits.sh has been working fine for a long time but again this has been on a mostly linear git history. It's also an ugly shellscript, not a proper Github Action. Note a shell script is probably easier to test locally..

See also (too) long discussion in

@carlescufi carlescufi merged commit 2dcb128 into zephyrproject-rtos:main Oct 30, 2023
12 checks passed
@fabiobaltieri fabiobaltieri deleted the compliance-west-update branch October 30, 2023 08:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area: Continuous Integration Trivial Changes that can be reviewed by anyone, i.e. doc changes, minor build system tweaks, etc.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants