Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cache in git workflow not speeding up subsequent runs #641

Open
aeppacher opened this issue Dec 2, 2021 · 5 comments
Open

Cache in git workflow not speeding up subsequent runs #641

aeppacher opened this issue Dec 2, 2021 · 5 comments

Comments

@aeppacher
Copy link

Our linting on our project is taking upwards of 4-5 minutes to run in git workflow. I thought a good optimization would be to cache the xo cache per branch and update it incrementally. That way subsequent commits will only have their linter run against that branches diff, the same way you would have it for local development.

My solution was to save the node_modules/.cache/xo-linter folder at the end of every successful run, and to restore it per branch on every subsequent run. This works fine, but the linting doesn't seem to be faster which makes me think the cache is somehow being invalidated.

I looked through the source to try and find how the cache invalidation works but I can only see code about creating a config which has the cache folder and passing that to eslint. Is this something that is caused by eslint, or any tips on how I might achieve this?

@spence-s
Copy link
Contributor

spence-s commented Dec 2, 2021

Caching in xo comes fully from eslint - and by default invalidates caches by file metadata. So copying/moving the cache between runs likely will invalidate it every time. See:

https://github.com/eslint/eslint/blob/a1f7ad77e2/lib/cli-engine/lint-result-cache.js
https://www.npmjs.com/package/file-entry-cache

Perhaps something like lint-staged could help you speed things up.

Or if that is too simple, perhaps we could expose cache options to cache with file content strategy from eslint which would probably resolve your use case, but I would leave that up to @sindresorhus - PRs are probably welcome.

@sindresorhus
Copy link
Member

Yes, we could support a cacheStrategy option.

https://eslint.org/docs/user-guide/command-line-interface#--cache-strategy

@aeppacher
Copy link
Author

@sindresorhus that sounds like exactly what I want. Would be a great enhancement for remote compilation users as well that might also see invalidation of meta data between runs.

@GriffinSauce
Copy link

Just to add some findings: lint-staged by default does some git stashing stuff for backups. It seems like this changes the metadata for files and defeats the ESLint cache.

To make lint-staged work with the cache you need to run it with --no-stash. This makes runs (after the first) much faster.

I'd rather run XO/ESLint with --cache-strategy content and keep the lint-staged backup logic intact. (you could argue this is an issue on their end as well but ... 🤷‍♂️ )

@EC-9624
Copy link

EC-9624 commented Jul 25, 2024

I'm facing the same problem while trying to speed up our CI by moving cache files around, but xo doesn't recognize the cache file. It seems like the cache-strategy content could solve this issue. Additionally, support for cache-locations to skip moving cache files around might also simplify the workflow.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

5 participants