Experimental macOS/MPS support #200
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Update: It seems like transformers 4.42 was causing the issue, but I have no idea why. |
tmills
merged commit 9f83b10
into
Machine-Learning-for-Medical-Language:main
9 checks passed
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR adds experimental MPS support, including updated package dependencies for macOS and MPS acceleration for models served by the API.
The following dependencies have been updated in
setup.cfg:PyRuSH:1.0.3.6→1.0.8An issue with the build script in older versions of PyRuSH prevented the wheel from building on macOS
torch:~=1.5→>=1.5, <=2.3.1More current versions of torch have much improved mps support, with more operations supported by the backend.
I realize this dependency now spans two major versions of pytorch... should we consider committing to 2.x?
transformers:~=4.15→>=4.15, <4.42transformers 4.42 seems to be causing the following error on mps when loading models:
RuntimeError: Placeholder storage has not been allocated on MPS device!This error doesn't occur with transformers <4.42.
This PR also adds some checks to use the MPS device when possible, specifically for models served by the API. Training scripts already use MPS by default because we're using the HF Trainer class, which does this for us.
This implementation Works On My Machine™, but more testing is needed to make sure this definitively solves all the build dependency issues. I'm opening this on main rather than dev because all it should do is increase compatibility for macOS users—but let me know if adapting this for the latest dev version makes more sense.