Update docs #588
Update docs #588
Conversation
michaelbenayoun
changed the base branch from
sync_transformers_and_accelerate
to
main
|
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update. |
michaelbenayoun
force-pushed
the
update_docs
branch
from
3aba5aa
to
a4d257c
Compare
|
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread. Thank you! |
1 similar comment
|
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread. Thank you! |
There was a problem hiding this comment.
We should add 2 sections to this guide:
- How to enable Sequence Parallelism?
- How to enable Pipeline Parallelism?
There was a problem hiding this comment.
Let's do it in another PR to keep it small. The PR is here: #657
There was a problem hiding this comment.
TBD if we keep recommending running notebooks on EC2 instances!
There was a problem hiding this comment.
Yes, same, let's make handle that in another PR.
| - Mistral models, such as [Mistral 7b (`mistralai/Mistral-7B-Instruct-v0.3`)](https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.2) | ||
| - Llama-2 models, such as [Llama-2 7b (`meta-llama/Llama-2-7b-hf`)](https://huggingface.co/meta-llama/Llama-2-7b-hf) | ||
|
|
||
| And many others! |
There was a problem hiding this comment.
WDYT of adding a section in https://huggingface.co/docs/optimum-neuron/en/package_reference/supported_models for the ones we support for training? and reference it here
There was a problem hiding this comment.
Why not but it is a bit "unclear". Because:
- We do not know which architecture is supported for regular training specifically.
- The supported architectures for distributed training is known, we can add that.
What does this PR do?