Fix pipelines #608
Fix pipelines #608
Conversation
The docker package must be updated to be compatible with the latest requests release. In the meantime we pin requests version.
dacorvo
force-pushed
the
fix_ci_pipelines_tgi
branch
2 times, most recently
from
cfa1c39
to
fb1d6fc
Compare
|
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update. |
dacorvo
force-pushed
the
fix_ci_pipelines_tgi
branch
from
b88ce62
to
68a6e24
Compare
dacorvo
force-pushed
the
fix_ci_pipelines_tgi
branch
2 times, most recently
from
c942f7c
to
53f7ed4
Compare
This base class will implement transformers PreTrainedModel methods that are not implemented in optimum PreTrainedModel base class.
dacorvo
force-pushed
the
fix_ci_pipelines_tgi
branch
from
53f7ed4
to
be54234
Compare
| neuron_config: Optional["NeuronDefaultConfig"] = None, | ||
| **kwargs, | ||
| ): | ||
| def __init__(self, model: "PreTrainedModel", config: "PretrainedConfig"): |
There was a problem hiding this comment.
For traced models, we are passing torch.jit._script.ScriptModule as model here.
There was a problem hiding this comment.
I realized that when the CI failed ... 🫣
| huggingface_token = use_auth_token | ||
| elif use_auth_token: | ||
| huggingface_token = HfFolder.get_token() | ||
| if hasattr(model, "device"): |
There was a problem hiding this comment.
Is it necessary, for traced model setting all dummy device as CPU would be fine I think.
There was a problem hiding this comment.
Yes, but I was wondering if at some point we wouldn't end up with models on XLA device.
| Whether the Neuron model has separated weights and neff graph (by setting `inline_weights_to_neff=False` during the compilation). | ||
| """ | ||
| return not self.config.neuron.get("inline_weights_to_neff", True) | ||
| def to(self, device: Union[str, torch.device]): |
There was a problem hiding this comment.
Do we need to add content if this is actually a no-op function.
There was a problem hiding this comment.
As long as it exist, someone might want to call it, so I added a check.
| @@ -0,0 +1,611 @@ | |||
| # coding=utf-8 | |||
There was a problem hiding this comment.
Maybe just keep it in modeling_base.py?
What does this PR do?
This fixes a regression in Neuron pipelines with
transformers == 4.40.2.The issue is actually fixed in transformers as a side-effect of this pull-request:
huggingface/transformers#30534
But the issue in
optimum-neuronactually comes from the fact that all neuron models are based onoptimum.OptimizedModelthat reimplementstransformers.PretrainedModelwithout providing all its methods (includingto()).This pull-request therefore introduces a new top-level
NeuronModelclass that inherits fromoptimum.OptimizedModeland provides the missing methods.All neuron model classes should now inherit from
NeuronModel.In the process,
NeuronBaseModelis renamed toNeuronTracedModelto remove any ambiguity.