Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add new Compatibility mode OR unify current Remote and Hosting mode #16

Open
NingW101 opened this issue Jun 14, 2024 · 0 comments
Open
Assignees
Labels
enhancement New feature or request

Comments

@NingW101
Copy link
Contributor

The logic of fetching models and model json files in transformers.js is as follows:

image

For the scenarios of env.allowLocalModels=true and env.allowRemoteModels=true (default), transformers.js will first try to get the model resources (including the model and model json file) from the local model path, and then try to get it from HuggingFace.

Current implementation Hosting mode:

env.allowLocalModels=true;
env.allowRemoteModels=false;

Remote mode:

env.allowLocalModels=false;
env.allowRemoteModels=true;

We cloud unify current 2 modes, just configure env.allowLocalModels=true, and then fetch models from local model path first, and then fetch it from HF.

Another major reason is because of the models jsons files(configs.json, tokenizer.json). Usually, the path to fetch these files follows the model files. If the configuration is to fetch the model files from HF, then these configuration files will also be obtained from HF. Because we have added these files into the repo, we hope to load the model json locally.

Pros

  • The model json files will be used
  • Solved a tricky case of loading a model from the local system in remote mode when HF is not accessible, the model json file could not be obtained, so even if the user had loaded the onnx model, the examples still did not work.
  • Code simplification
  • No need to maintain the USE_REMOTE_MODELS parameter in config.js (this file is also tracked by Git, so it may change for every time build production)

Cons

  • Errors that fail to request local model paths will be reported in Devtools -> Networks

Development work

For the examples integrated from transformer.js, there are no additional code modifications. For the stable transmission turbo -webgpu example, we should refactor the model fetching logic.

@NingW101 NingW101 added the enhancement New feature or request label Jun 14, 2024
@NingW101 NingW101 self-assigned this Jun 14, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant