Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Config] Enhance ModelRecord #435

Merged
merged 3 commits into from
May 30, 2024

Conversation

CharlieFRuan
Copy link
Contributor

There are three changes to ModelRecord this PR brings:

1. Update model ids to match HF repo name

We rename modelId in webllm.prebuiltAppConfig to be the exact same as the HF repo name. For most models, that means we simply append -MLC to the modelId. For the low-context version of the model, we would have {HF-repo}-1k, suggesting 1k context length.

As a result, we rename Phi2 and phi1.5 models since their modelId did not match with the repo name

  • Phi2-q4f32_1phi-2-q4f32_1-MLC
  • Phi1.5-q4f16_1phi-1_5-q4f16_1-MLC

2. Rename model_url and model_lib_url to model and model_lib

To better match with other platforms of MLC-LLM (e.g. iOS, Android), we rename the ModelRecord fields.

3. Remove resolve/main from model URL

Instead of "https://huggingface.co/mlc-ai/Llama-3-8B-Instruct-q4f16_1-MLC/resolve/main/", we now make it "https://huggingface.co/mlc-ai/Llama-3-8B-Instruct-q4f16_1-MLC/"; note the trailing / will be appended by us if it is not there.

Example

As an example, we would have:

    {
      model: "https://huggingface.co/mlc-ai/Llama-3-8B-Instruct-q4f16_1-MLC",
      model_id: "Llama-3-8B-Instruct-q4f16_1-MLC",
      model_lib: "path/to/Llama-3-8B-Instruct-q4f16_1-ctx1k_cs1k-webgpu.wasm",
    },

instead of

    {
      model_url: "https://huggingface.co/mlc-ai/Llama-3-8B-Instruct-q4f16_1-MLC/resolve/main/",
      model_id: "Llama-3-8B-Instruct-q4f16_1",
      model_lib_url: "path/to/Llama-3-8B-Instruct-q4f16_1-ctx4k_cs1k-webgpu.wasm",
    },

@CharlieFRuan CharlieFRuan changed the title [ModelRecord] Enhance ModelRecord [Config] Enhance ModelRecord May 30, 2024
@CharlieFRuan CharlieFRuan merged commit 896b012 into mlc-ai:main May 30, 2024
1 check passed
CharlieFRuan added a commit that referenced this pull request May 30, 2024
### Changes
Main changes include:
- New prebuilt models:
  - Phi3-mini
  - StableLM-2-zephyr-1.6B
  - Qwen1.5-1.8B
  - Hermes2-Pro-Llama-3-8B to prebuilt models
- Updates on `ModelRecord` fields
  - For detail see: #435
- Update all WASMs
  - For detail see: #433
  - Update all WASMs to v0.2.39
- Support grammar for Llama3, hence update examples/json-mode to use
`Llama3` and `Hermes2-pro-Llama3-8B` for function calling in
`examples/json-schema`
- Use `loglevel` package:
  - For details see #427
- Fix `index.js.map` issue for Vite
  - #420
- Enhance error handling and ServiceWorker

### TVMjs
TVMjs compiled at
apache/tvm@71f7af7
- Main changes include: 
  - apache/tvm#17031
  - apache/tvm#17028
  - apache/tvm#17021

### WASM version
- All wasms updated to 0.2.39 via
mlc-ai/binary-mlc-llm-libs#123 for new MLC-LLM
runtime (mainly grammar)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant