Skip to content

OpenLLaMA-Chinese, a permissively licensed open source instruction-following models based on OpenLLaMA

License

Notifications You must be signed in to change notification settings

FittenTech/OpenLLaMA-Chinese

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

14 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

OpenLLaMA-Chinese

OpenLLaMA-Chinese is a 100% free Chinese large language model, and can be utilized for both non-commercial and commercial purposes.

OpenLLaMA-Chinese is built on OpenLLaMA, which is a permissively licensed open-source reproduction of Meta AI's LLaMA 7B and 13B models, trained on the RedPajama dataset. OpenLLaMA also includes a smaller 3B variant of the LLaMA model. We have conducted fine-tuning on Chinese and English instructions using the OpenLLaMA base models and have made our weights publicly available.

News

[2023/06/29] We released the openllama 13b model by using evol intructions.

[2023/06/24] We use evol intructions from WizardLM to finetune the openllama 7B, the 13B Model will be avaliable in next week!.

Evol Instruction Examples

Evol Instructions Fine-tuning Weights:

Chinese Instructions Fine-tuning Weights:

English Instructions Fine-tuning Weights:

Chinese+English Instructions Fine-tuning Weights:

Data

For Chinese fine-tuning, we utilized the alpaca_data_zh_51k.json from the Chinese-LLaMA-Alpaca project.

For English fine-tuning, we employed the alpaca_data.json from the StanfordAlpaca project.

For fine-tuning with both English and Chinese instructions, we used data from both sources.

Usage

We modified the generate code from LLaMA-X.

To use the PyTorch inference code, follow these steps:

  1. Download the weights and update the base_model path in inference/gen_torch.py.
  2. Run the following command:
python inference/gen_torch.py

Pretraining and Finetuning

FittenTech offers LLMs pretraining and fine-tuning services. For more details, please visit https://llm.fittentech.com/.

Acknowledgments

We would like to express our gratitude to the developers of the following open-source projects, as our project builds upon their work:

License

We adopt the Apache License, following OpenLLaMA's license.

About

OpenLLaMA-Chinese, a permissively licensed open source instruction-following models based on OpenLLaMA

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages