Official implementation of "DoRA: Weight-Decomposed Low-Rank Adaptation"
-
Updated
Apr 28, 2024
Official implementation of "DoRA: Weight-Decomposed Low-Rank Adaptation"
[SIGIR'24] The official implementation code of MOELoRA.
Code for NOLA, an implementation of "nola: Compressing LoRA using Linear Combination of Random Basis"
An Efficient LLM Fine-Tuning Factory Optimized for MoE PEFT
[ICML'24 Oral] APT: Adaptive Pruning and Tuning Pretrained Language Models for Efficient Training and Inference
memory-efficient fine-tuning; support 24G GPU memory fine-tuning 7B
Official code implemtation of paper AntGPT: Can Large Language Models Help Long-term Action Anticipation from Videos?
CRE-LLM: A Domain-Specific Chinese Relation Extraction Framework with Fine-tuned Large Language Model
High Quality Image Generation Model - Powered with NVIDIA A100
Mistral and Mixtral (MoE) from scratch
Fine-tune StarCoder2-3b for SQL tasks on limited resources with LORA. LORA reduces model size for faster training on smaller datasets. StarCoder2 is a family of code generation models (3B, 7B, and 15B), trained on 600+ programming languages from The Stack v2 and some natural language text such as Wikipedia, Arxiv, and GitHub issues.
PEFT is a wonderful tool that enables training a very large model in a low resource environment. Quantization and PEFT will enable widespread adoption of LLM.
LLM projects
Finetuning Large Language Models
In this repo I will share different topics on anything I want to know in nlp and llms
Test results of Kanarya and Trendyol models with and without fine-tuning techniques on the Turkish tweet hate speech detection dataset.
This repository was commited under the action of executing important tasks on which modern Generative AI concepts are laid on. In particular, we focussed on three coding actions of Large Language Models. Extra and necessary details are given in the README.md file.
Fine Tuning pegasus and flan-t5 pre-trained language model on dialogsum datasets for conversation summarization to to optimize context window in RAG-LLMs
NTU Deep Learning for Computer Vision 2023 course
Add a description, image, and links to the peft-fine-tuning-llm topic page so that developers can more easily learn about it.
To associate your repository with the peft-fine-tuning-llm topic, visit your repo's landing page and select "manage topics."