Skip to content

Commit

Permalink
test
Browse files Browse the repository at this point in the history
  • Loading branch information
vaibhavad committed Apr 9, 2024
1 parent afd9a67 commit 0b31603
Showing 1 changed file with 6 additions and 0 deletions.
6 changes: 6 additions & 0 deletions docs/_pages/tutorial.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,12 @@ title: "LLM2Vec Tutorial: Steps for transforming any decoder-only model into a t
permalink: /tutorial/
---

<script type="text/javascript"

src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.3/MathJax.js?config=TeX-AMS-MML_HTMLorMML">

</script>

LLM2Vec consists of 3 simple steps to transform decoder-only LLMs into text encoders: 1) enabling bidirectional attention, 2) training with masked next token prediction, and 3) unsupervised contrastive learning. After the LLM2Vec transformation, the model can be further fine-tuned with supervised data. Here, we provide a tutorial on how to use the LlaMA models.

This tutorial will focus on the first two steps. After completing these steps, the model can be trained for unsupervised or supervised contrastive learning like any other encoder model.
Expand Down

0 comments on commit 0b31603

Please sign in to comment.