Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

sentence embeddings with Sbert. #1405

Merged
merged 7 commits into from
Jul 14, 2023
Merged

Conversation

abuelnasr0
Copy link
Contributor

this is an example of how to use RoBERTa from keras-nlp in another model and fine-tune it using siamese networks for semantic similarity tasks.
I fine tuned the model with two methods (regression objective and triplet objective).
the paper that this example is based on is sbert
here is a colab gist with the code.

Copy link
Contributor

@suvadityamuk suvadityamuk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks like an awesome example! Just filled in a few preliminary nitpicks after a quick first-pass on the content for this tutorial. I guess you may have to do a bit of a grammar check in-place to get things right, but the concepts look good to me!

Some high-level comments though, which are kinda repeated across the tutorial

  • Make sure you remove all the <br> tags, they don't need to be added explicitly
  • Make sure your links are all on the same lines, don't break them
  • Some places require capitalization. I think putting the text through a grammar checker should remove those errors fairly quickly

Thanks for this, I'm sure folks would love it! Delegating the rest to the able hands of @mattdangerw of course!

examples/nlp/sentence_embeddings_with_sbert.py Outdated Show resolved Hide resolved
examples/nlp/sentence_embeddings_with_sbert.py Outdated Show resolved Hide resolved
examples/nlp/sentence_embeddings_with_sbert.py Outdated Show resolved Hide resolved
examples/nlp/sentence_embeddings_with_sbert.py Outdated Show resolved Hide resolved
examples/nlp/sentence_embeddings_with_sbert.py Outdated Show resolved Hide resolved
examples/nlp/sentence_embeddings_with_sbert.py Outdated Show resolved Hide resolved
examples/nlp/sentence_embeddings_with_sbert.py Outdated Show resolved Hide resolved
examples/nlp/sentence_embeddings_with_sbert.py Outdated Show resolved Hide resolved
examples/nlp/sentence_embeddings_with_sbert.py Outdated Show resolved Hide resolved
examples/nlp/sentence_embeddings_with_sbert.py Outdated Show resolved Hide resolved
@abuelnasr0
Copy link
Contributor Author

@suvadityamuk Thanks!

Copy link
Member

@mattdangerw mattdangerw left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is really great! Thanks so much for the guide.

I left some comments, but they are mainly about readability. Overall the content is quite well explained.

Thanks!

examples/nlp/sentence_embeddings_with_sbert.py Outdated Show resolved Hide resolved
examples/nlp/sentence_embeddings_with_sbert.py Outdated Show resolved Hide resolved
examples/nlp/sentence_embeddings_with_sbert.py Outdated Show resolved Hide resolved
examples/nlp/sentence_embeddings_with_sbert.py Outdated Show resolved Hide resolved
examples/nlp/sentence_embeddings_with_sbert.py Outdated Show resolved Hide resolved
examples/nlp/sentence_embeddings_with_sbert.py Outdated Show resolved Hide resolved
examples/nlp/sentence_embeddings_with_sbert.py Outdated Show resolved Hide resolved
examples/nlp/sentence_embeddings_with_sbert.py Outdated Show resolved Hide resolved
Copy link
Member

@mattdangerw mattdangerw left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very nice! LGTM.

examples/nlp/sentence_embeddings_with_sbert.py Outdated Show resolved Hide resolved
Copy link
Member

@fchollet fchollet left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add the generated files!

@abuelnasr0
Copy link
Contributor Author

@fchollet I have just pushed the generated files.
@mattdangerw Thanks for the review!

@fchollet fchollet merged commit a2cf88d into keras-team:master Jul 14, 2023
1 of 3 checks passed
@fchollet
Copy link
Member

Thank you for the great contribution!

@abuelnasr0
Copy link
Contributor Author

@fchollet You are welcome

@abuelnasr0 abuelnasr0 deleted the SBERT-example branch August 8, 2023 20:05
@abuelnasr0 abuelnasr0 restored the SBERT-example branch August 8, 2023 20:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants