Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Text experiments #190

Open
wants to merge 5 commits into
base: main
Choose a base branch
from
Open

Text experiments #190

wants to merge 5 commits into from

Conversation

WillemSpek
Copy link
Collaborator

It's been a while since my last PR, but I haven't been idle. Just took me a while to get the text metrics to work. I have attempted to organize the grid search using XAI metrics. In general this PR contains the following contributions:

  1. Script to test against multiple hyperparameter configurations for textual data.
  2. Notebooks used to sample the original datasets according to model scores.
  3. The sampled datasets themselves.
  4. Improvement over the single deletion metric to test against a set of salient maps.
  5. Addition of sensitivity metric for textual data.
  6. A synonym replacer that attempts to perturb sentences by replacing words with their synonyms.
  7. Notebooks used to analyze the collected data and produce visualizations for them.

From a software engineering perspective, there is definitely still quite a bit to be improved here as the code is now a bit bloaty and ill-organized. Nevertheless, I will try to get this work as organized as possible whilst still running the code needed to obtain my results.

@review-notebook-app
Copy link

Check out this pull request on  ReviewNB

See visual diffs & provide feedback on Jupyter Notebooks.


Powered by ReviewNB

@WillemSpek WillemSpek self-assigned this Aug 9, 2023
Copy link
Contributor

@elboyran elboyran left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've left few comments here and there.
Looks good to me in general (a lot of work), I admit I did not run the code.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants