-
Notifications
You must be signed in to change notification settings - Fork 17
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
character level model support #121
Comments
Clinical genomics stuff on pause for now but when that starts again can try to fold in the CANINE support with all the recent updates |
Hoping to do some work on this issue this week since Timelines is on pause for ITCR. Will need to rebase the branch in my fork off of the error analysis branch and then adapt the error analysis code to work with character level. |
Still figuring out the most efficient way to do this since a couple of the approaches I tried also have some issues. Have also been thinking about how to do adapt character level stuff for relation extraction. Tagging has also been more expensive than its wordpiece counterpart, which would mean even more of a blowup for relex. Maybe there's some way to leverage sparse matrices for both |
Have something that's basically working modulo an issue with the batches being the wrong shape if one uses say layer 8 out of 12 on canine and some formatting tweaks for the error analysis The former issue might have something to do with the relatively unique downsampling -> upsampling part of the canine architecture. Actually only found out about this by accident since the layer 8 setting was something I was trying with a Roberta based model. Will look into both of these more though. |
Support for character based models, in particular Flair and canine
The text was updated successfully, but these errors were encountered: