-
Notifications
You must be signed in to change notification settings - Fork 51
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Test cases for languages #118
Comments
@inariksit 's GFTest would be good to use here, but that will involve more work to set up |
@heatherleaf I think gftest can be used as an inspiration to create test sentences, which are then stored in the corpus. gftest doesn't have any notion of translation equivalents, it just generates test cases for a single language (sure it can then linearise them in multiple languages), but you can't tell gftest "expect X to be translated as Y and alert me if not". For such a case, basic unit tests are much better idea. What would be nice, if gftest wasn't so slow for certain languages, would be to use the feature that tests a single grammar against its older version, and outputs all differences. But given that this feature can take hours to run for any reasonably complex language (and still minutes for, say, Dutch), it's not a good idea to put it in Travis. Regarding your issue #117 , I am already using gftest for sanity checking my big updates in Arabic. So for your point "2. There must be test cases: at least a simple parallel corpus or treebank with 10-20 sentences", I can easily include a set of sentences that were wrong before my update and are now correct. I haven't bothered to document my pull requests before, because in practice, I merge them immediately, and afaik there is nobody in the GF core team who knows Arabic to any relevant extent. But if we want to set a good example for future RGL contributors, I can start doing that for my future pull requests. |
Sounds like a good start. The format should probably use a few more line breaks though, for developer sanity:
To add a bit of structure:
|
I like your suggestion. Should the examples also specify the top-level category or just assume the default one? (Which is Perhaps the first line in each example should also specify its concrete grammar? Like this:
The advantage is then that we can test different modules (e.g., |
Regarding Haskell vs Shell: I would prefer non-dependence on Haskell, if it's not too complicated. It would be nice if RGL authors do not have to install Haskell. |
Can we define how to write test cases for RGL languages?
Here's a simple suggestion: every language (starting with new ones) should have a small corpus with positive examples. The corpus can consist of lines of the following kind:
jag sover i huset -- LangEng: I sleep in the house
huset är stort -- Lang: PhrUtt NoPConj (UttS (UseCl (TTAnt TPres ASimul) PPos (PredVP (DetCN (DetQuant DefArt NumSg) (UseN house_N)) (UseComp (CompAP (PositA big_A)))))) NoVoc
ibland sover jag inte
Testing should be fairly easy, and can be done automatically (perhaps part of Travis CI?):
The text was updated successfully, but these errors were encountered: