Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add tests for utility functions (resolves #50) #51

Merged

Conversation

m-charlton
Copy link
Contributor

Contributor checklist


Description

This PR addresses #50 by adding unit tests for the utils module as well as a few minor refactors:

  • Added type annotations for most of the functions for checking by mypy
  • Tinkered with error messages
  • Added error checking in functions get_language_words_to_ignore() & get_language_words_to_remove(). This puts them in line with other functions in the module.
  • Used assertCountEqual(), where list comparisons are involved, as I'm assuming that list order is not important. Please correct me if I'm wrong.

All these changes are in the second commit.

The first commit is the removal of the add_num_commas() & num_add_commas() functions

Although code coverage is 100%. I'm looking for feedback on test coverage, especially for check_command_line_args() & check_and_return_command_line_args()

Related issue

Noticed that there is a lot duplication of the same language data distributed throughout utils.py. Is there any interest in moving this data out to say a JSON file? This file would be loaded once on module import and then the utils functions could interrogate this loaded object.

I'm willing to go into more detail and/or write a PR.

f-strings can format numbers to use a comma as a thousands separator. The
`add_num_commas` & `num_add_commas` functions are now redundant.
* Unit tests for `utils` module
* Edit some error messages
* Add type annotations for `mypy` checks
@github-actions
Copy link

github-actions bot commented Sep 21, 2023

Thank you for the pull request!

The Scribe team will do our best to address your contribution as soon as we can. The following is a checklist for maintainers to make sure this process goes as well as possible. Feel free to address the points below yourself in further commits if you realize that actions are needed :)

If you're not already a member of our public Matrix community, please consider joining! We'd suggest using Element as your Matrix client, and definitely join the General and Data rooms once you're in. It'd be great to have you!

Maintainer checklist

  • The commit messages for the remote branch should be checked to make sure the contributor's email is set up correctly so that they receive credit for their contribution

    • The contributor's name and icon in remote commits should be the same as what appears in the PR
    • If there's a mismatch, the contributor needs to make sure that the email they use for GitHub matches what they have for git config user.email in their local Scribe-Data repo
  • The CHANGELOG has been updated with a description of the changes for the upcoming release (if necessary)

@m-charlton
Copy link
Contributor Author

The comment on line 199 of tests/load/test_update_utils.py needs to be removed. Slipped through final review.

@andrewtavis andrewtavis self-requested a review September 21, 2023 14:51
@andrewtavis
Copy link
Member

Thanks for sending this along, @m-charlton! I’ll try to get to the review in the coming days :) :)

@andrewtavis
Copy link
Member

Hey @m-charlton 👋 Just FYI I’m a bit under the weather, so the review will take a bit longer. Apologies!

@m-charlton
Copy link
Contributor Author

No worries. Get well soon. In the meantime I'll start to have look at #48

@andrewtavis
Copy link
Member

Thanks so much, @m-charlton! :) Happy to answer any Wikidata related questions if needed 😊

Copy link
Member

@wkyoshida wkyoshida left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there any interest in moving this data out to say a JSON file? This file would be loaded once on module import and then the utils functions could interrogate this loaded object.

You know what? I do like the idea actually 🤔 I'd be fine with having an issue for this (we can hash out the details there) and then an accompanying PR 🙌🚀


def test_get_language_qid_negative():
with pytest.raises(ValueError) as excp:
_ = utils.get_language_qid("Newspeak")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice easter eggs 😆

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Double plus good!

Please find enclosed language_meta_data.json. The description & used_by fields are comments. The languages field is where the data are held proper. Not sure about the placement of the remove & ignore fields.

Writen a working implementation. Currently refactoring.

language_meta_data.json.zip

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Appreciate the easter egg as well! 🐰🥚😊 Till now we just have a random three emoji selection from "🥳🎉" as the gender annotation in the app when the user types "Scribe", and when you click the annotation it the selection is redone 🙃 It's fun 😇

def test_get_scribe_languages():
test_case = unittest.TestCase()

# test for content, not order
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor nit here, @m-charlton: for comments we try to do capitalized and fully punctuated if it's its own line, and for inline comments they're not capitalized and have no period as it's here :) Happy to take care of this though!

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Go for it.

test_case = unittest.TestCase()

# test for content, not order
test_case.assertCountEqual(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You'd asked about this in the commit, and to me it's 100% fine! Thank you for giving this some though!


def test_get_language_words_to_ignore_negative():
with pytest.raises(ValueError) as excp:
_ = utils.get_language_words_to_ignore("JAVA")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Likely never will be in this here organization 😅



def test_get_path_from_et_dir():
# TODO: file path is same as above. Is this correct?
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes it is. I think at one point they were at different levels, but they're in the same directory now. We could change this to get_path_from_scribe_data_child_dir and replace both?

@andrewtavis
Copy link
Member

This is all looking really great here, @m-charlton! 😊 Sounds like there's a bit more work to get the JSON up and running, but I'm happy to approve and merge once that's finalized 🚀

@m-charlton
Copy link
Contributor Author

@andrewtavis could you open an issue for the loadable language data file idea? I've a couple of questions concerning data file format & directory/package layout.

@wkyoshida
Copy link
Member

@m-charlton I created #52 for the data file idea 🚀
Do feel free to open issues in the future as well! That is completely fine 👍

P.S. I'm okay with continuing the discussion there so this PR can get merged. Some refactoring will inevitably take place, as @m-charlton has started on already - so that new work can be tracked there 😄

Copy link
Member

@andrewtavis andrewtavis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense to me to track it in the new issue! Excited to get the first one of these done! 🚀

@andrewtavis andrewtavis merged commit 6714dea into scribe-org:main Oct 15, 2023
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants