Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: Prevent test suite artifact creation in work directory #6438

Merged
merged 25 commits into from
Sep 18, 2024

Conversation

larseggert
Copy link
Collaborator

@larseggert larseggert commented Oct 5, 2023

Also remove a few other stale test assets while I'm here.

Fixes #4020
Fixes #3520

@codecov
Copy link

codecov bot commented Oct 5, 2023

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 88.83%. Comparing base (c7f6bde) to head (6261c7b).
Report is 77 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #6438      +/-   ##
==========================================
+ Coverage   88.78%   88.83%   +0.04%     
==========================================
  Files         296      304       +8     
  Lines       41320    41505     +185     
==========================================
+ Hits        36687    36871     +184     
- Misses       4633     4634       +1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

dev/tests/settings_local.py Outdated Show resolved Hide resolved
dev/tests/settings_local.py Show resolved Hide resolved
@larseggert
Copy link
Collaborator Author

@rjsparks

CRITICALS:
?: (datatracker.E0006) A directory used by the I-D submission tool does not
exist at the path given in the settings file. The setting is:
IDSUBMIT_REPOSITORY_PATH = /assets/ietfdata/doc/draft/repository

@larseggert
Copy link
Collaborator Author

There are a ton of directories that are being created in docker/scripts/app-create-dirs.sh - it's unclear to me how many of them are actually needed to run a dev instance and the CI.

@larseggert
Copy link
Collaborator Author

@rjsparks I think I made all the changes you requested?

@rjsparks
Copy link
Member

rjsparks commented Dec 8, 2023

Yes, I'm holding this for more work post the feat/rfc merge.

@rjsparks
Copy link
Member

rjsparks commented Sep 4, 2024

This didn't get returned to post feat/rfc. We should review it fresh to see if it's still the right thing to do.
cc: @jennifer-richards

ietf/settings_test.py Outdated Show resolved Hide resolved
ietf/settings_test.py Outdated Show resolved Hide resolved
@jennifer-richards
Copy link
Member

I left a couple comments - cleaning up after the tests, even in /tmp or wherever those paths end up, would be nice but also tricky. If the tests pass against modern main, I would be inclined to accept this as an improvement and treat those future work.

@jennifer-richards
Copy link
Member

Hopefully not stepping on @larseggert's toes, but just pushed changes that address my concerns. I think it's ready for @rjsparks to re-review?

@larseggert
Copy link
Collaborator Author

No toes in the area :-) I won't get to this any time soon, so please do what you need to!

Copy link
Member

@rjsparks rjsparks left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am a little nervous at the "clean up on exit" vs "clean up as you go" approach as it adds to the recovery for the developer if the process goes away via kill -9 or the like, but am willing to try it.

Just one question about why we're trading a file that I think we should be keeping for PIL.

Comment on lines +107 to +108
img = Image.new('RGB', (200, 200))
img.save(photodst)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

remind me why we're not just making a copy of the profile-default.jpg again?
(I'm sensitive to the PIL import - it's large)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's just avoiding a file that (as far as I can tell) exists only for the tests.

I don't have an opinion one way or the other, but maybe there's a lighter weight library for generating an empty image for testing.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's PyPNG - much smaller it seems, but requires less clearly intentioned code to generate a blank image and is somewhat slower on a per-image basis. The import of PIL on my system looks to be a few tens of ms - and I think we probably only hit that once per test run. (Though I haven't actually done a profile to see if the import is being exercised more than once; not sure how to do that)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suspect there's a valid, small, literal we can use as a PNG, but not going to slow this down over it.
I think we should, however, spend time scrubbing our full dependency set and seeing if we can make it smaller, and PIL is one of the things I'm wondering if we can scrub completely away.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We use PIL for resizing images in Person.photo. If we do away with that (mis?)feature, then we can probably ditch even needing to fake an image.

Checked and PyPNG is a < 40kb python file. It can read and write PNGs, so if we do need to create one for some reason, it's a pretty minimal bit of baggage.

media/photo/profile-default.jpg Show resolved Hide resolved
@jennifer-richards
Copy link
Member

jennifer-richards commented Sep 18, 2024

I am a little nervous at the "clean up on exit" vs "clean up as you go" approach as it adds to the recovery for the developer if the process goes away via kill -9 or the like, but am willing to try it.

I'm sympathetic. The kill -9 doesn't worry me terribly because it's rare and the impact is just a couple of directories in /tmp left over and atexit is robust against most normal exits.

The cleanup-at-exit does worry me though because I'd much rather create the temp dir separately for each test to maintain isolation. Before adding the atexit business I tried using the temp directory mechanism we have in our TestCase class and it kinda sorta worked, but ran into bad interactions with the test startup sequence. It definitely interacted with our checks for the directories existing, but IIRC the statics serving for tests was also broken by this because it's only set up once for the entire test run.

@rjsparks rjsparks merged commit 3507466 into ietf-tools:main Sep 18, 2024
9 checks passed
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Sep 22, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Make test suite generate files outside of the source tree temporary directories not cleaned up after tests
3 participants