[BUG] Consider each site_package

[BennettRand]

Why are the changes needed?

site.getsitepackages() was returning

[
    '/home/username/project/python/.venv/lib/python3.10/site-packages',
    '/home/username/project/python/.venv/local/lib/python3.10/dist-packages',
    '/home/username/project/python/.venv/lib/python3/dist-packages',
    '/home/username/project/python/.venv/lib/python3.10/dist-packages'
]

which when run through os.path.commonpath is

'/home/username/project/python/.venv'

This makes it so that any mod_file compared to the site packages set, will never be a string in the site packages set, thus being included in the list of filed to be packaged for fast serialization.

What changes were proposed in this pull request?

Compare the common path of the module file to each site package in the site package set to see if the mosule file should be skipped.

How was this patch tested?

No tests added, all tests passed in Python 3.12.7

Manually inspected the file tree generated by a pyflyte -v run --remote ... command and the contents of the resulting .tar.gz file.

Setup process

Screenshots

Check all the applicable boxes

• I updated the documentation accordingly.
• All new and existing tests passed.
• All commits are signed-off.

Related PRs

Docs link

[welcome]

Thank you for opening this pull request! 🙌

These tips will help get your PR across the finish line:

• Most of the repos have a PR template; if not, fill it out to the best of your knowledge.
• Sign off your commits (Reference: DCO Guide).

[wild-endeavor]

any chance you can think of a simple unit test for this?

[thomasjpfan]

@BennettRand I think the simplest unit test is to monkeypatch site.getsitepackages and check that list_imported_modules_as_files does the right thing.

[BennettRand]

@thomasjpfan @wild-endeavor Just added it.
Should cover the general case and the corner case I ran into.

[thomasjpfan]

Thank you for the test! LGTM

[thomasjpfan]

For windows CI to pass, I think we'll either need to use os.path.join or pathlib.Path to define the paths.

(Windows uses \ instead of /)

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 51.42%. Comparing base (51f9a3e) to head (20538ea).
Report is 7 commits behind head on master.

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #2920      +/-   ##
==========================================
+ Coverage   46.91%   51.42%   +4.51%     
==========================================
  Files         199      199              
  Lines       20840    20840              
  Branches     2681     2681              
==========================================
+ Hits         9778    10718     +940     
+ Misses      10582     9517    -1065     
- Partials      480      605     +125     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[thomasjpfan]

@BennettRand The failing windows test looks related to this PR.