Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Internal download API: Add proper validated directory input #4981

Conversation

mcmonkey4eva
Copy link
Collaborator

@mcmonkey4eva mcmonkey4eva commented Sep 19, 2024

This is the core side internal update for a frontend PR: Comfy-Org/ComfyUI_frontend#890

Frontend PR depends also on #4980 which I separated for cleanliness (that one is a decent miniapi and can be pulled no problem, whereas this is a much larger change that may need some figuring out first)

Key point is where the old code had a loose vague directory input that could be anything but had to be relative to the base, the new code accepts a pair of values that must match what folder_paths already manages.
That is: a directory (eg checkpoints), and a path for an instance of that directory (eg C:/path/to/ComfyUI/models/checkpoints)

The matching frontend PR for this internal API will display the list of locations if there are multiple, or just select the main one if there's only one. (There's usually multiple though even on clean installs).

Because it's strictly whitelist matched against data the server already knows, this simplifies validation.

I also cleared out some redundant variables that weren't needed before and don't even make sense now (relative_path was never used)

@mcmonkey4eva
Copy link
Collaborator Author

tests are failing because patch('folder_paths.get_folder_paths', return_value=[temp_dir]) isn't working and I'm not sure why

@robinjhuang
Copy link
Collaborator

@mcmonkey4eva

tests-unit/prompt_server_test/download_models_test.py:137: AssertionError
____________________________ test_create_model_path ____________________________

tmp_path = PosixPath('/tmp/pytest-of-runner/pytest-0/test_create_model_path0')
monkeypatch = <_pytest.monkeypatch.MonkeyPatch object at 0x7f2378114bb0>

    def test_create_model_path(tmp_path, monkeypatch):
        mock_models_dir = tmp_path / "models"
        monkeypatch.setattr('folder_paths.models_dir', str(mock_models_dir))
    
        model_name = "test_model.sft"
        model_directory = "test_dir"
    
>       file_path = create_model_path(model_name, model_directory, mock_models_dir)
E       TypeError: create_model_path() takes 2 positional arguments but 3 were given

This test fails because you deleted one argument in the function definition.

@mcmonkey4eva
Copy link
Collaborator Author

funny timing you comment right when I finally get past the thing blocking me - overwriting folder_paths.get_folder_paths with patch was not working at all with no clear reason, but patching the folder_paths.folder_names_and_paths global var seems to work, so now I can move on to fixing the rest of the test issues lol

@robinjhuang
Copy link
Collaborator

There will be a set of tests that fail on Windows FYI if you are testing locally.

Trying to fix those here
#5018

@mcmonkey4eva
Copy link
Collaborator Author

mcmonkey4eva commented Sep 22, 2024

Tests for this PR are now fully passing (not counting the stuff that fails on windows, thank you robin for fixing those!)

use temp dir to avoid accidental fs pollution from tests
@mcmonkey4eva mcmonkey4eva added Feature A new feature to add to ComfyUI. Good PR This PR looks good to go, it needs comfy's final review. labels Sep 24, 2024
@comfyanonymous comfyanonymous merged commit 08c8968 into comfyanonymous:master Sep 24, 2024
9 checks passed
huchenlei added a commit that referenced this pull request Sep 25, 2024
@huchenlei
Copy link
Collaborator

Fails frontend test

Retry #2 ───────────────────────────────────────────────────────────────────────────────────────

    Error: Timed out 5000ms waiting for expect(locator).toBeVisible()

    Locator: locator('.download-complete')
    Expected: visible
    Received: <element(s) not found>
    Call log:
      - expect.toBeVisible with timeout 5000ms
      - waiting for locator('.download-complete')


      102 |     // Wait for the element with the .download-complete selector to be visible
      103 |     const downloadComplete = comfyPage.page.locator('.download-complete')
    > 104 |     await expect(downloadComplete).toBeVisible()
          |                                    ^
      105 |   })
      106 | })
      107 |

        at /home/runner/work/ComfyUI_frontend/ComfyUI_frontend/ComfyUI_frontend/browser_tests/dialog.spec.ts:104:36

  Slow test file: [chromium] › nodeSearchBox.spec.ts (41.3s)
  Slow test file: [chromium] › groupNode.spec.ts (21.7s)
  Slow test file: [chromium] › colorPalette.spec.ts (19.5s)
  Consider splitting slow test files to speed up parallel execution
  1 failed
    [chromium] › dialog.spec.ts:[84](https://github.com/Comfy-Org/ComfyUI_frontend/actions/runs/11025021246/job/30619191324#step:4:85):3 › Missing models warning › Should display a warning when missing models are found 
  6 skipped
  114 passed (5.8m)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Feature A new feature to add to ComfyUI. Good PR This PR looks good to go, it needs comfy's final review.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants