Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for explicit batch for all models #1522

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

ligaz
Copy link
Contributor

@ligaz ligaz commented Nov 16, 2022

Currently only the ONNX models support the explicit batch strategy. I have my own TRT model that has the NCHW input size and I need to be able to use it.

The fix is to remove the hardcoding of the ONNX models and check the input dimensions size. If the number of dimensions are 4 (NCHW) then we are in explicit batch mode.

Additionally the initialization of mModelType and mModelPath fields for Engine file types is done before loading the engine. Those fields are potentially used in the loading itself (they were used when we were checking the ONNX model type before the fix).

@ligaz
Copy link
Contributor Author

ligaz commented Nov 18, 2022

@dusty-nv Can you do a review of these changes? Is there a better way to handle this scenario?

@dusty-nv
Copy link
Owner

Hi @ligaz, thank you for the PR - yes, I believe this looks good but have not had a chance to test it yet. I have added it to my TODO queue however to be merged/tested into the dev branch, and from there into master. Thanks again!

@ligaz
Copy link
Contributor Author

ligaz commented Nov 21, 2022

@dusty-nv Do you want me to base this change on top of the dev branch?

@dusty-nv
Copy link
Owner

@ligaz it's not that big of a code change so I would probably just apply it to dev branch manually, but if you wanted to that is great too. What I need to do is test it against the non-ONNX models to make sure it doesn't inadvertently trigger any incorrect behavior with those

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants