VST windows don't behave properly on initial open with non-100% DPI

[Tom-Evers]

Operating System Info

Windows 10

Other OS

No response

OBS Studio Version

29.0.0

OBS Studio Version (Other)

No response

OBS Studio Log URL

https://obsproject.com/logs/NwSSvb3cBRuiiFfP

OBS Studio Crash Log URL

No response

Expected Behavior

This issue is very similar to obsproject/obs-vst#107

I'm developing a DPI-aware plugin, and am hindered by an issue that seems to have the same root cause.

• When opening the plugin, getRect(ERect** rect) gets called.
• I've got some dimensions to resize my contents to (let's say: my plugin needs 2000x2000 pixels at 100% scaling).
• My main monitor is at 175%, so I know I actually have to resize my contents to 3500x3500.
• I fill the ERect structure with 3500x3500.

Expected behavior is that the host window gets resized to 3500x3500

Current Behavior

The host window gets resized to 6125x6125 (175% of the size I returned in the ERect structure)

This is aberrant behavior: other VST hosts treat the ERect that I return as the true dimensions of my window.
Only OBS thinks it still needs to compensate for DPI.

Steps to Reproduce

1. Add a VST audio plugin that only reports its own size and doesn't resize its contents in response to window resizes (e.g. FabFilter)
2. Open plugin interface

Anything else we should know?

No response

[RytoEX]

The Steps To Reproduce are not actually Steps To Reproduce. Please provide actual Steps To Reproduce. It would be especially helpful if you could single out a VST that we can test against, preferably one that is free. Assume that not all OBS developers/contributors frequently use or are familiar with VSTs.

[Tom-Evers]

I've changed the steps to reproduce, taking the steps from obsproject/obs-vst#107

Let's say we're working on a big screen, and we've got our scaling set to 150%

The big problem is:

1. OBS asks the plugin for a size
2. The plugin reports a size
3. OBS scales that size to the current DPI
4. The plugin thinks that the size it reported is the size of its window
   (but in reality, OBS gave it a 150% sized window)

Many VST plugins might not show that anything is amiss: they might check the window size explicitly, and resize their contents to the window they actually got.

It all becomes a usability issue when the plugin resizes its contents to the window and saves those new dimensions.
When it gets closed and reopened, it reports its last known size (150%) to OBS, and OBS then gives it a window scaled to the current dpi (150% * 150% = 225%).
Close and reopen the plugin, it asks for a 225% window, OBS gives it an even larger window (225% * 150% = 337.5%), etc.

[crocodileGena]

@RytoEX @Tom-Evers - These GUI corruption are indeed troubling. I would add that changing the .exe setting as seen in the picture solves all GUI corruptions for VST plugins.

So that's a workaround that the users can perform and have everything working for them.
But OBS should have these sorted out-of-the-box and not by user workaround.
See https://learn.microsoft.com/en-us/windows/win32/hidpi/setting-the-default-dpi-awareness-for-a-process for reference

[RytoEX]

I would add that changing the .exe setting as seen in the picture solves all GUI corruptions for VST plugins.

I'd be cautious about recommending such workarounds. They are easily forgotten, and then cause issues down the road for people who enabled them and then forgot about them.

See https://learn.microsoft.com/en-us/windows/win32/hidpi/setting-the-default-dpi-awareness-for-a-process for reference

If I recall correctly, Qt largely handles DPI awareness for us (we do have to account for it in some places manually, such as cursor coordinates). However, it is a constant work in progress, and they are still making improvements in this area as new versions are released. It is possible that we've missed something somewhere when creating the VST window. As always, PRs to fix the issue are welcome.

Also, providing a link to a specific (preferably free) VST plugin that exhibits the issue would help us see the problem in action, which is often a great first step to fixing it. FabFilter Pro-C 2 (pictured in the original obs-vst Issue) has a free trial, though it would be ideal to be able to see this in a plugin without having to worry about a time limit.

[crocodileGena]

Thanks @RytoEX .
I mentioned the workaround simply to provide more information for the issue.
The issue, at least for the VST plugins I use, happen with all of them. I guess you could take Voxengo SPAN as a reference (it's free). Even though you can use the unlicensed version as well of any plugin, since we are only concerned with the window behavior, and not specific plugin functionality.
I will try to come up with PR to solve it.
Can you let me know if there are additional windowing issues in the application other than the VST?
I need to know the scope of the problem and solution, only VST plugins windows (hence VST window creation) or all over the place (hence Qt or Qt usage)?

[RytoEX]

Thanks @RytoEX . I mentioned the workaround simply to provide more information for the issue. The issue, at least for the VST plugins I use, happen with all of them. I guess you could take Voxengo SPAN as a reference (it's free). Even though you can use the unlicensed version as well of any plugin, since we are only concerned with the window behavior, and not specific plugin functionality. I will try to come up with PR to solve it. Can you let me know if there are additional windowing issues in the application other than the VST? I need to know the scope of the problem and solution, only VST plugins windows (hence VST window creation) or all over the place (hence Qt or Qt usage)?

I'm not aware of any other places in OBS Studio that have this specific problem with child window sizing. That's not to say that there are not other places where OBS Studio has issues with DPI scaling (e.g., multiple monitors, mixed scaling on multiple monitors). However, in my experience, most of the DPI scaling issues that have been fixed over the years are because they were fixed in Qt. There are some where we might not be handling coordinates or scaling correctly, but I recommend addressing each item individually rather than ballooning the scope.

Personally, I do not use VSTs, so I am not well-suited to answer any questions particular to VST usage. I can, however, investigate issues within OBS as time allows. If I can find time to look into this, or any proposed PRs, I will do so.

[xlinshan]

I've checked the code of obs-vst, and tried to modify it and compiled it. Turns out that Qt introduced the problem probably on Windows only, if not on Linux or MacOS. A workaround for this problem is to modify the window sizing behavior of vst plugin to use a width and height divided by the user's UI scale factor.

The related code's path is ./obs-studio/plugins/obs-vst/win/EditorWidget-win.cpp

Both the EditorWidget::buildEffectContainer() and EditorWidget::handleResizeRequest() use resize(rec->right - rec->left, rec->bottom - rec->top) to set the vst plugin's window size. The QWidget class has a qreal devicePixelRatioF() function inherited from QPaintDevice. Changing the resizing code to something like qreal scale_factor = devicePixelRatioF(); resize(rec->right - rec->left) / scale_factor, (rec->bottom - rec->top) / scale_factor); makes the window size of vst plugins correct.

This workaround can be used before Qt fixes their part.

[RytoEX]

I've checked the code of obs-vst, and tried to modify it and compiled it. Turns out that Qt introduced the problem probably on Windows only, if not on Linux or MacOS. A workaround for this problem is to modify the window sizing behavior of vst plugin to use a width and height divided by the user's UI scale factor.

The related code's path is ./obs-studio/plugins/obs-vst/win/EditorWidget-win.cpp

Both the EditorWidget::buildEffectContainer() and EditorWidget::handleResizeRequest() use resize(rec->right - rec->left, rec->bottom - rec->top) to set the vst plugin's window size. The QWidget class has a qreal devicePixelRatioF() function inherited from QPaintDevice. Changing the resizing code to something like qreal scale_factor = devicePixelRatioF(); resize(rec->right - rec->left) / scale_factor, (rec->bottom - rec->top) / scale_factor); makes the window size of vst plugins correct.

This workaround can be used before Qt fixes their part.

That doesn't sound like a workaround. That's how you're supposed to handle display scaling, which we do in other parts of the obs-studio codebase. See:

• obsproject/obs-browser@7143a01
• obsproject/obs-browser@5ede244
• 5cdd084
• 4611847
• b496f43
• 1fa45e6

The obs-vst codebase is just not as well maintained as obs-studio or obs-browser, so we may have just missed it, unless @WizardCM already tried this in his previous attempts on obsproject/obs-vst#107.

I don't know if fractional scaling just isn't as prevalent on Linux, but it is pretty common on Windows laptops as far as I know.

[xlinshan]

That doesn't sound like a workaround. That's how you're supposed to handle display scaling, which we do in other parts of the obs-studio codebase. See:

• obsproject/obs-browser@7143a01
• obsproject/obs-browser@5ede244
• 5cdd084
• 4611847
• b496f43
• 1fa45e6

The obs-vst codebase is just not as well maintained as obs-studio or obs-browser, so we may have just missed it, unless @WizardCM already tried this in his previous attempts on obsproject/obs-vst#107.

I don't know if fractional scaling just isn't as prevalent on Linux, but it is pretty common on Windows laptops as far as I know.

The codes linked are multiplying sizes by the scale factor instead of dividing sizes by the scale factor, so I think the modification I made is not quite a "supposed to be" behavior. I've tried some vst plugins on MacOS, where the sizes are not divided by the factor, but the window size is rendered correctly. I think the vst plugins are loaded OS-specifically, and the sizes of window are requested from the loaded vst plugins. I guess that, on Windows, the loaded vst plugins report their sizes in a way that Qt cannot accept directly, while on MacOS the reported sizes can be accept directly, and it is an external behavior that can only be complied with. It could be wrong, but I think it's quite possible.

Also, I found that the modification has some problem with multiple screens of different ui scales. When a vst plugin window is first created, the ui scale of main monitor is used, and if the obs window is on the sub-monitor with a different scale, the created vst plugin window will be too small or too big depending on the scale difference. However, resizing the window, if applicable, will refresh it to the correct size, and for HiDPI aware vst plugins, moving the window between the screens fixes it too.

I'll try to submit a pull request for the modification since it somehow makes the thing work. I have no experience on the pull request stuff of git and github, so it may not be done properly, just in case.