Wayland and PipeWire support

[ehfd]

Spun out from #34.

This is a placeholder issue for all discussions associated with supporting Wayland capture.
NVIDIA has introduced GBM support to its 510.xx drivers on Linux, and GStreamer's Pipewire plugin to capture Wayland would be mature enough by now. This also means that pulseaudio and Xorg no longer has to be matched with a timer.

Implementation of capturing Wayland using GStreamer's Pipewire plugin is accessible with https://github.com/H-M-H/Weylus or https://github.com/pavlobu/deskreen and some more Wayland capturing references are available in https://github.com/SunshineStream/Sunshine.
How to deploy a headless Sway DE on Wayland: https://github.com/bbusse/swayvnc-firefox.

[ehfd]

Relevant code: https://github.com/selkies-project/selkies-gstreamer/blob/master/src/selkies_gstreamer/gstwebrtc_app.py
Anyone with GStreamer experience, or willing to learn can solve this.
The PipeWire plugin of GStreamer should be able to do it.

https://github.com/selkies-project/selkies-gstreamer/blob/master/src/selkies_gstreamer/webrtc_input.py
This also needs to include input for Wayland interfaces. https://github.com/any1/wayvnc is an interesting reference.

[ehfd]

Since there must be another option to use keyboard and mouse operations using Wayland without uinput (in order to keep supporting Docker and Kubernetes), the virtual keyboard and virtual pointer specs may be an alternative. This is only supported for Sway and wlroots compositors.

xdg-desktop-portal can work out better for GNOME or KDE, as long as supporting Sway and wlroots as well.

The virtual keyboard and virtual pointer specs have nothing to do with uinput, confirmed.

[ehfd]

Luckily, it looks like WLR_BACKENDS=headless and WLR_LIBINPUT_NO_DEVICES=1 exist in wlroots.

VirtualGL for Wayland with VirtualGL/virtualgl#43 combined with WLR_BACKENDS=headless will be sufficient to replace VirtualGL + Xvfb.

[ehfd]

People are welcome to suggest or contribute attempts from using xdg-desktop-portal as it should be possible to do even now. It can be combined with the Pipewire plugin of GStreamer.

[ghost]

It seems that team-viewer is working on a wayland support

No idea how they get the input working.

[ehfd]

It seems that team-viewer is working on a wayland support

No idea how they get the input working.

I believe this would be through xdg-desktop-portal.

[ehfd]

https://github.com/boppreh/keyboard
https://github.com/boppreh/mouse
https://github.com/maliit/keyboard
Looks interesting.

[ehfd]

xdg-desktop-portal with GStreamer:

https://gitlab.gnome.org/-/snippets/39

[ehfd]

https://flatpak.github.io/xdg-desktop-portal/#gdbus-org.freedesktop.portal.RemoteDesktop
https://gitlab.gnome.org/-/snippets/39

https://flatpak.github.io/xdg-desktop-portal/#gdbus-org.freedesktop.portal.ScreenCast
https://gitlab.gnome.org/-/snippets/19

[ehfd]

Strong reference: Xpra-org/xpra#3750 (comment)

[ehfd]

Because of #95, we might be able to use uinput in Wayland without privileges.

[ehfd]

Weylus (https://github.com/H-M-H/Weylus) has all of the initial answers and since we are already on GStreamer, GStreamer-PipeWire can be promptly used with xdg-desktop-portal. However Wolf (https://github.com/games-on-whales/wolf) came up with a way to not rely on KDE's or GNOME's session manager and simply wrap a compositor inside a custom virtual screen.

Overriding uinput (and the keyboard/mouse as well?) is also very useful for gamepads in Wayland.

[ehfd]

gst-pipewire is essential for this capability. This GStreamer plugin is included in pipewire, not GStreamer.

Since PipeWire is not meant to be bundled with GStreamer unless Conda is used, the gst-env script for GStreamer build has been edited to find it with the environment variable GST_PLUGIN_PATH.

Check #34 (comment) for properties of pipewiresrc and pipewiresink.

https://gstreamer.freedesktop.org/documentation/gstreamer/running.html

https://gstreamer.freedesktop.org/documentation/installing/building-from-source-using-meson.html#how-does-it-work

For Conda, need conda-forge/staged-recipes#25800.