Import of Selkies JS Interposer for WebRTC to Linux Joystick support

[danisla]

Fixes #90
Fixes #55

New features

Import of Selkies JS Interposer for WebRTC to Linux Joystick support without uinput dependency

• LD_PRELOAD syscall interposer to emulate gamepads.
• Intercepts open() requests to /dev/input/jsX devices.
• Interposer connects to selkies unix domain socket.
• Interposer writes log to /tmp/selkies_js.log
• Selkies WebRTC app sends Javascript gamepad events to socket.
• Up to 4 simultaneous gamepads supported from WebRTC.
• Build as debian package and installed to /usr/local/lib/selkies-js-interposer/joystick_interposer.so
• Dev container updated to auto-load interposer.
• Update to github workflow to build and publish image and release asset.
• Add joystick interposer to example Dockerfile

Add turnserver to example docker image

• Helps with initial connectivity in environments like WSL with Docker.
• Can be overridden with env vars or cli args.

Supported Gamepads

I've tested this with apps like retroarch and it seems to work well.

Currently only Xpad like gamepads are emulated for the button and axes mapping.

I've tested with the following controllers:

• Xbox Series X/S Controller
• Stadia Controller
• 8Bitdo SN30 PRO

Others will surely work just fine, especially if they have similar button layouts to an Xpad.

Please let me know if there is a gamepad that doesn't work and we can try to accommodate.

Testing

The testing branch has been built with the new joystick interposer pre-installed and added to the default LD_PRELOAD environment variable. This branch also contains the integrated turnserver to the example container, hence the new port 3478 to the docker run command.

You can test it with the example docker container like this:

Ubuntu 22.04

docker run --pull=always --name selkies -it --rm -p 8080:8080 -p 3478:3478 ghcr.io/selkies-project/selkies-gstreamer/gst-py-example:testing-ubuntu22.04

For Ubuntu 20.04:

docker run --pull=always --name selkies -it --rm -p 8080:8080 -p 3478:3478 ghcr.io/selkies-project/selkies-gstreamer/gst-py-example:testing-ubuntu20.04

Usage

1. Connect to the web interface.
2. Connect a gamepad to your local computer.
3. Press any button on the gamepad to connect it to the web interface.
4. In the remote session, run a program like jstest-gtk to verify you see the Selkies Controller
5. Connect up to 4 total gamepads using the same method.
   

The example docker image has these environment variables set:

export LD_PRELOAD=/usr/local/lib/selkies-js-interposer/joystick_interposer.so
export SDL_JOYSTICK_DEVICE=/dev/input/js0

The entrypoint script also creates fake jsX files in /dev/input like below to help programs that try to list the /dev/input directory discover the joystick devices:

sudo mkdir -p /dev/input
sudo touch /dev/input/{js0,js1,js2,js3}

Some software programs use different methods to discover linux joysticks. The LD_PRELOAD method seems to work for most direct linux applications and the SDL_JOYSTICK_DEVICE env var helps with SDL applications.

[ehfd]

Wow. This actually works?...

One aspect of testing that should be done is with Wine games and applications.

[ehfd]

#28 (comment)

Could this approach be expanded to allow true relative mouse cursors?

[danisla]

Wow. This actually works?...

One aspect of testing that should be done is with Wine games and applications.

If someone can help test, that would be great.

[danisla]

#28 (comment)

Could this approach be expanded to allow true relative mouse cursors?

I think the approach would be similar, yes. My plan is to investigate doing this for mouse emulation later so we can completely remove the dependency on uinput and have full support in container environments.
However, Xorg has a funny way of loading core input devices like Mice and Keyboards so it might have to be a custom X11 module or a full uinput interposer. Either way, that's a separate task from the gamepad support in this PR.

[ehfd]

Good to know that the approach is similar with mouse devices.

Unfortunately, I don't have a real gamepad device at the moment and I am having a hard time with time management.

@xhejtman perhaps interested?
Or any of the community.

[ehfd]

Overall, looks okay but was just skimming. Made comments.

[ehfd]

This commit is a great milestone which is a step forward from all known Linux remote desktop implementations working in containers.