Gyroscope/Accelerometer support for PS4, PS3, and Switch modes

[vgf89]

This is WIP on my branch here: https://github.com/vgf89/GP2040-CE/tree/DS4-IMU
I'm using an MPU6050 breakout board. Once I'm pretty much done I'll submit a pull request.

TODO:

• Implement MPU6050 library, initial PoC test (spitting data out to analog stick state)
• Add missing PS4 report fields
• Extend gamepad state to include accel/gyro data (G's, deg/sec floats)
• Populate PS4 report (verify scale/format of accel/gyro)
• Test with PS4 and SteamInput (had to add 0x054c id to get motion on PC)
• Add drift calibration (on startup for now, but it needs to be saved in flash and put on a hotkey or something)
• Implement AddonOptions/WebUI
  • SDA/SCL pins, i2c block, i2c speed, address 0x68/0x69
  • sensor rates
  • sensor scale (precision vs max speed tradeoff)
  • low pass filter setting (should decrease jitter at the expense of responsiveness and latency) These settings seem mostly unnecessary now that I've played with them. Low pass filter increases latency and decreases throughput too much, and the amount of noise and accuracy seems similar on different scale settings. Compared to a real DS4, the highest speeds and scales actually seems right. Their IMU is extremely similar in capabilities anyways, so I'm not too surprised.
  • Orientation options: N W S E aka 0 90 180 270
  • Upside-down checkbox
  • Gyro calibration from within UI, or a hotkey?
• Verify ideal default options
• PS3, Switch support?
• Optional: Make timestamp time dependent
• Optional: Add automatic gyro drift calibration at standstill

Have you checked the GP2040-CE documentation HERE to confirm this feature doesn't already exist?

Yes

Which problem is this feature request solving?

PS4, PS3, and Switch controllers support gyroscope and accelerometer aka IMU inputs. Many popular Switch games use it (Zelda BOTW/TOTK where it's required to complete some puzzles, Splatoon for precision aiming, optional Mario Kart steering control) and a handful of PS3 and PS4 titles do as well. Steam Input in particular allows heavy customization of controllers including gyro aiming, which makes PC mouse-based first and third person shooters enjoyable. GP2040-CE does not have any way of collecting or packaging motion data.

Describe the solution you would like to see

It looks like analog controls are being actively worked on (which is awesome BTW!), but adding motion controls as well would make it fairly trivial to create nearly fully-featured handheld controllers for Switch, PS3, and PS4 systems.

Adding motion to PS4 mode especially would help in making more featureful controllers for PC/Linux(Steam Deck)/Steam Input since that could support analog triggers and a touchpad as well. It might be easiest to add motion to this mode as it's currently just padded out (or in the mystery field I guess) in the report descriptor:

GP2040-CE/headers/gamepad/descriptors/PS4Descriptors.h

Line 129 in 13d8daa

uint32_t padding : 24;

Looks like the descriptor for PS3 cuts off before it gets to the motion data. I'm pretty unfamiliar with USB/HID descriptors. How easy would it be to extend this to include motion data?

GP2040-CE/headers/gamepad/descriptors/HIDDescriptors.h

Line 122 in 13d8daa

uint8_t r2_axis;

I don't know enough about Switch to have any suggestions there and its descriptor looks... more barebones/less documented than the other ones? Or I'm just not looking in the right places. Anyways I've found some relevant discussion as well as a what appears to be a PR for a complete Switch Pro Controller emulator for Arduino Pro Micro. I've linked all of that below.

Of course we would also need support for a 6DoF IMU chip to make any of this usable. It appears that many IMUs support both I2C and SPI. For starters, Switch controllers supposedly to use an LSM6DS, for which Sparkfun has a breakout board, documentation, and a tutorial. (EDIT: MPU6050 breakouts are also widely available and way cheaper)

References:
https://www.psdevwiki.com/ps4/DS4-USB#Data_Format
http://eleccelerator.com/wiki/index.php/DualShock_3#HID_Report_Mapping
dekuNukem/Nintendo_Switch_Reverse_Engineering#7
https://github.com/matlo/GIMX-firmwares/pull/5/files
https://www.sparkfun.com/products/18020

Describe alternatives you've considered

I've just had a look around the repo and found the relevant documentation I've linked above. Hacking together a custom controller firmware might be possible with GIMX as a base, but I'd much prefer to use an RP2040 board instead of an Arduino if/when I start trying to build controllers.

Can you submit a pull request?

WIP

Are you planning on working on this feature yourself?

PS4 WIP. Not sure about PS3 or Switch yet.

[dogtopus]

For PS4, byte 10 and 11 needs to be filled with the timestamp, and byte 13-24 are the IMU readings (all in int16 little endian).

Also the bit 1 (bitmask 0x02) needs to be set in 0x03 report byte 4, and the 5 int16 parameters (likely for the defining how to interpret the IMU data) from byte 8 onwards should be set to appropriate values. I'm not sure what the parameters are but had some guess here although I don't have good ways to verify them.

[TheTrainGoes]

Hi @vgf89,

Are you on the Discord? If so please send me a DM.

[vgf89]

@TheTrainGoes Sorry for the delay, I just sent you a DM under the handle HoloPengin

[vgf89]

I think this is pretty much ready to go for a draft PR. As soon as I've verified that the rebase worked I'll open up a PR and upload a test build for feedback.

The only mode supported right now is PS4. Full Accelerometer and Gyro support is included. I had to switch to an official Vendor ID and Product ID (instead of the Panthera controller) for Windows and Linux to see the IMU data, though IIRC my PS4 itself didn't care and happily ingested the IMU data either way.

There is a checkbox to enable the gyro drift calibration on the next boot. It is enabled by default, so the gyro will calibrate when the user reboots the first time after enabling the addon, and the setting gets turned off automatically after doing so. Enabling the checkbox again in the UI will trigger calibration on bootup again.

Incomplete things, probably won't implement myself:

I've not attempted adding Switch support, but I suspect the report descriptor of the Hori Pokken we pretend to be doesn't fit motion data anywhere. We'll need to swap that out with one that does (Official, Power A, Gulikit, 8bitdo, anything recent that's not a Hori). The Switch has Zelda and Splatoon with make heavy use of gyro aim, so people will want this support if they build handheld Switch controllers based on GP2040.

I recently tried messing with the PS3 report and report descriptor but all I ended up doing was breaking all inputs when doing so (even with "fixed" descriptors people have written to go with pc drivers) and my PS3 didn't seem to recognize any inputs either. There's not too much you can do with the IMU here anyways, just steer in racing games, control the cyclone in Ratchet & Clank, or play Super Rub 'a' Dub and Flower. It'd be a nice to have, but not too important. For full motion controls with aiming and whatnot, we'd want to look into emulating PS Move instead.

Someone else with more HID knowledge might be able to figure these things out, but I don't plan to right now.

[vgf89]

So it turns out the 0x03 feature report stuff (thanks @dogtopus for the reference implementation) is all that I needed to get this working everywhere on pc, linux/steamos, and PS4, all without using s0ny's vidpid. SDL was originally ignoring the imu both because it wasn't enabled in the feature bitfield and because the imu parameters were zeroes. We also might need to hardcode the 0x03 response length to sizeof(output_0x03) instead of ignoring it if reqlen was wrong, but I need to actually test without that change again to be sure.

I need to choose correct scale values and clean up unnecessary changes I had made and push again, This suddenly is a lot closer to being complete though.

I still want to make some sort of accel calibration routine for webconfig but that'll be another day.

[dogtopus]

Glad to hear that Steam input respects that flag. I know hid-sony didn't because it's made by $**y.

[voltron4lyfe]

Hi @vgf89 -- are you still working on this?

[nikitalita]

@vgf89 what did you have left to do on your branch besides the ps3/switch support?

[mikepparks]

With the work implemented in #1084, the core of gyroscope and accelerometer support has been added, and are expected to release with version 0.7.10. Please check out the pull request details for information on what was implemented. Additional device support will need to be implemented via addons to expand on these features as the current implementation uses the already-existing Wii extension support to add MotionPlus (gyro) and Nunchuk (accelerometer) values.

Also in that release, PS3-PS5 modes will support this data. Switch support will depend on a new input mode that is not based on the Pokken controller in order for the necessary sensor data to be available. That work will come with a future release.