How do I increase the AI Deck WiFi streamer frequency?

[shreeram-murali]

Hi everyone!

I've just been getting started with the AI Deck and I ran the wifi-img-streamer/opencv-viewer.py according to the documentation.

Everything works fine and I'm receiving images (324 x 244) at a rate of roughly 7 fps. Is there a way I can increase this? I'm assuming I'd have to trade-off on the resolution, and that's fine -- is there a way I can stream lower resolution images at a higher rate?

Thanks!

[evoggy]

Hi!

You could try to lower the image resolution like you suggested, I think just changing the camera config to PI_CAMERA_QQVGA should do it, then the image should be 160x120. Don't forget to update the CAM_WIDTH and CAM_HEIGHT accordingly. The WiFi image streamer should work out of the box since the resolution is sent with the image.

[Shu6hamSingha1]

Hi @evoggy!

I tried your approach. However, it worked for JPEG ENCODING only and not for RAW ENCODING. Moreover, the image streaming rate increased to around 12 fps only.

Also, the console tab in CFCLIENT shows that captureTime (image capturing time for the onboard camera) is around 70 ms, which is equivalent to around 15 fps. But according to the datasheet of Himax HM01B0 (used in AI deck), it can work upto 60 fps. While debugging the issue, I found that the following code (Line 271) in wifi-img-streamer.c is adding the delay of 70 ms:
xEventGroupWaitBits(evGroup, CAPTURE_DONE_BIT, pdTRUE, pdFALSE, (TickType_t)portMAX_DELAY);.

I want to achieve at least 30 fps of image streaming rate from the AI deck onboard camera for my application. Please let me know how I can achieve this.

[evoggy]

Hi!

Sorry for the late reply on this one. The line you are referring to is waiting until a full image has been captured by the camera, so this is why the delay is there. It should be possible to increase the FPS, we've heard from users which have higher rates, but we haven't had time to investigate it yet. But like you say, I think the current limitation is from the camera capture.

[knmcguire]

hi @Shu6hamSingha1

@evoggy is currently not around as he will be on a holiday for 3 more weeks.

As I understood it, the limitation is on the GAP8 side, and it can not capture more quickly than that, so achieving the full 60 fps, or even 30 fps will be difficult on the aideck. Then we need to create a more dedicated wifistreaming deck with a chip that is better able to handle the image capturing of the camera, as the gap8 is probably not suitable for that.

I think that @evoggy can explain it better, so I'll leave this topic open for him to answer once he is back.

[shreeram-murali]

hi @knmcguire

If I understand it correctly, the GAP8 chip constraints the WiFi streaming frequency — is that right? Would I be able to achieve 60 fps (or 30 fps) onboard the AI deck for image processing?

[evoggy]

Hi!

As far as I've seen the current limitation is the camera capture which takes too long. So with the current implementation you would not be able to reach this. Although the camera itself should be able to reach 60 FPS, it seems as if it's not running on full speed, so this needs to be improved in the firmware.