image.value is empty byte string

[mgreenbe]

I'm having a strange issue. When I capture an image from the image recorder, it's value ends up being the empty byte string. If I then ask for the value in the subsequent cell, I get the bytes. What's causing this asynchrony? I tried to "fix" it by adding a delay between defining image and asking for its value, but that didn't work. What's the "correct" way of reliably grabbing the current camera frame's value?

[martinRenou]

This is actually the correct behavior. This asynchronicity is due to the communication between the Python back-end and the JavaScript front-end.

Another way of dealing with it is to define a callback function that will be called when the image value has changed:

def on_value_changed(*args, **kwargs):
    # The value has changed! (Note that the following print won't show up in JupyterLab, 
    # it only shows up in the classical Jupyter Notebook. You need to use the Output 
    # widget if you want to capture streams in JLab)
    print(image_recorder.image.value)
    print(args, kwargs)
    # Do whatever you want with your newly received image

image_recorder.image.observe(on_value_changed, 'value')

image_recorder.recording = True

The Ouptut widget I mention in the comment: https://ipywidgets.readthedocs.io/en/stable/examples/Output%20Widget.html

This observe method comes from traitlets https://traitlets.readthedocs.io/en/stable/using_traitlets.html#observe (All interactive widgets are HasTraits classes internally)

[mgreenbe]

The callback doesn't seem to running. I would have expected to see an infinite sequence of "The callback ran." strings printed (once for each frame?) and the did_callback_run changed to True.

[martinRenou]

Try defining did_the_callback_run as being a global variable in your callback. Otherwise, it is seen as a local variable by Python if I'm correct.

def on_value_changed(_):
    global did_the_callback_run

    did_the_callback_run = True

Also, I suggest using the Output widget for capturing the stdout and stderr of your callbacks. Because sometimes the Notebook does not know where to put the output of the streams and simply discard them, that is why you don't see anything printed.

from ipywidgets import Output

out = Output()
out.layout.border = 'solid'
display(out)

@out.capture()
def on_value_changed(_):
    global did_the_callback_run
    print("It works!!")
    did_the_callback_run = True

I would have expected to see an infinite sequence of "The callback ran.

It will print it infinitely only if you set recording to True at the end of the callback, that will create an infinite callback loop.

from ipywidgets import Output

out = Output()
out.layout.border = 'solid'
display(out)

@out.capture()
def on_value_changed(_):
    global did_the_callback_run
    print("It works!!")
    did_the_callback_run = True

    image_recorder.recording = True

I would agree that recording is a bad name, it should be something like a capture_frame() method.

[martinRenou]

Also, it might be super slow to capture frames infinitely... ipywebrtc needs some love in terms of optimization (PRs welcome!).

So try using the sleep function from Python in your callback to reduce the capture framerate.

[pmeier]

@martinRenou Your suggestions for the infinite loop does not work (at least not for ipywebrtc==0.6.0):

Every time I execute the last cell again, it will add a line to the output

but the point was to not needing to do that.

[pmeier]

I found if you display the recorder and hit the camera icon, this actually triggers an infinite loop. Looking at the code, I have no idea what is executed by pressing the camera icon. Does someone have some pointers for me to check what is happening besides setting recorder.recording = True?

[coderforlife]

I am running into a similar problem.

If the value recorder.recording is set to True manually then the callback is called once, by the recorder.recording is still True during the callback. Since it is already True, there is no effect in changing it to True again. However, when you click the camera icon, the recorder.recording is False during the callback and thus setting it to True has an effect.

However, I have noticed that if the callback takes just a bit too long to complete, it will still fail to be called again, even if set to True.

[jiapei100]

Same here... Still buggy... Magic webrtc ....
I can display live camera, but if I do any process, the resultant image won't be able to show live in another window.
How??
I mean:

• The FIRST window shows original images captured from camera
• The SECOND window shows the filtered captured images in live

[gino-m]

Hi! Was this ever resolved? Also in our case the on_value_changed callback is never called and the image returned is empty (empty byte array).