Deadlock when mixing Audio and Video

[SteveMcFarlin]

My issue is very similar to #59. The difference is my state flow is different.

gstreamer 1.20.3
gstcefsrc master with this patch applied: philn@3b9d7eb

My coded pipeline is identical to the one in the README. I enter into PAUSED to preload a web page. After it has loaded I enter into PLAYING. This will cause a deadlock when trying to send an EOS on the message bus. If I remove the audio then everything works as expected. Also replacing gstcefsrc/demux with a simple audio and video test source works fine. I did try setting queues to leaky, but this ends up causing AV sync issues.

My only thoughts are to maybe dynamically configure the output to the pipeline prior to going into PLAYING state, or modifying gstcefsrc to monitor the states and simply dump all AV data unless in PLAYING state. I really need the page to be preloaded and data flowing but not recording while in the PAUSED state. Maybe using a splitmuxsink might work. I am totally open to any ideas anyone may have.

[SteveMcFarlin]

Adding a state transition monitor to cefsrc element class, and then adding a simple toggle to dump data unless in the PLAYING state fixes my issue.

[fairbairn]

We've encountered the same issues with deadlocks.

On the latest master branch here, if the CPU is under heavy load, the cefsrc / cefdemux flow deadlocks and will not recover. The pipeline stalls and won't proceed.

We had hoped that phil's patch might work, but honestly, we're uncertain how to perform the patch to this code base from his. We attempted even to run phil's audio-ts branch latest as the plugin instead, but it's out of sync with certain features in the latest CEF, so we reverted. We used the 3 modified files from audio-ts on top of this master, and that ran, but it hung.

@philn any chance the "hack" above could be integrated into your solution? Obviously filling up buffers, detecting them, dumping the audio, and changing the play state is non-ideal, because the audio is going to get corrupted, but it would at least continue.

[MathieuDuponchelle]

@philn 's branch rebased with no conflicts on master, I have merged it now. You probably want to give master a spin again. As for this issue here, a test case with that two-step approach to going to PLAYING, showing the deadlock reliably would be helpful :)

[fairbairn]

That's great.

I do want to add, that from master, we altered the pipeline to instead sink with an flvmux and route the output to an NGINX RTMP server, and it never hung. We overtaxed the CPU, ran 12 pipelines in parallel on a GPU machine, and it never hung.

So I believe the problems stated before, although certainly a concern in terms of stability, may have been caused by the mux's characteristics.

[MathieuDuponchelle]

Perhaps simply a matter of porting matroskamux from collectpads to aggregator then

[fairbairn]

Yes. In fact, using a different mux seems to resolve the issue.

…

On Mon, May 1, 2023, 9:24 AM Mathieu Duponchelle ***@***.***> wrote: Perhaps simply a matter of porting matroskamux to collectpads then — Reply to this email directly, view it on GitHub <#69 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABILNGO3XOLR4UP4N4OG6B3XD62R3ANCNFSM6AAAAAAUETP5D4> . You are receiving this because you commented.Message ID: ***@***.***>