Support ScreenCapture loopback

[Kree0]

#876
I added a new host: ScreenCaptureKit in macos, which only provides input devices for capturing the sound of the Display.
When I test it in KVM virtualized mocOS-14, it captures sound intermittently, I don't know why this happens, can anyone help me check the code?

[louis030195]

any update on this? need asap for https://github.com/louis030195/screen-pipe

how can i help?

i just tried this PR to record screen to wav and it works

https://gist.github.com/louis030195/013fda2fa6b373f815bbeeb39e524090

[Kree0]

any update on this? need asap for https://github.com/louis030195/screen-pipe

how can i help?

i just tried this PR to record screen to wav and it works

https://gist.github.com/louis030195/013fda2fa6b373f815bbeeb39e524090

So it works fine in macOS? Maybe my previous testing method was incorrect.

[louis030195]

@Kree0 yes works well, I use it to record .mp3 files & transcribe with OpenAI's whisper 24/7 on my Macbook pro M3 max

[rustdesk]

Good job.

[louis030195]

update:

latest macos release broke this PR (15.0). issue does not happen on 14.5

thread '' panicked at /Users/matthewdi/.cargo/registry/src/index.crates.io-6f17d22bba15001f/objc_id-0.1.1/src/id.rs:52:9:
Attempted to construct an Id from a null pointer
note: run with RUST_BACKTRACE=1 environment variable to display a backtrace
libc++abi: terminating due to uncaught foreign exception
zsh: abort ./target/release/screenpipe --debug

[Kree0]

update:

latest macos release broke this PR (15.0). issue does not happen on 14.5

thread '' panicked at /Users/matthewdi/.cargo/registry/src/index.crates.io-6f17d22bba15001f/objc_id-0.1.1/src/id.rs:52:9:
Attempted to construct an Id from a null pointer
note: run with RUST_BACKTRACE=1 environment variable to display a backtrace
libc++abi: terminating due to uncaught foreign exception
zsh: abort ./target/release/screenpipe --debug

Can you run it with RUST_BACKTRACE=1 to show the full stack?

[louis030195]

will focus on this PR in upcoming days

[Poordeveloper]

will focus on this PR in upcoming days

Did you get a chance to check the crash?

[louis030195]

https://github.com/SSheldon/rust-objc-id/blob/6527cdf28fef2b155c67c1681f548ecfd794003c/src/id.rs#L52

seems the crash happens here

but on the other hand i see that 15.0 macos is beta right?

so maybe this will be fixed by apple (idk)

will just push to prod my repo with this branch as deps and people that use beta macos wont have access to the feature

[louis030195]

done: mediar-ai/screenpipe@ca12e0d#diff-43858b5b9d45785def40a91685eced0da516c6e6db0038862945361fab28cdde

[louis030195]

fyi there is memory leak

[louis030195]

here memory leak

[louis030195]

or actually this one too?

@Kree0 do you have any idea how to fix this leak? i use this branch in production and it grows memory forever, trying to find a fix

[louis030195]

fyi $150 bounty on fixing this: mediar-ai/screenpipe#183 (comment)

[louis030195]

fyi working on major leaks (there are multiple) related to screencapturekit

mediar-ai/screenpipe#236

[Kree0]

Sorry for the late reply. I have been busy in the past few months. The memory leak seems to have been resolved in mediar-ai/screencapturekit-rs, mediar-ai/screencapturekit-rs. Should I replace the dependencies with them?

[louis030195]

@Kree0 considering we should implement this PR using another lib than screencapturekit-rs

mediar-ai/screenpipe#297

currently i experienced 5-8 users having issues with screencapture kit when macos version below 13, usually seg fault, not sure it's an issue in my implementation or something with macos api

https://github.com/yury/cidre

[kaiwen-wang]

Did any of you try https://crates.io/crates/screen-capture-kit yet?

I also tried ruhear on https://crates.io/crates/ruhear, which seems to cause null pointer problems aizcutei/ruhear#6

[louis030195]

what is the best way to impl this?

@Kree0 @rustdesk @Poordeveloper @kaiwen-wang

$300 bounty to make this work without seg fault or memory leaks

mediar-ai/screenpipe#297 (comment)

[Kree0]

This looks a little difficult. And I don't have a mac device or time to do this in the near future, can see if others have time.

[thewh1teagle]

See yury/cidre#23
This crate seems more stable and makes it easier to add screen capture support to cpal.

[Kree0]

See yury/cidre#23 This crate seems more stable and makes it easier to add screen capture support to cpal.

I‘ll take a look in upcoming days.