ffmpeg: Hardware accelerated using MediaCodec ?

[Shoresh613]

I think it would be really neat to be able to decode and encode video on the phone using the GPU and MediaCodec. Currently it is only enabled for a JVM context seemingly (?).

What was the reason for only enabling it for that context, and not for a shell context? Can it be enabled for a shell context without much trouble? If so, that would be great.

Tagging @truboxl who enabled the JVM context.

[twaik]

It is not possible. Using MediaCodec requires using real GLES + ANativeWindow inside non-JVM app, which is not possible without building agaist private Android APIs.

[Shoresh613]

Ok thanks 🙏

[tomty89]

@Shoresh613 Just to clarify, what I meant on SE was that the API itself requires a JVM context. An analogy is the AudioTrack audio output driver implemented in VLC. If you actually have some app bundles and wraps VLC in their java/kotlin code (i.e., the app "itself"), you can use the driver, but if you run it from Termux, you can't. What works in Termux would be support in programs that makes use of (public?) native(/C?) APIs, like OpenSLES or AAudio in the case of audio output.

And I'm not sure why @truboxl enabled the support while we seemingly can't make use of it in Termux.

There seems to be something called libmediandk in Android. I'm have no idea when it was introduced, or whether it is / will be supported in ffmpeg though.

For your reference:
https://developer.android.com/ndk/guides/stable_apis#libmediandk

[yuipp]

Some guy successful use h264_mediacodec codec on Android 8, I ask him for details

[yuipp]

just writed to him in PM, when see answer - write here

[tomty89]

One possibility I happen to think of is, maybe one can write a little java wrapper and compile and convert it to a dex and use /system/bin/app_process to run ffmpeg?

[truboxl]

ffmpeg 6.0 onwards has automatic fallback to use mediacodec when Java method fails.
FFmpeg/FFmpeg@0405679 FFmpeg/FFmpeg@0ff18a7
Termux usecases always exclusively rely on this fallback to use mediacodec.

ffmpeg mediacodec support is still relatively unexplored territory so expect bugs.
Feel free to submit bug reports to ffmpeg upstream since we barely patch ffmpeg at all.