[XB1] Player switches to avc1 codec after suspend/resume

[victorpasoshnikov]

b/173550025

Change-Id: I72d733e1036c672de491b627c0026ddf88345426

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 58.12%. Comparing base (25efb1e) to head (cdb9911).
Report is 3 commits behind head on main.

❗ Current head cdb9911 differs from pull request most recent head 963322e

Please upload reports for the commit 963322e to get more accurate results.

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #1504   +/-   ##
=======================================
  Coverage   58.12%   58.12%           
=======================================
  Files        1774     1774           
  Lines       85781    85781           
=======================================
+ Hits        49862    49863    +1     
+ Misses      35919    35918    -1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[victorpasoshnikov]

At the time of suspend Cobalt releases player, all D3D12 objects, including vp9/av1 compiled shaders and extended resources. In time of resume Cobalt tries to resume player. This process looks as

1. start async extended resources acquiring
2. start async shaders compilation
3. request supported codecs
   But at the time of 3) the 1) and 2) are still not completed so Cobalt makes decision to use avc1 codec.
   What I offer:
4. don't clear the cache of supported codecs after shader compile;
5. don't release compiled shaders and ID3D12Devece as well with them;
   In this case Cobalt makes codecs choice based on previously cached available codecs. When it tries to create vp9 or av1 decoder it maybe wait a reasonable time (in my cases - 1- 2s) until extended resources will be acquired and the rest D3D12 objects will be created

[TyHolc]

At the time of suspend Cobalt releases player, all D3D12 objects, including vp9/av1 compiled shaders and extended resources. In time of resume Cobalt tries to resume player. This process looks as

1. start async extended resources acquiring
2. start async shaders compilation
3. request supported codecs
   But at the time of 3) the 1) and 2) are still not completed so Cobalt makes decision to use avc1 codec.
   What I offer:
4. don't clear the cache of supported codecs after shader compile;
5. don't release compiled shaders and ID3D12Devece as well with them;
   In this case Cobalt makes codecs choice based on previously cached available codecs. When it tries to create vp9 or av1 decoder it maybe wait a reasonable time (in my cases - 1- 2s) until extended resources will be acquired and the rest D3D12 objects will be created

When you say don't clear the cache, are you referring to keeping it in memory or keeping it on disk, like in ApplicationData::Current->LocalCacheFolder?

I would say we should avoid the first case, but the second case is fine, though I'd need to verify which file location we should use as they have different lifetimes.

[victorpasoshnikov]

When you say don't clear the cache

I mean removing this code
if (is_av1_shader_compiled_ && is_vp9_shader_compiled_) {
MimeSupportabilityCache::GetInstance()->ClearCachedMimeSupportabilities();
}
MimeSupportabilities is std::vector in memory. It consists of few elements and needs in small amount of memory

[victorpasoshnikov]

5. don't release compiled shaders

I mean don't call
Av1VideoDecoder::ReleaseShaders();
VpxVideoDecoder::ReleaseShaders();
In this case decoders will keep their compiled shaders, i.e. binary arrays in memory. There are not too large to take them into account

[victorpasoshnikov]

And keep in mind that this PR depends on #1524. If you make positive decision for current PR, please don't merge it before I do rebase & may be refactoring of it.

[TyHolc]

5. don't release compiled shaders

I mean don't call Av1VideoDecoder::ReleaseShaders(); VpxVideoDecoder::ReleaseShaders(); In this case decoders will keep their compiled shaders, i.e. binary arrays in memory. There are not too large to take them into account

I don't think we should keep memory while in a suspended state, I tried to find documentation from Microsoft to see if there was precedent for that but I couldn't.

Is the codec switching very noticeable to users? If not, I think we can leave it as-is.

[amurovanyi]

Is the codec switching very noticeable to users?

By switching to AVC in the worst case users loose 2K resolution and HDR. For high quality videos this may be quite noticeable.

[TyHolc]

5. don't release compiled shaders

I mean don't call Av1VideoDecoder::ReleaseShaders(); VpxVideoDecoder::ReleaseShaders(); In this case decoders will keep their compiled shaders, i.e. binary arrays in memory. There are not too large to take them into account

I don't think we should keep memory while in a suspended state, I tried to find documentation from Microsoft to see if there was precedent for that but I couldn't.

Is the codec switching very noticeable to users? If not, I think we can leave it as-is.

I'm still not fully convinced we should keep shaders in-memory in a suspended state. @jasonzhangxx can you take a look?