From 2c9eb7d1d51fca3759c65aed031c47ac1984673b Mon Sep 17 00:00:00 2001 From: Paul Berberian Date: Mon, 2 Sep 2024 15:22:51 +0200 Subject: [PATCH] [Proposal] Improve `FREEZING` mechanisms and add `Representation` deprecation Situation ========= We have lately seen on some LG and Philips TVs what we call "infinite `FREEZING`" occurences: the playback position (the `HTMLMediaElement`'s `currentTime` property) was not advancing and most of the time video media was not playing either (though in some occurences audio was) despite having largely enough media data in media buffers. Sadly our usual trick to restart playback, seeking close to the current position, didn't seem to have an effect. Most of those freezing cases were happening as playback switched from a video quality to another (sometimes both had to be specific, sometimes only the destination one had an impact). It is probably better to detect which situations causes which problems on which devices to work-around it either in the RxPlayer (when for example we KNOW that a device has issues with a given codec in general) or inside the application (when we only reproduced it on a specific application and there's many unknowns left). Yet we found out after encountering issues again and again, especially on some smart TVs, that it may not be realistic to catch in advance (before production) all potential breakages that could exist. So instead, we thought about the possibility to define an heuristic detecting that we're probably encountering this type of issue, to then work-around it automatically. Rough idea of the solution ========================== So, the idea is to detect when a `FREEZING` case occurs right when switching from a given video/audio quality to another. When that's the case: 1. We first try to "un-freeze" playback, by e.g. performing a small seek. This is what we already do today in `FREEZING` cases. 2. If that doesn't work (we're still freezing), I added here a second mechanism: we "deprecate" (the name can change) the corresponding new quality played, so it isn't played anymore in most cases. So basically once we see that switching to another quality lead to `FREEZING`, we avoid playing that quality for the rest of playback. Unlike other scenarios where we prevent playback of qualities (in the RxPlayer called `Representation`), here the deprecation is only considered if there's other, non-deprecated, qualities we can play on the current track: meaning that if all available qualities are deprecated, we will play them again. This is to protect against errors in cases where all qualities would be deprecated: because our heuristic is not 100% sure science, we would here prefer there to keep playback potentially happening (the alternative would have been stopping playback on an error). As of now, the deprecation has no impact on the API: corresponding `Representation` are still returned as if they are playable through our `getVideoTrack` API for example, and the user can still choose to call `lockVideoRepresentation` on deprecated `Representation` without knowing. As of now, the "main thread" part of the code isn't even aware of deprecation happening, for it we're just curiously never seem to be playing some `Representation`. This also means that our `DEBUG_ELEMENT` feature won't even notify for now if there's deprecation happening. This may change in the future, but for now, this is because I did not want to complexify this experiment too much. PS: I like this "deprecation" concept, as it seems to me to be portable for much more future cases where we would prefer not playing qualities because they seem to present issues (e.g. if `MEDIA_ERR_DECODE` or `BUFFER_APPEND_ERROR` errors seem to only happen with some `Representation`). Bonus: also reloading in other cases ==================================== There are several other posibilities for infinite `FREEZING`: Period changes gone wrong, random decoding issues, etc. To also provide a better solution for those cases, I choose to "reload" (in the RxPlayer, "reloading" means removing and re-creating all the media buffers, leading to a temporary black screen) if the initial "un-freezing" attempt (the seek-based one) didn't have any effect. Unlike when the freezing seems to be linked to a quality change though, there's no deprecation happening in that case. Implementation ============== FreezeResolver -------------- I renamed the very-specific `DecipherabilityFreezeDetector` class - which only handled freezes linked to DRM - into a more general `FreezeResolver` class, which will now perform all un-freezing attempts (even the seekinf one, which previously happened in the `RebufferingController` in main thread whereas it's now in `core` - thus optionally running in our `Worker`). The `FreezeResolver`'s `onObservation` method has to be called even when playback goes well, at each produced media observation, because it needs to construct its history of played segments (see next chapter). Quality switch detection ------------------------ Knowing whether we switched from a given media quality to another is not straightforward and we can never really know for sure as there's edge cases where it might be device-specific (e.g. segment replacements, segment pushing still pending etc.). However, we have a rough idea by inspecting the `HTMLMediaElement`'s `currentTime` property (which is roughly the current media position being played), and the content of our `SegmentInventory` class, which stores metadata about all media segments available in the buffer. From there, I maintain a short-term history in the `FreezeResolver` of what seemed to be the current video AND audio quality played. If a `FREEZING`, which we do not seem to be able to fix by seeking, seems to coincide with a quality switch, we propose to deprecate the `Representation`. For now this also lead to a `RELOADING` operation, though it may not even be required in some cases (e.g. replacing segments and seeking in place might do the trick). Yet reloading should have more chance of fixing it (even though it leads to a temporary black screen, where a still video frame would have been less disruptive for users). Avoiding deprecated `Representation` ------------------------------------ Then, an `AdaptationStream` is able to filter out deprecated `Representation` (unless it has no choice), when asking the `AdaptiveRepresentationSelector` (managing our Adaptive BitRate logic) which `Representation` should be played (though I'm still not sure whether the `AdaptiveRepresentationSelector` should actually be the one doing that?). Where we go from there ====================== This heuristic seems somewhat risky. as we're essentially blacklisting qualities from ever be playing again on the current content. So what I thought was to put the deprecation mechanism behind an experimental `experimental.enableRepresentationDeprecation` `loadVideo` option, that applications would have to enable (making it also possible to disable at any time easily through some config). However the reload mechanism if un-freezing fails seems OK to me for enabling it by default. If we do notice clear improvements, we might think of enabling the deprecation mechanism by default, removing the `experimental.enableRepresentationDeprecation` option. Thoughts? --- .../common/DecipherabilityFreezeDetector.ts | 147 ------- src/core/main/common/FreezeResolver.ts | 402 ++++++++++++++++++ src/core/main/worker/content_preparer.ts | 29 +- src/core/main/worker/worker_main.ts | 51 ++- .../stream/adaptation/adaptation_stream.ts | 49 ++- .../orchestrator/stream_orchestrator.ts | 8 +- src/default_config.ts | 27 ++ .../api/__tests__/option_utils.test.ts | 3 + src/main_thread/api/option_utils.ts | 30 ++ src/main_thread/api/public_api.ts | 5 + .../init/media_source_content_initializer.ts | 48 ++- .../init/multi_thread_content_initializer.ts | 7 + .../utils/create_core_playback_observer.ts | 3 + .../init/utils/rebuffering_controller.ts | 29 +- src/manifest/classes/index.ts | 4 +- src/manifest/classes/manifest.ts | 52 ++- src/manifest/classes/representation.ts | 10 + src/manifest/index.ts | 4 +- src/multithread_types.ts | 8 + .../media_element_playback_observer.ts | 38 +- src/playback_observer/types.ts | 2 + .../worker_playback_observer.ts | 2 + src/public_types.ts | 17 + 23 files changed, 733 insertions(+), 242 deletions(-) delete mode 100644 src/core/main/common/DecipherabilityFreezeDetector.ts create mode 100644 src/core/main/common/FreezeResolver.ts diff --git a/src/core/main/common/DecipherabilityFreezeDetector.ts b/src/core/main/common/DecipherabilityFreezeDetector.ts deleted file mode 100644 index 6364227692..0000000000 --- a/src/core/main/common/DecipherabilityFreezeDetector.ts +++ /dev/null @@ -1,147 +0,0 @@ -/** - * Copyright 2015 CANAL+ Group - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -import log from "../../../log"; -import type { IFreezingStatus, IRebufferingStatus } from "../../../playback_observer"; -import getMonotonicTimeStamp from "../../../utils/monotonic_timestamp"; -import type SegmentSinksStore from "../../segment_sinks"; - -export default class DecipherabilityFreezeDetector { - /** Emit the current playback conditions */ - private _segmentSinksStore: SegmentSinksStore; - - /** - * If set to something else than `null`, this is the monotonically-raising - * timestamp used by the RxPlayer when playback begin to seem to not start - * despite having decipherable data in the buffer(s). - * - * If enough time in that condition is spent, special considerations are - * taken at which point `_currentFreezeTimestamp` is reset to `null`. - * - * It is also reset to `null` when and if there is no such issue anymore. - */ - private _currentFreezeTimestamp: number | null; - - constructor(segmentSinksStore: SegmentSinksStore) { - this._segmentSinksStore = segmentSinksStore; - this._currentFreezeTimestamp = null; - } - - /** - * Support of contents with DRM on all the platforms out there is a pain in - * the *ss considering all the DRM-related bugs there are. - * - * We found out a frequent issue which is to be unable to play despite having - * all the decryption keys to play what is currently buffered. - * When this happens, re-creating the buffers from scratch, with a reload, is - * usually sufficient to unlock the situation. - * - * Although we prefer providing more targeted fixes or telling to platform - * developpers to fix their implementation, it's not always possible. - * We thus resorted to developping an heuristic which detects such situation - * and reload in that case. - * - * @param {Object} observation - The last playback observation produced, it - * has to be recent (just triggered for example). - * @returns {boolean} - Returns `true` if it seems to be such kind of - * decipherability freeze, in which case you should probably reload the - * content. - */ - public needToReload(observation: IDecipherabilityFreezeDetectorObservation): boolean { - const { readyState, rebuffering, freezing } = observation; - const bufferGap = - observation.bufferGap !== undefined && isFinite(observation.bufferGap) - ? observation.bufferGap - : 0; - if (bufferGap < 6 || (rebuffering === null && freezing === null) || readyState > 1) { - this._currentFreezeTimestamp = null; - return false; - } - - const now = getMonotonicTimeStamp(); - if (this._currentFreezeTimestamp === null) { - this._currentFreezeTimestamp = now; - } - const rebufferingForTooLong = - rebuffering !== null && now - rebuffering.timestamp > 4000; - const frozenForTooLong = freezing !== null && now - freezing.timestamp > 4000; - - if ( - (rebufferingForTooLong || frozenForTooLong) && - getMonotonicTimeStamp() - this._currentFreezeTimestamp > 4000 - ) { - const statusAudio = this._segmentSinksStore.getStatus("audio"); - const statusVideo = this._segmentSinksStore.getStatus("video"); - let hasOnlyDecipherableSegments = true; - let isClear = true; - for (const status of [statusAudio, statusVideo]) { - if (status.type === "initialized") { - for (const segment of status.value.getLastKnownInventory()) { - const { representation } = segment.infos; - if (representation.decipherable === false) { - log.warn( - "Init: we have undecipherable segments left in the buffer, reloading", - ); - this._currentFreezeTimestamp = null; - return true; - } else if (representation.contentProtections !== undefined) { - isClear = false; - if (representation.decipherable !== true) { - hasOnlyDecipherableSegments = false; - } - } - } - } - } - - if (!isClear && hasOnlyDecipherableSegments) { - log.warn( - "Init: we are frozen despite only having decipherable " + - "segments left in the buffer, reloading", - ); - this._currentFreezeTimestamp = null; - return true; - } - } - return false; - } -} - -/** Playback observation needed by the `DecipherabilityFreezeDetector`. */ -export interface IDecipherabilityFreezeDetectorObservation { - /** Current `readyState` value on the media element. */ - readyState: number; - /** - * Set if the player is short on audio and/or video media data and is a such, - * rebuffering. - * `null` if not. - */ - rebuffering: IRebufferingStatus | null; - /** - * Set if the player is frozen, that is, stuck in place for unknown reason. - * Note that this reason can be a valid one, such as a necessary license not - * being obtained yet. - * - * `null` if the player is not frozen. - */ - freezing: IFreezingStatus | null; - /** - * Gap between `currentTime` and the next position with un-buffered data. - * `Infinity` if we don't have buffered data right now. - * `undefined` if we cannot determine the buffer gap. - */ - bufferGap: number | undefined; -} diff --git a/src/core/main/common/FreezeResolver.ts b/src/core/main/common/FreezeResolver.ts new file mode 100644 index 0000000000..b98ac54019 --- /dev/null +++ b/src/core/main/common/FreezeResolver.ts @@ -0,0 +1,402 @@ +import { config } from "../../../experimental"; +import log from "../../../log"; +import type { IAdaptation, IPeriod, IRepresentation } from "../../../manifest"; +import type { + IFreezingStatus, + IRebufferingStatus, + ObservationPosition, +} from "../../../playback_observer"; +import getMonotonicTimeStamp from "../../../utils/monotonic_timestamp"; +import type SegmentSinksStore from "../../segment_sinks"; +import type { IBufferedChunk } from "../../segment_sinks"; + +/** Describe a strategy that can be taken to un-freeze playback. */ +export type IFreezeResolution = + | { + /** + * Set when there is a freeze which seem to be specifically linked to a, + * or multiple, content's `Representation`. + * + * In that case, the recommendation is to avoid playing those + * `Representation` at all. + */ + type: "deprecate-representations"; + /** The `Representation` to avoid. */ + value: Array<{ + adaptation: IAdaptation; + period: IPeriod; + representation: IRepresentation; + }>; + } + | { + /** + * Set when there is a freeze which seem to be fixable by just + * "flushing" the buffer, e.g. generally by just seeking to another, + * close, position. + */ + type: "flush"; + value: { + /** + * The relative position, when compared to the current playback + * position, we should be playing at after the flush. + */ + relativeSeek: number; + }; + } + | { + /** + * Set when there is a freeze which seem to be fixable by "reloading" + * the content: meaning re-creating a `MediaSource` and its associated + * buffers. + */ + type: "reload"; + value: null; + }; + +/** + * Sometimes playback is stuck for no known reason, despite having data in + * buffers. + * + * This can be due to relatively valid cause: performance being slow on the + * device making the content slow to start up, decryption keys not being + * obtained / usable yet etc. + * + * Yet in many cases, this is abnormal and may lead to being stuck at the same + * position and video frame indefinitely. + * + * For those situations, we have a series of tricks and heuristic, which are + * implemented by the `FreezeResolver`. + * + * @class FreezeResolver + */ +export default class FreezeResolver { + /** Emit the current playback conditions */ + private _segmentSinksStore: SegmentSinksStore; + + /** Contains a short-term history of what content has been played recently. */ + private _lastSegmentInfo: { + /** Playback history for the video data. */ + video: IPlayedHistoryEntry[]; + /** Playback history for the audio data. */ + audio: IPlayedHistoryEntry[]; + }; + + /** + * Information on the last attempt to un-freeze playback by "flushing" buffers. + * + * `null` if we never attempted to flush buffers. + */ + private _lastFlushAttempt: { + /** Monotonically-raising timestamp at the time when we attempted the flush. */ + timestamp: number; + /** Playback position at which the flush was performed, in seconds. */ + position: number; + } | null; + + /** + * If set to something else than `null`, this is the monotonically-raising + * timestamp used by the RxPlayer when playback begin to seem to not start + * despite having decipherable data in the buffer(s). + * + * If enough time in that condition is spent, special considerations are + * taken at which point `_decipherabilityFreezeStartingTimestamp` is reset to + * `null`. + * + * It is also reset to `null` when and if there is no such issue anymore. + */ + private _decipherabilityFreezeStartingTimestamp: number | null; + + constructor(segmentSinksStore: SegmentSinksStore) { + this._segmentSinksStore = segmentSinksStore; + this._decipherabilityFreezeStartingTimestamp = null; + this._lastFlushAttempt = null; + this._lastSegmentInfo = { + audio: [], + video: [], + }; + } + + /** + * Check that playback is not freezing, and if it is, return a solution that + * should be atempted to unfreeze it. + * + * Returns `null` either when there's no freeze is happening or if there's one + * but there's nothing we should do about it yet. + * + * Refer to the returned type's definition for more information. + * + * @param {Object} observation - The last playback observation produced, it + * has to be recent (just triggered for example). + * @returns {Object|null} + */ + public onNewObservation( + observation: IFreezeResolverObservation, + ): IFreezeResolution | null { + const now = getMonotonicTimeStamp(); + this._addPositionToHistory(observation, now); + + const { readyState, rebuffering, freezing, fullyLoaded } = observation; + const polledPosition = observation.position.getPolled(); + const bufferGap = + observation.bufferGap !== undefined && isFinite(observation.bufferGap) + ? observation.bufferGap + : 0; + + const { + UNFREEZING_SEEK_DELAY, + UNFREEZING_DELTA_POSITION, + FREEZING_FLUSH_FAILURE_DELAY, + } = config.getCurrent(); + + const isFrozen = + freezing !== null || + // When rebuffering, `freezing` might be not set as we're actively pausing + // playback. Yet, rebuffering occurences can also be abnormal, such as + // when enough buffer is constructed but with a low readyState (those are + // generally decryption issues). + (rebuffering !== null && readyState === 1 && (bufferGap >= 6 || fullyLoaded)); + + if (!isFrozen) { + this._decipherabilityFreezeStartingTimestamp = null; + return null; + } + + const recentFlushAttemptFailed = + this._lastFlushAttempt !== null && + now - this._lastFlushAttempt.timestamp < FREEZING_FLUSH_FAILURE_DELAY.MAXIMUM && + now - this._lastFlushAttempt.timestamp >= FREEZING_FLUSH_FAILURE_DELAY.MINIMUM && + Math.abs(polledPosition - this._lastFlushAttempt.position) < + FREEZING_FLUSH_FAILURE_DELAY.POSITION_DELTA; + + if (recentFlushAttemptFailed) { + log.warn( + "FR: A recent flush seemed to have no effect on freeze, checking for transitions", + ); + const toDeprecate = []; + for (const ttype of ["audio", "video"] as const) { + const segmentList = this._lastSegmentInfo[ttype]; + if (segmentList.length === 0) { + break; + } + let initialOccurenceOfLastSegment = segmentList[segmentList.length - 1]; + let recentQualityChangeSegment: IPlayedHistoryEntry | undefined; + for (let i = segmentList.length - 2; i >= 0; i--) { + const segment = segmentList[i]; + if (segment.segment === null) { + recentQualityChangeSegment = segment; + break; + } else if ( + segment.segment.infos.representation.uniqueId !== + initialOccurenceOfLastSegment.segment?.infos.representation.uniqueId && + initialOccurenceOfLastSegment.timestamp - segment.timestamp < 5000 + ) { + recentQualityChangeSegment = segment; + break; + } else if ( + initialOccurenceOfLastSegment.segment !== null && + segment.segment.start === initialOccurenceOfLastSegment.segment.start + ) { + initialOccurenceOfLastSegment = segment; + } + } + if ( + recentQualityChangeSegment !== undefined && + recentQualityChangeSegment.segment !== null + ) { + if (initialOccurenceOfLastSegment.segment === null) { + log.debug("FR: Freeze when beginning to play a content, reloading"); + return { type: "reload", value: null }; + } else if ( + initialOccurenceOfLastSegment.segment.infos.period.id !== + recentQualityChangeSegment.segment.infos.period.id + ) { + log.debug("FR: Freeze when switching Period, reloading"); + return { type: "reload", value: null }; + } else if ( + initialOccurenceOfLastSegment.segment.infos.representation.uniqueId !== + recentQualityChangeSegment.segment.infos.representation.uniqueId + ) { + log.warn( + "FR: Freeze when switching Representation, deprecating", + initialOccurenceOfLastSegment.segment.infos.representation.bitrate, + ); + toDeprecate.push({ + adaptation: initialOccurenceOfLastSegment.segment.infos.adaptation, + period: initialOccurenceOfLastSegment.segment.infos.period, + representation: initialOccurenceOfLastSegment.segment.infos.representation, + }); + } + } + } + + if (toDeprecate.length > 0) { + this._decipherabilityFreezeStartingTimestamp = null; + return { type: "deprecate-representations", value: toDeprecate }; + } else { + this._decipherabilityFreezeStartingTimestamp = null; + return { type: "reload", value: null }; + } + } + if ( + freezing !== null && + !observation.position.isAwaitingFuturePosition() && + now - freezing.timestamp > UNFREEZING_SEEK_DELAY + ) { + this._lastFlushAttempt = { + timestamp: now, + position: polledPosition + UNFREEZING_DELTA_POSITION, + }; + + return { + type: "flush", + value: { relativeSeek: UNFREEZING_DELTA_POSITION }, + }; + } + + if ((bufferGap < 6 && !fullyLoaded) || readyState > 1) { + this._decipherabilityFreezeStartingTimestamp = null; + return null; + } + + if (this._decipherabilityFreezeStartingTimestamp === null) { + this._decipherabilityFreezeStartingTimestamp = now; + } + const rebufferingForTooLong = + rebuffering !== null && now - rebuffering.timestamp > 4000; + const frozenForTooLong = freezing !== null && now - freezing.timestamp > 4000; + + if ( + (rebufferingForTooLong || frozenForTooLong) && + getMonotonicTimeStamp() - this._decipherabilityFreezeStartingTimestamp > 4000 + ) { + let hasOnlyDecipherableSegments = true; + let isClear = true; + for (const ttype of ["audio", "video"] as const) { + const status = this._segmentSinksStore.getStatus(ttype); + if (status.type === "initialized") { + for (const segment of status.value.getLastKnownInventory()) { + const { representation } = segment.infos; + if (representation.decipherable === false) { + log.warn( + "FR: we have undecipherable segments left in the buffer, reloading", + ); + this._decipherabilityFreezeStartingTimestamp = null; + return { type: "reload", value: null }; + } else if (representation.contentProtections !== undefined) { + isClear = false; + if (representation.decipherable !== true) { + hasOnlyDecipherableSegments = false; + } + } + } + } + } + + if (!isClear && hasOnlyDecipherableSegments) { + log.warn( + "FR: we are frozen despite only having decipherable " + + "segments left in the buffer, reloading", + ); + this._decipherabilityFreezeStartingTimestamp = null; + return { type: "reload", value: null }; + } + } + return null; + } + + /** + * Add entry to `this._lastSegmentInfo` for the position that is currently + * played according to the given `observation`. + * + * @param {Object} observation + * @param {number} currentTimestamp + */ + private _addPositionToHistory( + observation: IFreezeResolverObservation, + currentTimestamp: number, + ): void { + const position = observation.position.getPolled(); + for (const ttype of ["audio", "video"] as const) { + const status = this._segmentSinksStore.getStatus(ttype); + if (status.type === "initialized") { + for (const segment of status.value.getLastKnownInventory()) { + if (segment.start <= position && segment.end > position) { + this._lastSegmentInfo[ttype].push({ + segment, + position, + timestamp: currentTimestamp, + }); + } + } + } else { + this._lastSegmentInfo[ttype].push({ + segment: null, + position, + timestamp: currentTimestamp, + }); + } + if (this._lastSegmentInfo[ttype].length > 100) { + const toRemove = this._lastSegmentInfo[ttype].length - 100; + this._lastSegmentInfo[ttype].splice(0, toRemove); + } + + const removalTs = currentTimestamp - 60000; + let i; + for (i = 0; i < this._lastSegmentInfo[ttype].length; i++) { + if (this._lastSegmentInfo[ttype][i].timestamp > removalTs) { + break; + } + } + if (i > 0) { + this._lastSegmentInfo[ttype].splice(0, i); + } + } + } +} + +/** Entry for the playback history maintained by the `FreezeResolver`. */ +interface IPlayedHistoryEntry { + /** + * Segment and related information that seemed to be played at the + * associated timestamp and playback position. + * + * Note that this is only a guess and not a certainty. + */ + segment: null | IBufferedChunk; + /** + * Playback position, in seconds, as seen on the `HTMLMediaElement`, at which + * we were playing. + */ + position: number; + /** Monotonically-raising timestamp for that entry. */ + timestamp: number; +} + +/** Playback observation needed by the `FreezeResolver`. */ +export interface IFreezeResolverObservation { + /** Current `readyState` value on the media element. */ + readyState: number; + /** + * Set if the player is short on audio and/or video media data and is a such, + * rebuffering. + * `null` if not. + */ + rebuffering: IRebufferingStatus | null; + /** + * Set if the player is frozen, that is, stuck in place for unknown reason. + * Note that this reason can be a valid one, such as a necessary license not + * being obtained yet. + * + * `null` if the player is not frozen. + */ + freezing: IFreezingStatus | null; + /** + * Gap between `currentTime` and the next position with un-buffered data. + * `Infinity` if we don't have buffered data right now. + * `undefined` if we cannot determine the buffer gap. + */ + bufferGap: number | undefined; + position: ObservationPosition; + /** If `true` the content is loaded until its maximum position. */ + fullyLoaded: boolean; +} diff --git a/src/core/main/worker/content_preparer.ts b/src/core/main/worker/content_preparer.ts index 4200801215..5ab14a61e9 100644 --- a/src/core/main/worker/content_preparer.ts +++ b/src/core/main/worker/content_preparer.ts @@ -26,7 +26,7 @@ import type { IManifestRefreshSettings } from "../../fetchers"; import { ManifestFetcher, SegmentFetcherCreator } from "../../fetchers"; import SegmentSinksStore from "../../segment_sinks"; import type { INeedsMediaSourceReloadPayload } from "../../stream"; -import DecipherabilityFreezeDetector from "../common/DecipherabilityFreezeDetector"; +import FreezeResolver from "../common/FreezeResolver"; import { limitVideoResolution, throttleVideoBitrate } from "./globals"; import sendMessage, { formatErrorForSender } from "./send_message"; import TrackChoiceSetter from "./track_choice_setter"; @@ -84,7 +84,13 @@ export default class ContentPreparer { currentMediaSourceCanceller.linkToSignal(contentCanceller.signal); - const { contentId, url, hasText, transportOptions } = context; + const { + contentId, + url, + hasText, + transportOptions, + enableRepresentationDeprecation, + } = context; let manifest: IManifest | null = null; // TODO better way @@ -148,13 +154,12 @@ export default class ContentPreparer { }, currentMediaSourceCanceller.signal, ); - const decipherabilityFreezeDetector = new DecipherabilityFreezeDetector( - segmentSinksStore, - ); + const freezeResolver = new FreezeResolver(segmentSinksStore); this._currentContent = { cmcdDataBuilder, contentId, - decipherabilityFreezeDetector, + enableRepresentationDeprecation, + freezeResolver, mediaSource, manifest: null, manifestFetcher, @@ -328,6 +333,12 @@ export interface IPreparedContentData { * conditions with a CDN. */ cmcdDataBuilder: CmcdDataBuilder | null; + /** + * If `true`, the RxPlayer can enable its "Representation deprecation" + * mechanism, where it avoid loading Representation that it suspect + * have issues being decoded on the current device. + */ + enableRepresentationDeprecation: boolean; /** * Interface to the MediaSource implementation, allowing to buffer audio * and video media segments. @@ -342,10 +353,10 @@ export interface IPreparedContentData { */ manifest: IManifest | null; /** - * Specific module detecting freezing issues due to lower-level - * decipherability-related bugs. + * Specific module detecting freezing issues and trying to work-around + * them. */ - decipherabilityFreezeDetector: DecipherabilityFreezeDetector; + freezeResolver: FreezeResolver; /** * Perform the adaptive logic, allowing to choose the best Representation for * the different types of media to load. diff --git a/src/core/main/worker/worker_main.ts b/src/core/main/worker/worker_main.ts index c3532e789b..598a89eda3 100644 --- a/src/core/main/worker/worker_main.ts +++ b/src/core/main/worker/worker_main.ts @@ -507,6 +507,7 @@ function loadOrReloadPreparedContent( const { contentId, cmcdDataBuilder, + enableRepresentationDeprecation, manifest, mediaSource, representationEstimator, @@ -515,14 +516,6 @@ function loadOrReloadPreparedContent( } = preparedContent; const { drmSystemId, enableFastSwitching, initialTime, onCodecSwitch } = val; playbackObservationRef.onUpdate((observation) => { - if (preparedContent.decipherabilityFreezeDetector.needToReload(observation)) { - handleMediaSourceReload({ - timeOffset: 0, - minimumPosition: 0, - maximumPosition: Infinity, - }); - } - // Synchronize SegmentSinks with what has been buffered. ["video" as const, "audio" as const, "text" as const].forEach((tType) => { const segmentSinkStatus = segmentSinksStore.getStatus(tType); @@ -530,6 +523,48 @@ function loadOrReloadPreparedContent( segmentSinkStatus.value.synchronizeInventory(observation.buffered[tType] ?? []); } }); + + const freezeResolution = preparedContent.freezeResolver.onNewObservation(observation); + if (freezeResolution !== null) { + switch (freezeResolution.type) { + case "reload": { + log.info("WP: Planning reload due to freeze"); + handleMediaSourceReload({ + timeOffset: 0, + minimumPosition: 0, + maximumPosition: Infinity, + }); + break; + } + case "flush": { + log.info("WP: Flushing buffer due to freeze"); + sendMessage({ + type: WorkerMessageType.NeedsBufferFlush, + contentId, + value: { + relativeResumingPosition: freezeResolution.value.relativeSeek, + relativePosHasBeenDefaulted: false, + }, + }); + break; + } + case "deprecate-representations": { + log.info("WP: Planning Representation deprecation due to freeze"); + const content = freezeResolution.value; + if (enableRepresentationDeprecation) { + manifest.deprecateRepresentations(content); + } + handleMediaSourceReload({ + timeOffset: 0, + minimumPosition: 0, + maximumPosition: Infinity, + }); + break; + } + default: + assertUnreachable(freezeResolution); + } + } }); const initialPeriod = diff --git a/src/core/stream/adaptation/adaptation_stream.ts b/src/core/stream/adaptation/adaptation_stream.ts index 6ba8b16daf..78e765ed08 100644 --- a/src/core/stream/adaptation/adaptation_stream.ts +++ b/src/core/stream/adaptation/adaptation_stream.ts @@ -94,11 +94,9 @@ export default function AdaptationStream( let previouslyEmittedBitrate: number | undefined; const initialRepIds = content.representations.getValue().representationIds; - const initialRepresentations = content.adaptation.representations.filter( - (r) => - arrayIncludes(initialRepIds, r.id) && - r.decipherable !== false && - r.isSupported !== false, + const initialRepresentations = getRepresentationList( + content.adaptation.representations, + initialRepIds, ); /** Emit the list of Representation for the adaptive logic. */ @@ -160,8 +158,13 @@ export default function AdaptationStream( cancelCurrentStreams.cancel(); } const newRepIds = content.representations.getValue().representationIds; - const newRepresentations = content.adaptation.representations.filter((r) => - arrayIncludes(newRepIds, r.id), + + // NOTE: We expect that the rest of the RxPlayer code is already handling + // cases where the list of playable `Representation` changes: + // decipherability updates, `Representation` deprecation etc. + const newRepresentations = getRepresentationList( + content.adaptation.representations, + newRepIds, ); representationsList.setValueIfChanged(newRepresentations); cancelCurrentStreams = new TaskCanceller(); @@ -493,3 +496,35 @@ export default function AdaptationStream( return bufferGoalRatio; } } + +/** + * Construct the list of the `Representation` to play, based on what's supported + * and what the API seem to authorize. + * @param {Array.} availableRepresentations - All available + * Representation in the current `Adaptation`, including unsupported ones. + * @param {Array.} authorizedRepIds - The subset of `Representation` + * that the API authorize us to play. + * @returns {Array.} + */ +function getRepresentationList( + availableRepresentations: IRepresentation[], + authorizedRepIds: string[], +): IRepresentation[] { + const filteredRepresentations = availableRepresentations.filter( + (r) => + arrayIncludes(authorizedRepIds, r.id) && + r.decipherable !== false && + !r.deprecated && + r.isSupported !== false, + ); + if (filteredRepresentations.length > 0) { + return filteredRepresentations; + } + // Retry without deprecated `Representation` + return availableRepresentations.filter( + (r) => + arrayIncludes(authorizedRepIds, r.id) && + r.decipherable !== false && + r.isSupported !== false, + ); +} diff --git a/src/core/stream/orchestrator/stream_orchestrator.ts b/src/core/stream/orchestrator/stream_orchestrator.ts index c3c30cfcb8..7cca2ff9f9 100644 --- a/src/core/stream/orchestrator/stream_orchestrator.ts +++ b/src/core/stream/orchestrator/stream_orchestrator.ts @@ -17,11 +17,7 @@ import config from "../../../config"; import { MediaError } from "../../../errors"; import log from "../../../log"; -import type { - IManifest, - IDecipherabilityUpdateElement, - IPeriod, -} from "../../../manifest"; +import type { IManifest, IUpdatedRepresentationInfo, IPeriod } from "../../../manifest"; import type { IReadOnlyPlaybackObserver } from "../../../playback_observer"; import isNullOrUndefined from "../../../utils/is_null_or_undefined"; import queueMicrotask from "../../../utils/queue_microtask"; @@ -295,7 +291,7 @@ export default function StreamOrchestrator( * @returns {Promise} */ async function onDecipherabilityUpdates( - updates: IDecipherabilityUpdateElement[], + updates: IUpdatedRepresentationInfo[], ): Promise { const segmentSinkStatus = segmentSinksStore.getStatus(bufferType); const ofCurrentType = updates.filter( diff --git a/src/default_config.ts b/src/default_config.ts index bffc6498f9..8c805024a5 100644 --- a/src/default_config.ts +++ b/src/default_config.ts @@ -556,6 +556,33 @@ const DEFAULT_CONFIG = { */ UNFREEZING_DELTA_POSITION: 0.001, + /** + * `FREEZING` is a situation where the playback does not seem to advance despite + * all web indicators telling us we can. + * Those may be linked to device issues, but sometimes are just linked to + * performance or it may be just decryption negotiations taking more time than + * expected. + * + * Anyway we might in the RxPlayer "flush" the buffer in that situation to + * un-stuck playback (this is usually done by seeking close to the current + * position), + * + * Yet that "flush" attempt may not in the end be succesful. + * + * If a flush was performed more than `FREEZING_FLUSH_FAILURE_DELAY.MINIMUM` + * milliseconds ago and less than `FREEZING_FLUSH_FAILURE_DELAY.MAXIMUM` + * milliseconds ago, yet a `FREEZING` situation at roughly the same playback + * position (deviating from less than + * `FREEZING_FLUSH_FAILURE_DELAY.POSITION_DELTA` seconds from it) is + * encountered again, we will consider that the flushing attempt was unsuccesful + * and try more agressive solutions (such as reloading the content). + */ + FREEZING_FLUSH_FAILURE_DELAY: { + MAXIMUM: 20000, + MINIMUM: 4000, + POSITION_DELTA: 1, + }, + /** * The RxPlayer has a recurring logic which will synchronize the browser's * buffers' buffered time ranges with its internal representation in the diff --git a/src/main_thread/api/__tests__/option_utils.test.ts b/src/main_thread/api/__tests__/option_utils.test.ts index 255808c924..dd47023176 100644 --- a/src/main_thread/api/__tests__/option_utils.test.ts +++ b/src/main_thread/api/__tests__/option_utils.test.ts @@ -257,6 +257,9 @@ describe("API - parseLoadVideoOptions", () => { textTrackElement: undefined, textTrackMode: "native", url: undefined, + experimentalOptions: { + enableRepresentationDeprecation: false, + }, }; it("should throw if no option is given", () => { diff --git a/src/main_thread/api/option_utils.ts b/src/main_thread/api/option_utils.ts index 4a933059bc..dbfce3d554 100644 --- a/src/main_thread/api/option_utils.ts +++ b/src/main_thread/api/option_utils.ts @@ -65,29 +65,55 @@ export interface IParsedConstructorOptions { /** * Base type which the types for the parsed options of the RxPlayer's * `loadVideo` method exend. + * @see ILoadVideoOptions */ interface IParsedLoadVideoOptionsBase { + /** @see ILoadVideoOptions.url */ url: string | undefined; + /** @see ILoadVideoOptions.transport */ transport: string; + /** @see ILoadVideoOptions.autoPlay */ autoPlay: boolean; + /** @see ILoadVideoOptions.initialManifest */ initialManifest: ILoadedManifestFormat | undefined; + /** @see ILoadVideoOptions.keySystems */ keySystems: IKeySystemOption[]; + /** @see ILoadVideoOptions.lowLatencyMode */ lowLatencyMode: boolean; + /** @see ILoadVideoOptions.minimumManifestUpdateInterval */ minimumManifestUpdateInterval: number; + /** @see ILoadVideoOptions.requestConfig */ requestConfig: IRequestConfig; + /** @see ILoadVideoOptions.startAt */ startAt: IParsedStartAtOption | undefined; + /** @see ILoadVideoOptions.enableFastSwitching */ enableFastSwitching: boolean; + /** @see ILoadVideoOptions.defaultAudioTrackSwitchingMode */ defaultAudioTrackSwitchingMode: IAudioTrackSwitchingMode | undefined; + /** @see ILoadVideoOptions.onCodecSwitch */ onCodecSwitch: "continue" | "reload"; + /** @see ILoadVideoOptions.checkMediaSegmentIntegrity */ checkMediaSegmentIntegrity?: boolean | undefined; + /** @see ILoadVideoOptions.checkManifestIntegrity */ checkManifestIntegrity?: boolean | undefined; + /** @see ILoadVideoOptions.manifestLoader */ manifestLoader?: IManifestLoader | undefined; + /** @see ILoadVideoOptions.referenceDateTime */ referenceDateTime?: number | undefined; + /** @see ILoadVideoOptions.representationFilter */ representationFilter?: IRepresentationFilter | string | undefined; + /** @see ILoadVideoOptions.segmentLoader */ segmentLoader?: ISegmentLoader | undefined; + /** @see ILoadVideoOptions.serverSyncInfos */ serverSyncInfos?: IServerSyncInfos | undefined; + /** @see ILoadVideoOptions.mode */ mode: IRxPlayerMode; + /** @see ILoadVideoOptions.cmcd */ cmcd: ICmcdOptions | undefined; + /** @see ILoadVideoOptions.experimentalOptions */ + experimentalOptions: { + enableRepresentationDeprecation: boolean; + }; __priv_manifestUpdateUrl?: string | undefined; __priv_patchLastSegmentInSidx?: boolean | undefined; } @@ -465,6 +491,10 @@ function parseLoadVideoOptions(options: ILoadVideoOptions): IParsedLoadVideoOpti mode, url, cmcd: options.cmcd, + experimentalOptions: { + enableRepresentationDeprecation: + options.experimentalOptions?.enableRepresentationDeprecation === true, + }, }; } diff --git a/src/main_thread/api/public_api.ts b/src/main_thread/api/public_api.ts index 6c4e1d2e19..92ad2990d4 100644 --- a/src/main_thread/api/public_api.ts +++ b/src/main_thread/api/public_api.ts @@ -766,6 +766,7 @@ class Player extends EventEmitter { segmentLoader, serverSyncInfos, mode, + experimentalOptions, __priv_manifestUpdateUrl, __priv_patchLastSegmentInSidx, url, @@ -918,6 +919,8 @@ class Player extends EventEmitter { autoPlay, bufferOptions, cmcd, + enableRepresentationDeprecation: + experimentalOptions.enableRepresentationDeprecation, keySystems, lowLatencyMode, transport: transportPipelines, @@ -960,6 +963,8 @@ class Player extends EventEmitter { autoPlay, bufferOptions, cmcd, + enableRepresentationDeprecation: + experimentalOptions.enableRepresentationDeprecation, keySystems, lowLatencyMode, transportOptions, diff --git a/src/main_thread/init/media_source_content_initializer.ts b/src/main_thread/init/media_source_content_initializer.ts index 1ebd02ded7..bac7c5f0b9 100644 --- a/src/main_thread/init/media_source_content_initializer.ts +++ b/src/main_thread/init/media_source_content_initializer.ts @@ -27,7 +27,7 @@ import AdaptiveRepresentationSelector from "../../core/adaptive"; import CmcdDataBuilder from "../../core/cmcd"; import { ManifestFetcher, SegmentFetcherCreator } from "../../core/fetchers"; import createContentTimeBoundariesObserver from "../../core/main/common/create_content_time_boundaries_observer"; -import DecipherabilityFreezeDetector from "../../core/main/common/DecipherabilityFreezeDetector"; +import FreezeResolver from "../../core/main/common/FreezeResolver"; import SegmentSinksStore from "../../core/segment_sinks"; import type { IStreamOrchestratorOptions, @@ -51,7 +51,7 @@ import type { } from "../../public_types"; import type { ITransportPipelines } from "../../transports"; import areArraysOfNumbersEqual from "../../utils/are_arrays_of_numbers_equal"; -import assert from "../../utils/assert"; +import assert, { assertUnreachable } from "../../utils/assert"; import createCancellablePromise from "../../utils/create_cancellable_promise"; import isNullOrUndefined from "../../utils/is_null_or_undefined"; import noop from "../../utils/noop"; @@ -666,9 +666,7 @@ export default class MediaSourceContentInitializer extends ContentInitializer { speed, cancelSignal, ); - const decipherabilityFreezeDetector = new DecipherabilityFreezeDetector( - segmentSinksStore, - ); + const freezeResolver = new FreezeResolver(segmentSinksStore); if (mayMediaElementFailOnUndecipherableData) { // On some devices, just reload immediately when data become undecipherable @@ -683,9 +681,15 @@ export default class MediaSourceContentInitializer extends ContentInitializer { ); } + // Handle "FREEZING" situation and try to un-freeze playbackObserver.listen( (observation) => { - if (decipherabilityFreezeDetector.needToReload(observation)) { + const freezeResolution = freezeResolver.onNewObservation(observation); + if (freezeResolution === null) { + return; + } + + const triggerReload = () => { let position: number; const lastObservation = playbackObserver.getReference().getValue(); if (lastObservation.position.isAwaitingFuturePosition()) { @@ -698,6 +702,32 @@ export default class MediaSourceContentInitializer extends ContentInitializer { ? !playbackObserver.getIsPaused() : autoPlay; onReloadOrder({ position, autoPlay: autoplay }); + }; + + switch (freezeResolution.type) { + case "reload": { + log.info("Init: Planning reload due to freeze"); + triggerReload(); + break; + } + case "flush": { + log.info("Init: Flushing buffer due to freeze"); + const currentTime = playbackObserver.getCurrentTime(); + const relativeResumingPosition = freezeResolution.value.relativeSeek; + const wantedSeekingTime = currentTime + relativeResumingPosition; + playbackObserver.setCurrentTime(wantedSeekingTime); + break; + } + case "deprecate-representations": { + const contents = freezeResolution.value; + if (this._settings.enableRepresentationDeprecation) { + manifest.deprecateRepresentations(contents); + } + triggerReload(); + break; + } + default: + assertUnreachable(freezeResolution); } }, { clearSignal: cancelSignal }, @@ -1151,6 +1181,12 @@ export interface IInitializeArguments { * When set to an object, enable "Common Media Client Data", or "CMCD". */ cmcd?: ICmcdOptions | undefined; + /** + * If `true`, the RxPlayer can enable its "Representation deprecation" + * mechanism, where it avoid loading Representation that it suspect + * have issues being decoded on the current device. + */ + enableRepresentationDeprecation: boolean; /** Every encryption configuration set. */ keySystems: IKeySystemOption[]; /** `true` to play low-latency contents optimally. */ diff --git a/src/main_thread/init/multi_thread_content_initializer.ts b/src/main_thread/init/multi_thread_content_initializer.ts index b45fa44faa..d8371288c4 100644 --- a/src/main_thread/init/multi_thread_content_initializer.ts +++ b/src/main_thread/init/multi_thread_content_initializer.ts @@ -154,6 +154,7 @@ export default class MultiThreadContentInitializer extends ContentInitializer { value: { contentId, cmcd: this._settings.cmcd, + enableRepresentationDeprecation: this._settings.enableRepresentationDeprecation, url: this._settings.url, hasText: this._hasTextBufferFeature(), transportOptions, @@ -1831,6 +1832,12 @@ export interface IInitializeArguments { * When set to an object, enable "Common Media Client Data", or "CMCD". */ cmcd?: ICmcdOptions | undefined; + /** + * If `true`, the RxPlayer can enable its "Representation deprecation" + * mechanism, where it avoid loading Representation that it suspect + * have issues being decoded on the current device. + */ + enableRepresentationDeprecation: boolean; /** Every encryption configuration set. */ keySystems: IKeySystemOption[]; /** `true` to play low-latency contents optimally. */ diff --git a/src/main_thread/init/utils/create_core_playback_observer.ts b/src/main_thread/init/utils/create_core_playback_observer.ts index 1d18caf1a4..005d40ea7a 100644 --- a/src/main_thread/init/utils/create_core_playback_observer.ts +++ b/src/main_thread/init/utils/create_core_playback_observer.ts @@ -82,6 +82,8 @@ export interface ICorePlaybackObservation { rebuffering: IRebufferingStatus | null; freezing: IFreezingStatus | null; bufferGap: number | undefined; + /** If `true` the content is loaded until its maximum position. */ + fullyLoaded: boolean; } /** @@ -149,6 +151,7 @@ export default function createCorePlaybackObserver( }, readyState: observation.readyState, speed: lastSpeed, + fullyLoaded: observation.fullyLoaded, }; } diff --git a/src/main_thread/init/utils/rebuffering_controller.ts b/src/main_thread/init/utils/rebuffering_controller.ts index 23d12eab46..bf5b7a25b5 100644 --- a/src/main_thread/init/utils/rebuffering_controller.ts +++ b/src/main_thread/init/utils/rebuffering_controller.ts @@ -95,39 +95,16 @@ export default class RebufferingController extends EventEmitter { const discontinuitiesStore = this._discontinuitiesStore; const { buffered, position, readyState, rebuffering, freezing } = observation; - const { - BUFFER_DISCONTINUITY_THRESHOLD, - FREEZING_STALLED_DELAY, - UNFREEZING_SEEK_DELAY, - UNFREEZING_DELTA_POSITION, - } = config.getCurrent(); + const { BUFFER_DISCONTINUITY_THRESHOLD, FREEZING_STALLED_DELAY } = + config.getCurrent(); if (freezing !== null) { const now = getMonotonicTimeStamp(); - - const referenceTimestamp = - prevFreezingState === null - ? freezing.timestamp - : prevFreezingState.attemptTimestamp; - - if ( - !position.isAwaitingFuturePosition() && - now - referenceTimestamp > UNFREEZING_SEEK_DELAY - ) { - log.warn("Init: trying to seek to un-freeze player"); - this._playbackObserver.setCurrentTime( - this._playbackObserver.getCurrentTime() + UNFREEZING_DELTA_POSITION, - ); - prevFreezingState = { attemptTimestamp: now }; - } - if (now - freezing.timestamp > FREEZING_STALLED_DELAY) { if (rebuffering === null) { playbackRateUpdater.stopRebuffering(); @@ -137,8 +114,6 @@ export default class RebufferingController extends EventEmitter} items + */ + public deprecateRepresentations( + items: Array<{ + period: Period; + adaptation: Adaptation; + representation: Representation; + }>, + ) { + const updates = []; + for (const item of items) { + const period = this.getPeriod(item.period.id); + if (period === undefined) { + continue; + } + const adaptation = period.getAdaptation(item.adaptation.id); + if (adaptation === undefined) { + continue; + } + const representation = adaptation.getRepresentation(item.representation.id); + if (representation === undefined) { + continue; + } + representation.deprecated = true; + updates.push({ + manifest: this, + period, + adaptation, + representation, + }); + } + if (updates.length > 0) { + this.trigger("deprecationUpdate", updates); + } + } + /** * @deprecated only returns adaptations for the first period * @returns {Array.} @@ -721,8 +765,8 @@ function updateDeciperability( adaptation: Adaptation; representation: Representation; }) => boolean | undefined, -): IDecipherabilityUpdateElement[] { - const updates: IDecipherabilityUpdateElement[] = []; +): IUpdatedRepresentationInfo[] { + const updates: IUpdatedRepresentationInfo[] = []; for (const period of manifest.periods) { for (const adaptation of period.getAdaptations()) { let hasOnlyUndecipherableRepresentations = true; diff --git a/src/manifest/classes/representation.ts b/src/manifest/classes/representation.ts index 2ea0bfebc2..5f2e77bc04 100644 --- a/src/manifest/classes/representation.ts +++ b/src/manifest/classes/representation.ts @@ -112,6 +112,15 @@ class Representation implements IRepresentationMetadata { * @see ITrackType */ public trackType: ITrackType; + /** + * When set to `true`, the `Representation` should not be played, unless + * there's no other choice. + * + * Note that this property should __NEVER__ be updated directly on an + * instanciated `Representation`, you are supposed to rely on + * `Manifest` methods for this. + */ + public deprecated: boolean; /** * @param {Object} args @@ -124,6 +133,7 @@ class Representation implements IRepresentationMetadata { ) { this.id = args.id; this.uniqueId = generateRepresentationUniqueId(); + this.deprecated = false; this.bitrate = args.bitrate; this.codecs = []; this.trackType = trackType; diff --git a/src/manifest/index.ts b/src/manifest/index.ts index b42643cefd..5ce324d28e 100644 --- a/src/manifest/index.ts +++ b/src/manifest/index.ts @@ -1,5 +1,5 @@ import type { - IDecipherabilityUpdateElement, + IUpdatedRepresentationInfo, ICodecSupportInfo, Period, Adaptation, @@ -26,7 +26,7 @@ export type IAdaptation = Adaptation; export type IRepresentation = Representation; export type { - IDecipherabilityUpdateElement, + IUpdatedRepresentationInfo, ICodecSupportInfo, IPeriodsUpdateResult, IRepresentationIndex, diff --git a/src/multithread_types.ts b/src/multithread_types.ts index a9460942cd..e58dec39d8 100644 --- a/src/multithread_types.ts +++ b/src/multithread_types.ts @@ -107,6 +107,12 @@ export interface IContentInitializationData { * When set to an object, enable "Common Media Client Data", or "CMCD". */ cmcd?: ICmcdOptions | undefined; + /** + * If `true`, the RxPlayer can enable its "Representation deprecation" + * mechanism, where it avoid loading Representation that it suspect + * have issues being decoded on the current device. + */ + enableRepresentationDeprecation: boolean; /** * URL at which the content's Manifest is accessible. * `undefined` if unknown. @@ -410,6 +416,8 @@ export interface ISerializedPlaybackObservation { * `undefined` if we cannot determine the buffer gap. */ bufferGap: number | undefined; + /** If `true` the content is loaded until its maximum position. */ + fullyLoaded: boolean; } /** diff --git a/src/playback_observer/media_element_playback_observer.ts b/src/playback_observer/media_element_playback_observer.ts index 0dca709b49..7341f41a0c 100644 --- a/src/playback_observer/media_element_playback_observer.ts +++ b/src/playback_observer/media_element_playback_observer.ts @@ -467,6 +467,13 @@ export default class PlaybackObserver { Infinity; } + const fullyLoaded = hasLoadedUntilTheEnd( + basePosition, + currentRange, + mediaTimings.ended, + mediaTimings.duration, + this._lowLatencyMode, + ); const rebufferingStatus = getRebufferingStatus({ previousObservation, currentObservation: mediaTimings, @@ -475,7 +482,7 @@ export default class PlaybackObserver { lowLatencyMode: this._lowLatencyMode, withMediaSource: this._withMediaSource, bufferGap, - currentRange, + fullyLoaded, }); const freezingStatus = getFreezingStatus( @@ -502,6 +509,7 @@ export default class PlaybackObserver { freezing: freezingStatus, bufferGap, currentRange, + fullyLoaded, }); if (log.hasLevel("DEBUG")) { log.debug( @@ -646,7 +654,7 @@ function getRebufferingStatus({ withMediaSource, lowLatencyMode, bufferGap, - currentRange, + fullyLoaded, }: { /** Previous Playback Observation produced. */ previousObservation: IPlaybackObservation; @@ -676,22 +684,11 @@ function getRebufferingStatus({ * `undefined` if we cannot determine this due to a browser issue. */ bufferGap: number | undefined; - /** - * Range of buffered data where the current position is (`basePosition`). - * - * `null` if we've no buffered data at the current position. - * `undefined` if we cannot determine this due to a browser issue. - */ - currentRange: { start: number; end: number } | null | undefined; + /** If `true` the content is loaded until its maximum position. */ + fullyLoaded: boolean; }): IRebufferingStatus | null { const { REBUFFERING_GAP } = config.getCurrent(); - const { - position: currentTime, - duration, - paused, - readyState, - ended, - } = currentObservation; + const { position: currentTime, paused, readyState, ended } = currentObservation; const { rebuffering: prevRebuffering, @@ -699,14 +696,6 @@ function getRebufferingStatus({ position: prevTime, } = previousObservation; - const fullyLoaded = hasLoadedUntilTheEnd( - basePosition, - currentRange, - ended, - duration, - lowLatencyMode, - ); - const canSwitchToRebuffering = readyState >= 1 && observationEvent !== "loadedmetadata" && @@ -943,5 +932,6 @@ function getInitialObservation(mediaElement: IMediaElement): IPlaybackObservatio freezing: null, bufferGap: 0, currentRange: null, + fullyLoaded: false, }); } diff --git a/src/playback_observer/types.ts b/src/playback_observer/types.ts index 554c93b1cb..f98150fa10 100644 --- a/src/playback_observer/types.ts +++ b/src/playback_observer/types.ts @@ -140,6 +140,8 @@ export interface IPlaybackObservation extends Omit