Proposed Deprecation of FTL Protocol in OBS #4021
Replies: 29 comments 84 replies
-
|
What I want to know is what FTL offers the world that other protocols, such as WebRTC and RIST, do not? RIST is an open RTP/UDP standard, and WebRTC has widespread adoption and can solve a lot of the same problems. If those protocols had been supported by OBS before these FTL-based projects had been started, would they have still been started? What is the advantage of FTL over other protocols beyond the mere fact that OBS supports it? It's worth noting that OBS is very likely going to begin working on adding generic WebRTC support, with the ability for services to create plugins that offer their signaling method of choice. |
Beta Was this translation helpful? Give feedback.
-
|
While SRT is supported it does not seem that there is anything using it? I haven't gone all the way through SRT yet however I am struggling to even find adequate documentation that I could even base an implementation off of. Furthermore, I am not entirely sure how FTL negatively impacts OBS right now to justify its removal without the implementation of a suitable alternative (ex. WebRTC, RIST). What is desired, in my use case at least, is a straightforward protocol to deliver RTP packets and FTL provides that with a very easy implementation. I would also like to challenge your classification of FTL as a dead technology. In my opinion a dead technology is one which is no longer used or adopted, yet there are multiple services still using FTL including mine. There are also plans to standardize and continue the development of the FTL protocol and I feel that deprecating it without at least seeing what the future holds would be jumping the gun a bit. |
Beta Was this translation helpful? Give feedback.
-
This might be a reasonable place to start: https://github.com/obsproject/obs-studio/wiki/Streaming-With-SRT-Protocol I would also invite @eric to share some of his experiencing using SRT in production settings.
From the OP:
Essentially, it comes down to a support problem, which brings with it the weight of maintenance. People come into our support channels complaining about problems and we track down their problems to their use of FTL for one reason or another. We could spend time working on fixing OBS's issues with FTL, which could be time-consuming, but it's hard to justify doing that when so few people are using it anymore, especially with Mixer dying. It was developed to solve a Beam/Mixer problem, not an Internet problem, and as such it wasn't really developed with openness in mind (as evidenced by Mixer's refusal to release a server-side implementation). If we are going to spend time on a protocol in OBS, we would rather it be one that was open from the beginning, and actually has community and industry buy-in. The only reason YouNow, Glimesh, etc. even bothered to base their services on FTL wasn't because it was a good protocol, but for two reasons: 1) it was supported by OBS, and 2) it wasn't RTMP. In my opinion, those aren't good enough reasons to pick a protocol. Standardization should also be a factor, or else every company out there would invent their own proprietary protocol and want to get it into OBS. I know, because other companies have tried to do that in OBS -- they propose some sort of non-standard janky protocol, and try to get OBS to support it for their own use. It has happened, and we've turned them down. If I'm being completely honest here (and this is my own opinion), I don't think we should have accepted the FTL output in OBS to begin with, and it was aggressive of Mixer to push it by fiat from the beginning. The only reason FTL still exists at all is because still OBS supports it. It's a reverse chicken-and-egg problem. If your issue is that there isn't currently a viable alternative to FTL in OBS for your purposes, then fine, that's a valid concern. We would be willing to talk about adjusting the timeline until a viable alternative is in place (most likely WebRTC). But I have yet to hear the merits of why FTL itself should continue to exist in light of other protocols that do the same thing but with more industry support and existing standardization. I'm all ears, though. |
Beta Was this translation helpful? Give feedback.
-
It is not. SRT cannot compete with the low delay of FTL at all. And in addition to that, in order to deliver the video to a web browser, this requires both buffering and remuxing of the packet stream, which adds further delays.
I don't quite understand how adding protocols is a bad thing. At the very least, you should allow the addition of protocols through a plugin that gets installed separately, such that (at least on Windows) your signed and pre-built binaries can be used with any protocol and manually specified ingest server anyone could possibly want. Currently, that's not possible. Yes, someone can fork OBS and build their own binaries, but unless these binaries become well known, anti-cheat systems everywhere will block them from capturing video games (had been a big problem with FTL enabled builds of OBS before it got added to the official releases), require the purchasing (and proper use!) of expensive security certificates from an approved certification authority or just simply fail to run because modern security systems would block them ("[anti-virus] hasn't seen an executable with this file size and sha2 checksum before, for your safety, you may not run this"). This was the key motivation to even try to get it into OBS, rather than publishing and updating a fork, where whenever an update was released, BattlEye and EasyAntiCheat (the most popular solutions video game vendors were using at the time) would have it blocked, and people would roll back to the older version or never bother to update in the first place.
That OBS supports it is a bonus. FTL is proven technology. Beam/Mixer have been using it together with Janus for ~6 years (only about 2 of which it has been in OBS).
There is no viable alternative, period. Even using the full WebRTC stack in OBS (current implementations do exist) doesn't offer as low of a delay as using FTL does (it's not a real concern to shave off those 50-100ms, whereas FTL's transport was designed for just that). When using FTL with a good ingest implementation (something that's not quite there, yet, either), getting the internal delays in OBS down would be the next logical step (they're still quite significant sometimes, even though they could be as low as 7-8ms with a GPU encoder these days). |
Beta Was this translation helpful? Give feedback.
-
|
Serious question: Do you take bribes, and if so, how much would it take for you to reconsider? FTL is the only proven solution that offers a low delay by design, and also the solution that offers the lowest possible delay out of everything else out there. There is (currently) no other way to deliver low latency video like this to a web browser at all, and all the the different transport protocols that are left either inherently impose a significant video delay, require expensive transposition at the ingest (adds a delay!) or can't deal with synchronized video at all (requires a buffer to work, adds delays, and includes a retransmission mechanism that would add accumulative delay whenever an error occurs). Out of the three reasons you stated, the first two are just plain wrong, and the third one, while understandable, usually comes down to one thing and one thing only: a bad internet connection (be that jitter, overprovisioned and congested routes to the ingest or other forms of packet loss) |
Beta Was this translation helpful? Give feedback.
-
Can you explain how the first two reasons are just plain wrong? From the OP:
This is at the very least a matter of opinion, and our opinion is that it's dead. If you have a different opinion fine, but that doesn't mean the reason is "wrong". If you want to keep FTL around, then your job is to change our opinion. Convince us that FTL is not actually dead and is worth keeping around in context of other standards that had more standardized and open development with more industry buy-in.
Are you suggesting that Mixer actually did have an open source server component released publicly? Can you link it to me?
Are you offering to take ownership of the FTL implementation in OBS's code base, then? |
Beta Was this translation helpful? Give feedback.
-
This is actually something that we have been discussing and may be willing to do |
Beta Was this translation helpful? Give feedback.
-
|
Beta Was this translation helpful? Give feedback.
-
I am assuming here that this was also SRT to the client player, right?
FTL is not resilient on packet loss by design. Its goal is to offer the lowest latency possible. If you add a retransmission mechanism, you need to have a (rather large in this scenario) buffer both on the output and the ingest's input. If you lose a frame, you wait for the next one, because by the time you realize something's amiss, that frame that would need retransmission is already out of date. Instead, you communicate to the output that you'd like to have a keyframe as soon as possible and continue from there. This will cause the video to shortly freeze because a frame was lost, but keeps the delay low, which is what you want here (resilience to be as real-time as possible, rather than resilience to deliver every frame). FTL works over "bad connections" (temporarily congested), it doesn't work over "broken" connections (that is those which intentionally block or drop traffic) or connections with insufficient bandwidth to reliably deliver a keyframe whenever one is needed. |
Beta Was this translation helpful? Give feedback.
-
I believe your assumption of SRT's inner workings here is incorrect, as far as I'm aware SRT uses forward error correction instead of retransmission for low latency packet loss resilience. (Correct me if I'm wrong) EDIT: pkviet informed me that RIST does FEC, and SRT does too, but only since the very latest release. SRT used to not support FEC. My point still stands, FEC is better than no resilience by design, and incurs no latency penalty. |
Beta Was this translation helpful? Give feedback.
-
That depends on how it works. In typical failure situations, entire UDP packets (multiple of which would make up a h.264 frame) are lost. To account for this, a significant amount of bandwidth needs to be used for this error correction, and for the example of Reed-Solomon coding in order to recover a frame of which a packet was lost, at least as many more packets with the same size of the lost packets are required. In order to compute the correcting code, you also need an output buffer and compute the counter value over all packets of a frame. This means incurring a delay of at least one frame for both the output buffer and the input buffer (actual delay depends on the framerate as a result), and additionally, the delay of computing the correcting codes. It may be possible to immediately send off what is in the output buffer while the computation happens and then not send any error correction packets, if all packets were delivered error free, but this difficult (you'd need to be able to reliably do all of that in under 3ms) to get right and introduces about the same delay as retransmitting lost packets, possibly a worse one on the ingest, since error correcting computation is required in addition to transmitting extra packets. If packets can be delivered and repaired significantly faster than frames are generated this works out great, in resource constrained environments (typical situation where issues occur) not as much. |
Beta Was this translation helpful? Give feedback.
-
SRT implements a fixed latency window that is uses for retransmits of lots packets. Any packet that was retransmitted and received within the window will not be lost in the final output. Anything that wasn't received before the latency window ends is lost. FEC is currently marked as experimental in SRT and is not something that I have enabled in my usages so far. The guidance for SRT is to set your latency window to be 4x Round Trip Time to give enough time for multiple retransmits on lossy connections. You can set it lower, but you are increasing the likelihood that you have missed packets. |
Beta Was this translation helpful? Give feedback.
-
SRT does support both: ARQ (Automatic Repeat reQuest) and FEC, either exclusive or even in combination.
more information you can find here: https://github.com/Haivision/srt/blob/master/docs/packet-filtering-and-fec.md |
Beta Was this translation helpful? Give feedback.
-
|
There's a lot to unpack here, and I am still digesting the issues brought up by the OBS team & the current situation of the industry, so I'm not quite ready to give my opinion yet. I do have a question about the timeline though so we can make sure that we haven't wasted the last 6 months of complete volunteer development time on Glimesh :).
Glimesh is not quite launched yet, we are close but not close enough to call it yet. The very short February 1st timeline puts real stress on rushing through the remaining work on the project to try and ensure it get's a fighting chance. As I'm sure you know just how popular OBS is for streaming users, launching without first party support for it would be unfortunate. My questions are:
|
Beta Was this translation helpful? Give feedback.
-
We'll be on the lookout for your submission!
It's interesting to me that the reliability of the protocol was mentioned as a feature, because if there's one main takeaway I've had from this discussion so far, it's that FTL's dedication to low latency is a double-edged sword whose other edge is low reliability. From @NCommander's post:
Indeed, this low reliability seems to be the thing that gave FTL its reputation among our crew of OBS support volunteers as "buggy", as many users' frame dropping problems would be solved by them switching to RTMP on the same network. @NCommander described FTL as "proven" in terms of its performance. As someone on the service side putting FTL through its paces, what has been your experience while testing FTL in suboptimal network connections, as might be found among a general user populace? I'd also like to suggest another possibility in terms of the timetable. What if, instead of removing FTL on January 1st, 2022, we committed to implementing WebRTC output that allowed people to create plugins for their SDP exchange method of choice (WISH/Janus/Mediasoup/etc.), and then removed FTL an appropriate number of months after that was released? |
Beta Was this translation helpful? Give feedback.
-
|
Hi @dodgepong, Hopefully you've had a chance to read the engineering side of the Glimesh context I posted below - I think it provides some good insight into why we ended up here.
This matches my understanding after spending the past 6+ months implementing an FTL service. If you are broadcasting from a connection that's especially susceptible to packets being dropped or delayed, your stream will suffer as a result of the focus on latency. To address this long-term, we've considered allowing ingest over alternate protocols for those who have less stable connections and are willing to sacrifice latency for stream stability.
I think that saying FTL is "proven" is appropriate... with the added disclaimer that low-latency streaming requires a reasonably reliable connection. In our limited experience using FTL normal amounts of packet loss around 2-5% are tolerable (FTL can nack packets and WebRTC clients have a jitter buffer that can adjust for some delay), but above that and you start to experience regular interruptions in your stream. I think there may have been a lack of messaging around this 'disclaimer' that resulted in a lot of people expecting more out of FTL than it could promise to deliver, and bothering the wrong folks when it failed to. I'm not married to any particular protocol to achieve low latency, though, and I think my post below expresses why FTL happened to be that choice right now.
This sounds like a reasonable direction to me! Ideally for us, the timeline for deprecation of FTL would stretch until the alternative (WebRTC sounds like the leading contender) is made generally available on the client and adequate time has been given for services like ours to migrate and validate. I am a little bit nervous about the prospect of vendor-specific signaling plugins being required, since that presents a new hurdle for users of WebRTC-enabled services needing to play mix-and-match with plugins on their client. I hope that sooner rather than later a true standard signaling protocol will finally emerge and that can be an in-box plugin for OBS. Ideally that requirement would be met before FTL is deprecated so Glimesh won't again have to place bets on protocols with uncertain futures. Thanks again for helping facilitate this discussion! Really appreciate your time and dedication to ensuring that your smaller service partners have a voice here. |
Beta Was this translation helpful? Give feedback.
-
|
Hi there @haydenmc,
I did read it and appreciated the insight, and I apologize for not responding to it directly. My line of questioning was trying to determine what technical features of FTL drew you to it over alternatives, and your post more or less sounded like what I expected -- that is, you didn't choose FTL because it had specific intrinsic features that you wanted, so much as the fact that it was the lowest-latency protocol supported by OBS, and that a streaming service could successfully leverage the protocol (at least, to the level of success that Mixer achieved despite the protocol's technical shortcomings, shutdown notwithstanding). In other words, it was more of a practical decision than a technical one. Is that a fair assessment?
I think this is a smart idea at least as a safety net (presuming you're referring to RTMP). OBS is a free program, which means it has a very low barrier to entry. People from all walks of life use the program, from all countries and all social classes. The population of people who want to use these services but can't afford or even get access to optimal internet conditions is larger than many of us in the tech world may realize.
I think both sides were at fault when it came to supporting FTL with Mixer -- we didn't do enough to warn users about the network quality prerequisites, nor did Mixer. On the OBS side, we have quite a bit of work to do to improve our onboarding process. I think we also could have had better technical cooperation with Mixer, too, in terms of getting help troubleshooting users' strange FTL behaviors, as well as helping us understand how the protocol differs from RTMP so that we could more intelligently diagnose and advise users who were experiencing problems.
I'm glad that's a suitable compromise for you. I hope I haven't come across as overly authoritarian during this discussion -- I did start my post stating that at least one purpose of this discussion was to see if the deprecation proposal and timetable were reasonable, and I hope people have recognized that we are open to debate and negotiation on this subject. I've learned a lot over the last few days. I was hesitant to mention the possibility of waiting for a suitable replacement in the OP because, to be frank, I'm not sure when that will happen. I think WebRTC is definitely the next protocol we want to work on supporting, but I can't guess when that support will actually arrive, practically speaking. That said, I think this discussion has helped us understand that the feature should probably be moved up in priority, and it's something we've discussed extensively already over the last few months. We're putting together some plans now, but we have no promises on a ship date. Due to that uncertainty, we wanted to get the FTL deprecation discussion going sooner rather than later to dissuade any more services from investing too heavily into FTL in the event we do indeed decide to go ahead with the deprecation.
My understanding is that WISH (WHIP? whatever) is intending to become that standard signaling protocol, which is why we are keeping an eye on it. That said, I think it's reasonable for us to start by shipping built-in SDP plugins for the most popular WebRTC servers (Janus, Mediasoup, Millicast, Wowza, Evercast, others? I don't have a comprehensive list), and I wouldn't be surprised if there's a wider body of developers who would be interested in contributing to that effort (thinking specifically about @agouaillard-cosmo as well as the folks at Caffeine such as @patjones).
If we were to change the deprecation schedule such that FTL is removed only after X months have passed of an alternative being shipped with OBS, then the idea would be that we need to set X to be a value such that services currently using FTL as a primary ingest can safely experiment with and deploy with the alternative before FTL is actually gone. My current gut instinct for the value of X is anywhere from 6-12. Also, I think this is another good reason you might want to support RTMP as a backup protocol. |
Beta Was this translation helpful? Give feedback.
-
|
I've updated the calendar in the OP based on this discussion. However, that shouldn't be interpreted as dismissive of the arguments against deprecation FTL to begin with, just an update based on this discussion. Retraction of our deprecation plans is still on the table. |
Beta Was this translation helpful? Give feedback.
-
|
Apologies for my delayed response -
I think that's a fair assessment - at the same time I don't have too many technical complaints about FTL (it was pretty simple to implement, and serves its purpose pretty well with given disclaimers above). Had WebRTC with WHIP been supported at the time, that may well have ended up being our choice instead, given it's positioned better for broader support.
Not at all. I'm very appreciative of your openness to our perspective, and I think the new timeline meets our needs well.
We'll be keeping an eye on it as well, and will look to support the end result after we wind up shipping. Thanks for the timeline update - I think this meets the needs of Glimesh, and thanks once again for hearing us out! |
Beta Was this translation helpful? Give feedback.
-
|
So, here's my response, written w/ input from those in the FTL dev group; apologize for the length but I wanted to cover all points here. So, talking from a historical context, FTL's entire design goal was to get sub second (ideally 500ms or less, real world, we got to 200-250ms) from streamer to the browser under "good" to ideal Internet conditions without add-on software. To that end, it was specifically designed to skip a muxing step, allow for easy implementation of a O(1) hard real time ingest (a naive FTL ingest can be implemented simply by modifying a few bits of each RTP packet in transit). The entire intent was to implement like Stadia or xCloud in browser, and it was overwhelmingly successful in that role. While Microsoft didn't push Beam/Mixer Plays, the FTL technology has a six year history of doing exactly what it was designed to do. Creating a new streaming protocol was not done lightly at Mixer, and frankly, none of the alternatives promise to hit the real time concerns of "in browser" streaming that FTL can, and I'm not convinced that even in ideal situations they will. The primary difference between WebRTC, RIST, and HST vs. FTL is that FTL is designed to lose packets and intentionally does not give any notion of reliable packet delivery. While there is a small retransmission buffer built into ftl-sdk (which was implemented after I stepped away from FTL), it's (as far as I can tell) primarily to smooth out H.264 issues. From the RIST specification, the re-transmission effect is much larger, and doesn't appear to have an upper bound on how far it can queue. The entire intent appears to re-implement TCP/IP reliability onto of RTP. While this will give good performance in good conditions and be manageable in bad, it creates a problem if you need to stay in realtime. With FTL, it essentially YOLOs each packet (and this plays into the last mile, but I'll get to that). RIST on the other hand is specifically designed as reliable and has a large middleware level to allow resumption in case of stream interruption. While RIST is indeed built on RTP and SRTP, it doesn't specifically promise a latency goal. While it might be possible to get the same low latency performance over RIST, it's not a design goal of the protocol, and TBH, I'm less than fond of tossing out the "known to work" solution for one that is not proven. RIST’s retransmission facilities are also not really designed for the notion of synchronization. The client can request transmission via block or specific packets, but not to a specific KF interval like can be done with H.264/FTL streaming. While retransmission is technically optional, it does exist in FTL today for a specific purpose. The original VP8/Opus FTL implementation actually didn’t have any re-transmission facilities, and could hit 200-250ms latency real world. I do intend to look at reviving this functionality. In a RIST world, you send NAKs to get the packet retransmitted and you have a fairly large state machine on-top of that, which basically is TCP/IP implemented onto of UDP with RTP. Furthermore, RIST still doesn’t handle signaling of information like video metadata and similar which is a specific feature that Charon (FTL’s handshake) implements. While RIST might be a replacement for FTL at some day, it doesn’t actually provide anything over the current implementation, and still requires additional infrastructure to be used. In short, actually adopting RIST doesn’t solve a problem. This brings me to the topic of WISH/WHIP and WebRTC. To be frank, I’m not certain why WISH or WHIP is being brought up; given that we have two separate specifications that try to solve WebRTC signaling, it’s pretty clear this isn’t a solved problem. Before I dig into either, I do want to talk a bit about WebRTC’s own signaling mechanism: the Session Description Protocol (SDP). SDP underpins all WebRTC calls, and is also used in VOIP. It’s a nightmarishly complex handshake, with a very loosely defined standard. When I looked at doing FTL originally, I did actually look at the possibility of building the signaling information into SDP, and decided that the best solution was to run as fast as I can. WISH and WHIP both appear to be extensions of SDP which means they inherit all the problems with SDP. While it is a slightly old article, this sums up the pain points pretty well: https://webrtchacks.com/webrtc-sdp-inaki-baz-castillo/. I highly recommend going through WebRTC Hack’s section about SDP before advocating any solution built around it. I’ll start by talking about WISH first: the IETF specification is in the Internet-Draft stage, and still in the Proposed WG state: https://datatracker.ietf.org/wg/wish/about/. I’m not even certain there’s a reference implementation of the specification. Furthermore, WISH is a direct extension of the SDP protocol which means all the pain points are primarily there. SDP has a fiendishly complex protocol negotiation because its essentially intended to make two VoIP phones from different manufacturers agree on some sorta standard to talk with each other. WebRTC not only uses this for talking to browsers, but adds a very complicated stack of ICE connectivity. Even though things like STUN have gone to the wayside, it’s pretty hard to say that this is a desirable thing. In contrast, if you can make TCP and UDP connections from point to point, you can stream with FTL because its designed to go in one direction and only has an “accept/deny” state vs. offer/accept machinery. In effect, to adopt WISH into OBS would mean integrating a full WebRTC stack, signaling layer, and more to accomplish what FTL does with two RTP streams, a signaling protocol, and some basic invariants. There’s also the notion that both RIST and WISH are either new, or still being drafted. While both have the advantage of being an open standard from the get-go, they have no field test history, and neither are trying to hit the original sub second latency goal of the original FTL protocol. This brings me to SRT. Unlikely RIST or WISH, SRT is specifically designed as a low latency solution. However, SRT has two fundamental problems which make it unsuitable for use on “streamer->web browser pipeline”. The first is that SRT is not wire compatible with RTP nor does it use the same encoding standard. Part of the trick of FTL is that the packet generated by OBS is in fact the same packet sent to the browser with the least amount of post-processing done in the middle. Based off the benchmarks I did at Beam, each time we mux, demux, or re-encode a packet, we pick up a significant lag since in most cases, you need to have a fully assembled keyframe set to re-encode from location to location. FTL specifically bypasses that requirement by putting out something that WebRTC (with slight modifications) can digest directly. However, that’s not the only problem with SRT. SRT lacks a signaling mechanism. FTL’s Charon protocol acts as a signal for its two RTP streams, and can send metadata, signals keep alive, and could be potentially extended if ever needed. SRT merely defines a wire protocol and doesn’t specify any inband signaling. In short, you’d still need something like FTL’s Charon protocol to use SRT, and unless browsers adopt native SRT support, you still won’t be able to hit the same latency benchmarks you can hit with FTL. There’s also a final thing to consider. At the heart of this discussion, we’re talking about ripping out a proven technology for something that only exists on paper. While theoretically, you could build low latency livestreaming solutions on several of the technologies listed here, no one has actually done it, and proved it works in the real world. FTL has, and it did so for more than half a decade. There’s something to be said for a proven solution. It is true that under Mixer, FTL wasn’t an open protocol, and the only implementation was Mixer itself. This was not the intent of FTL. FTL was intended as an open standard, and most of my reference notes were posted in the FTL-SDK. MSFT removed them, but they’re still in the git history. Furthermore, multiple people have reimplemented the server side from scratch using those notes, and the “last mile” part is easily implemented using the Janus gateway which was the same software the first FTL implementations were built on. Furthermore, the FTL RTP streams are essentially designed that the packet coming out of OBS is the packet that will go into WebRTC. The FTL implementation we used only had to change the SSRC in flight, and later ones had to add SRTP when it became mandated. Most of the work of the FTL standards effort is to essentially create a canonical version since there are some server behaviors that can’t be defined from the client, but on the whole, FTL is well understood, it’s documented (and getting bettered), and we’ll likely either code or canonicalize an existing implementation as the de facto reference implementation, and then resume development to add new features and to bring FTL out of the pit it was left in. In short, FTL was and is the only standard that is specifically targeting latency as it’s number 1 concern. Neither HST, RIST, or WISH can claim that. Furthermore, I need to ask a question: what is the ongoing cost to keep this code in OBS? ftl-sdk isn’t what I call a fast moving target, and most of it hasn’t changed in a long time. |
Beta Was this translation helpful? Give feedback.
-
|
I appreciate the anecdote, but compared to the problems that we repeatedly saw in our support channels, your case sounds like an outlier. As @dori4n said above:
Perhaps "intended to work" reaches too far, here. What about "the protocol is more sensitive to poor network conditions"? |
Beta Was this translation helpful? Give feedback.
-
|
WISH is aworking group, WHIP is a suggested protocol that exists as a draft. |
Beta Was this translation helpful? Give feedback.
-
I was under the impression that the name changed from WHIP to WISH to avoid an existing IETF name collision. |
Beta Was this translation helpful? Give feedback.
-
FTL connects to a TCP port to negotiate a UDP port at the ingest to use to send RTP frames to. To some internet service providers, who make use of network filtering equipment (most prominently variants of Sandvine NIS) to block or restrict P2P file sharing (hi Comcast!, I guess it's Xfinity now...), this type of data exchange looks like traffic to either block entirely or limit the packet forwarding rate for (and this was the leading cause of problems for people using FTL on Beam/Mixer!) where users tested bandwidth availability (even to the same IP endpoint!) would then not match what bandwidth is available to FTL's RTP-over-UDP transport. How ISPs behave here is often very unpredictable. If these UDP packets are delayed or dropped such that they can no longer arrive on time for the ingest server to assemble a frame on time and send it to a viewer, playback for a viewer can either not start at all for lack of a keyframe, or the video would constantly freeze as a frame is lost or incomplete, while the player waits to receive a new keyframe. If the network conditions are such that keyframes can never be delivered (they are much larger than p-frames) or at least one packet of a p-frame is consistently lost (and not merely delivered out of order into a buffer) at a rate equal or higher than the keyframe interval, then smooth playback cannot be achieved. |
Beta Was this translation helpful? Give feedback.
-
|
So, with Mixer, at least the original FTL implementations transcoded the incoming FTL stream to RTMP, and downgraded the user to RTMP if their Internet connection was loosing too many packets. I don't know if that feature survived in the MSFT issue, but it was very dependent on the streamers ability to keep sending packets at a high clip. Back in the Tachyon days, there was a very large warning that you needed good Internet to stream FTL to Beam. I can see that MSFT added some provisions for a media test but it doesn't look like they use it anywhere. That sort of test (if implemented properly) would probably stop bad bugs from coming into OBS, and its something we could add to FTL without breaking backwards compatibility with the existing clients/codebase. |
Beta Was this translation helpful? Give feedback.
-
|
Howdy all, Once again appreciate your time and the opportunity to discuss this in an open forum! Happy to answer some of your questions so hopefully you can understand our history of decision making and ultimately our perspective w.r.t. FTL. The answers I provide are from the perspective of janus-ftl-plugin, which is the technology currently powering Glimesh.tv. The Glimesh.tv project started around July with the demise of Mixer. @clone1018 as the founder of Glimesh can probably provide more context on the relationship between Glimesh and the end of Mixer - but a substantial part of the motivation was to provide a new option for true low-latency (sub-second, and ideally way lower) streaming, as the Mixer shutdown left a void in this space. At the time, SRT support in OBS had only recently been released a few months prior in mid March. At the time (and arguably still today), SRT in OBS was still a nascent technology - with no SRT services represented in services.json, and no broadly available consumer-facing streaming platforms with SRT support that I could find with a quick search. As a new platform with many uphill challenges ahead of us, it made the most sense to leverage an existing protocol that had a proven track-record at scale, and what seemed at the time to be solid client support. Plus we knew with confidence from Mixer's example that we would be able to meet the low-latency expectation we set out to achieve. Knowing what I know now about the perspective of the OBS team and the alternatives available, what would I choose today? My perspective here is that a lot of these new protocols present a substantial risk. I don't have a whole ton of technical concerns - at the core of the issue, I just need to get a bunch of packets from OBS to my service as quickly as I can - I'm not super nitpicky over how that happens, though niceties like nack/retransmit and forward-error-correction are probably best for a good experience. My biggest concerns are about the politics and shifting industry focus on "what's next." Right now I see three protocols on the table - SRT, RIST, and some form of WebRTC. SRT is the only choice with support in OBS available today, making that our only alternative available right now. With Glimesh going live soon, that leaves us with the uncomfortable choice of either choosing to support a protocol deprecated on the client-side, or choosing to be one of the first guinea pigs to implement SRT for a large-scale consumer-facing streaming platform. In general I don't mind being a guinea pig and pushing forward with newer, potentially better technologies - but when I am shipping a product to a customer (a non-technical consumer, no less), a solid, proven foundation to build that experience on is the first priority, and currently that foundation is FTL. Long term maybe the answer is SRT/RIST/WebRTC/etc - but I feel the best approach to reach that future is a gradual transition to these new technologies, migrating off of FTL when we can say with confidence that the new technologies can meet or exceed the existing bar for a good customer experience. Betting entirely on SRT now would mean we open ourselves up to the possibility that the industry decides to move in a different direction later on - we could spend a few months adding SRT support only to discover that next year it's being deprecated in favor of RIST or WebRTC. With FTL as our backup, we have the ability to experiment with the new nascent protocols without being burned if they don't work out - then once we have some indication that the new protocol has met our expectations and gained enough traction in industry, we can fully commit and pull the plug on the old stuff. I hope that answers the questions you had, please let me know if there are any areas I can clarify! Thanks once again for your time and the opportunity to discuss this. I know that dealing with issues like this are burdensome in both time and mental energy, so I really appreciate it! |
Beta Was this translation helpful? Give feedback.
-
|
SRT has large and growing support in the professional video broadcast industry in both hardware and software encoders/decoders. SRT is not currently a good choice for a transport between a server and clients, but I believe is a good option for OBS to server transports. Currently there are some deficiencies in using SRT with OBS:
SRT is a basic messaging protocol, with a convention of sending MPEG-TS over it, but it can be used to send anything. It would be easy to find or create a standard for running speedtests and latency tests (or establishing a stream and reading the latency stats from the SRT socket itself) and using that to provide good defaults for latency. The max bandwidth setting can be derived from the bitrate specified on the encoder for the audio and video streams. With this ability to measure appropriate SRT latency settings, it would provide fairly low latency (200-300ms in the US if there are POPs on both coasts). |
Beta Was this translation helpful? Give feedback.
-
|
The problem is that the SRT media packets and the RTP one differs and that appears to extend to how frame information is encoded. That means you'd have to convert one to the other at a CPU and latency hit. I don't know how different the two are, but most off the shelf implementations like FFMPEG usually require a full KF interval to transcode across protocols (which was our experience in converting FTL->RTMP). We're using RTP because that's what WebRTC uses to avoid a transcoding, muxing or demuxing step. I don't deny SRT would work great for low latency video ingest, but in this case, it has to stay low latency to the last mile. It also doesn't define a specific signaling layer (neither does RTP, but FTL provides that role). |
Beta Was this translation helpful? Give feedback.
-
|
SRT header is 16 bytes, RTP header is (at minimum) 12 bytes - more if extension is used.. I've never dealt with video over RTP before I work in call centres so only ever voice and I've only ever seen 12 bytes.. So with the RTP header removed, how large is your packet? Can you add 16 bytes to it and still fit into the MTU? Edit: or do you need to keep the RTP header? If so, then you would need to accomodate the extra 16 bytes |
Beta Was this translation helpful? Give feedback.
-
|
I'd like to give a real-world example that clearly validates WebRTC viability as more widely used standard instead of FTL. Google Stadia uses WebRTC for delivering real-time audio/video in production today. I do not think any kind of consumer live streaming needs exceed strict requirements of gaming on a remote machine. Signaling is a concern for many with WebRTC, but with WebRTC stack in OBS it shouldn't be a big deal to swap one signaling with the other or support multiple for some time or in general (for instance one based on SDP and another on ORTC types in JSON). WHIP draft should be more than enough to get things going. With that in mind and the fact that the WebRTC is ultimately being used with SFU to deliver content to viewers, WebRTC will be present in the stack regardless, so having FTL makes people have to support 2 protocols and create bridges between them instead of just one and using off-the-shelf SFUs with little to no modifications. |
Beta Was this translation helpful? Give feedback.
-
|
Hi @nazar-pc, and thanks for posting. I noticed that you work with Restream.io, which had a history of supporting FTL until Mixer died. Can you comment on your experience with the FTL protocol at Restream, or is that more of a question for @Dead133? |
Beta Was this translation helpful? Give feedback.
-
|
We have similar impression to OBS feedback above. It is great when it works, but with flaky connectivity users were disappointed with quality and there wasn't much our support team could help them with except decreasing bitrate or using RTMP. And in many cases switching to RTMP resulted in 1-2 seconds latency on Mixer that I don't remember being a deal-breaker for most users, Mixer worked really well even with RTMP. |
Beta Was this translation helpful? Give feedback.
-
|
Thanks for the feedback. In your post above, you suggest that WebRTC might be a better alternative to FTL for the low latency use case. Despite that, looking at the merits of FTL on its own, do you think it ought to be deprecated, or do you think support for the protocol should remain present in OBS? Why or why not? (Note that I am not asking if you like the protocol, I am asking if you think its merits are deserving of continued support in OBS) I imagine that might be a complicated question for someone who works at Restream, since obviously Restream is motivated to support whatever protocols are available from the services they support. So if Restream end up supporting a service like Glimesh that uses FTL, then Restream would also likely want users to have access to FTL in their encoder of choice. |
Beta Was this translation helpful? Give feedback.
-
|
We've actually un-shipped FTL ingesting support a few months after Mixer shutdown and don't plan to bring it back. Support for protocol needs significantly sized platforms that we have integration/partnership with that support FTL and/or there is a significant demand for it to justify support (we have a small team as well). Currently we see neither of those 2 options. So at Restream we don't really care if FTL is supported by OBS or not at this point. I do see use cases for WebRTC though and would like to invest my (even spare) time into adding support for it if/when upstream OBS has it. Suggested deprecation makes sense to me with maintainers resources being as limited as they are. Services using FTL today should be able to migrate away given enough time and comparable alternatives readily available. With OBS being a major encoder it causes developers with limited resources to chose from already available protocols, I can understand that and would probably do the same given such circumstances. Though I don't see how it enables use cases that are strictly not possible with any other protocol if not being limited with what OBS supports today. |
Beta Was this translation helpful? Give feedback.
-
|
This is a very long discussion already, I just wanted to make a few bullet points:
|
Beta Was this translation helpful? Give feedback.
-
|
30 fps: 12 - 3 = 9 frames * 33.3333 ms/frame = 300 ms end-to-end latency |
Beta Was this translation helpful? Give feedback.
-
|
@SCG82 it seems wowza is not enabled in the released builds of obs-studio-webrtc ? I only see Millicast and custom webrtc. I'd be interested in testing to wowza (seems you authored the wowza output code) |
Beta Was this translation helpful? Give feedback.
-
|
Beta Was this translation helpful? Give feedback.
-
|
patch wowza servers ? lol |
Beta Was this translation helpful? Give feedback.
-
|
We believe wowza is aware of the issues. There are doing a U-turn after having so heavily invested in SRT and denied any value to webrtc for years, so it takes time:
The recent leave of both their original VP and chief architect and their original founder and CTO (who still remain as a board member) (e.g. https://www.linkedin.com/in/scottkellicker/) might make it more difficult to engage in new strategic technical ventures. WHIP is very new, just passed IETF dispatch approval in December. It will only have it first BOF meeting at the next IETF meeting in march, publicly open for the first time. Activity has not started. Everybody will be welcome into the discussion and the decision process. You just need to be able to hum (https://tools.ietf.org/html/rfc7282). |
Beta Was this translation helpful? Give feedback.
-
|
So, I'm going to sum up things as I read them right now, as of right now, there are multiple solutions that aren't FTL that can get subsecond latency, and some might do a better job of it. However, for something like obs-webrtc, I'm still not seeing a specification that can handle the authentication step like you have in RTMP and in FTL. Maybe I've overlooked it. I suppose you could implement a 401 HTTP authentication step, or add something to the SDP handshake in WHIP, but that doesn't appear to exist as implemented or defined right now. No one is disagreeing that WebRTC does infact work for the last mile, but to actually get low latency from the streamer to the viewer, you need to emit a packet stream very close to what WebRTC will take (SRTP wrapping being the exception). Otherwise you have to introduce a transcoding step, and usually to transcode, you need a full KF interval. That essentially makes SRT and other non-RTP wire protocols a non-starter unless/until WebRTC can support a non-RTP based stream. I will note that obs-webrtc will likely suffer the same sort of packet dropouts FTL will since it basically works on the same mechanism of transmitting RTP streams that FTL does. |
Beta Was this translation helpful? Give feedback.
-
|
Er, one thing I do need to add, I'm not entirely sure you could easily route WHIP across a backend infrastructure. While FTL was intended to use SRTP, it was only in an authentication role (ultimately, FTL used source based identification). WHIP uses DTLS, which would require an additional decrypt stage before routing from the ingest daemon to a Janus server since in any real world deployment, those two would almost certainly be disconnected as they were at Beam/Mixer. I'm not sure how much of an impact on latency that would have, but I can see it making implementation of an ingest daemon more complicated. |
Beta Was this translation helpful? Give feedback.
-
|
@NCommander as shown in my previous message, I think there are a lot of your statements and concerns that could be addressed by a dive in the documentation / specification draft / implementation. I want to keep this thread short, so I stay at your disposal to discuss in as many detail as you would want your concerns off thread anytime you see fit. Please note that the millicast platform which handles up to 2 millions concurrent viewers for a given single source, uses WHIP, with the usual HTTP token for authentification and access control. Tencent LEB (Live Event Broadcast) whose volume is two order of magnitude higher, also uses WHIP for ingest. A few bullets point again to illustrate my point:
Happy hacking. (1) - "The key strength issue could be resolved by going to 2048-bit RSA keys or more, but that would delay call setup by several additional seconds. Instead of changing the RSA key size, Chrome 52 implements ECDSA keys (Elliptic Curve Digital Signature Algorithm) for use in certificates. These are as strong as 3072-bit RSA keys‚ but several thousand times faster: call setup overhead with ECDSA is just a few milliseconds." |
Beta Was this translation helpful? Give feedback.
-
This is fine (and not actually that long, either, it's in low three digit millisecond territory), since this is just for establishing a transport connection. You don't wait and queue video while this happens. You start sending from real-time as soon as you've got your symmetric key negotiated. |
Beta Was this translation helpful? Give feedback.
-
|
One thing which is on the plus side for FTL against webrtc is that it can use our obs encoders, including hardware acceleration e.g. nvenc, quicksync, amf. It doesn't seem the case with obs-webrtc (dr alex fork of obs):
Regarding (1), I found this project though: https://github.com/sonysuqin/WebRTCOBSEncoder ; it'd be interesting to be able to have the same thing for obs-webrtc. |
Beta Was this translation helpful? Give feedback.
-
|
Nothing in WebRTC spec prevents usage of hardware encoders. |
Beta Was this translation helpful? Give feedback.
-
i'm aware; that's not my point. Regarding the current implementation of webrtc (so obs-studio-webrtc of Dr Alex) or the future implementation in obs, imo it'd be a very good thing to be able to leverage our encoders. If I read well the code of Dr Alex fork, the webrtc outputs use raw frames; it'd be convenient if we could pass them our encoded data. |
Beta Was this translation helpful? Give feedback.
-
|
@pkviet This is an implementation detail. This is not a protocol discussion anymore. wrt hardware acceleration, in OBS:
Why AV1 (or VP9 mode 2), because of the support for HDR, and up to 12 bits, 4:4:4 in webrtc with it. the risk of using the same approach as the korean one, is to cut the RTP feedback loop, losing NACK, FEC, and all the other adaptability and resilience mechanisms while doing it. It would be better to inject the encoder in the media engine. |
Beta Was this translation helpful? Give feedback.
-
korean ? |
Beta Was this translation helpful? Give feedback.
-
ah very cool
excellent
i was wondering if we could leverage it; that's an interesting idea. |
Beta Was this translation helpful? Give feedback.
-
|
I am happy to have in depth discussion about any of your requirements anytime you guys want. |
Beta Was this translation helpful? Give feedback.
-
|
Slight correction. Our fork https://github.com/MyFreeCams/Sidekick/blob/master/ObsBroadcast/X264Encoder.cpp uses x264 directly, not the output of OBS' x264 encoder (for OBS' x264 encoder to be used, a few parameters need to be changed for WebRTC: b_filler = 0, analyse.i_weighted_pred = 0). |
Beta Was this translation helpful? Give feedback.
-
|
https://github.com/sonysuqin/WebRTCOBSEncoder |
Beta Was this translation helpful? Give feedback.
-
|
thanks for the wealth of info @agouaillard-cosmo |
Beta Was this translation helpful? Give feedback.
-
|
By the way @agouaillard-cosmo you can click in the "Write a reply" box beneath a thread to reply to just that post if you like, rather than make a new thread in the discussion at the bottom of the page. Might help keep the conversation more organized. (Discussions are a new feature of Github that we are still testing out.) |
Beta Was this translation helpful? Give feedback.
-
|
I apologize, i m new to this and on mobile the visual difference is less obvious than on desktop. I will be carefull. |
Beta Was this translation helpful? Give feedback.
-
|
So after looking through the posts i have a question about what issues can arise with how WebRTC works in browsers and if these could be potential if not already existing issues with some of the implementations of WebRTC in obs that exists as a fork. |
Beta Was this translation helpful? Give feedback.
-
|
For client to server use cases (like with OBS) you can avoid sending any IP addresses because server is reachable directly, so nothing leaks beyond IP address from which you do the request anyway. Also there is mDNS even when browser is used in a more traditional way. |
Beta Was this translation helpful? Give feedback.
-
|
Let's imaging you were leaking your IP in OBS (whatever the protocol). It would only be "leaked" to the ingest node of the platform you are streaming to, which is very likely to know who you are already through authentication. RTMP, FTL, and many other protocol do not make any attempt at hiding the origin IP of the stream. Content verification, age verification, ... there are many reasons why anonymous streaming to a platform is at best a bad idea and at at most illegal from the platform point of view. If you are trying to hide your IP to the platform you are streaming to, you might be in breach of their EULA. Now, on the other side, let's imaging that the last mile delivery protocol used by the platform was leaking IP. It would only leak to the viewer the IP of the egress node of the platform, not of the original source of the stream. As a matter of fact, many CDNs use webrtc's Datachannel based solutions to smooth last mile delivery (peer5, ...) |
Beta Was this translation helpful? Give feedback.
-
|
Ok so if i get the gist of it correctly the only places that would technically get the IP of the streamer is the OBS client and the ingest server no matter what is happening on the ingest server or the OBS client and in terms of streaming that would be acceptable as that is 100% necessary for an actual stream to be live. |
Beta Was this translation helpful? Give feedback.
-
|
Hi all, I have greatly appreciated the conversation that has taken place here over the last several days. I have learned a lot! I'd like to summarize my current understanding of the situation and my thoughts on how we might proceed regarding this issue. First is the question of what FTL brings to the table that is unique relative to other, more widely-used protocols. I don't actually think the answer is that it can provide lower latency than the alternatives, as it's clear WebRTC can achieve similar latency by disabling reliability features. Instead, it seems to me that the main appeal of FTL is its simplicity, particularly on the side of the service provider. WebRTC can do lots of things, and as such, it brings a lot of baggage. FTL, on the other hand, was developed for a specific use case (multiple simultaneous ingest of ad-hoc user-generated live video content), and as such it is able to remove a lot of WebRTC's cruft. FTL's downside (and the OBS Project's main objection to it) is that, in pursuit of absolute low latency, FTL has been designed without mechanisms that would improve stream stability for users streaming under poor network conditions. This has historically led to an increased burden of support placed on OBS volunteers, and more generally leads to poor user experiences when FTL's network requirements are not properly communicated or detected. In my opinion, the advantage of FTL's simplicity on the part of the service provider does not matter a whole lot to users in practice. My impression is that users don't generally care what protocol they are using, they just want their stream to work as advertised. FTL correctly advertises extremely low latency (especially in the marketing materials of services that leverage it) but this advertisement frequently lacks the caveat about the necessary network reliability. Here are my current thoughts on the path forward:
A few thoughts on things that can be done that should help all parties during this time:
I welcome further thoughts and discussion about this, including from other members of the OBS Project besides myself. |
Beta Was this translation helpful? Give feedback.
-
|
I think one of the major points for keeping FTL support for now, is that it is currently the only free open-source solution for getting low-latency content from OBS to an in-browser player, as far as I am aware. That is Project Lightspeed. While I can get behind the pros and cons of FTL vs WebRTC on paper, it is a moot discussion if users don't actually have the option to choose one over the other in practice. The current situation is that if FTL support is removed from OBS at this point, that essentially removes the option of streaming low-latency content to a browser without migrating to a paid subscription service; a major hurdle for most people who just do this as a hobby in their free time. |
Beta Was this translation helpful? Give feedback.
-
|
Hello everybody Although this is already a long thread, I would like to jump on this discussion and highlight some bullet points for SRT and raise some interest for a native SRT integration into OBS, rather than using the ffmpeg SRT integration.
Currently using SRT in OBS is still a bit clumsy, since it's ffmpeg based and all parameters have to be typed into a command line. A native SRT integration into OBS could bring a lot of benefits, not just ease of use.
If there is interest on learning more details on these features, I can organise a Webinar about SRT for OBS developers and explain these feature in a bit more detail and also answer questions. with best regards, |
Beta Was this translation helpful? Give feedback.
-
|
Thanks for the reply. Discussions around improving OBS's SRT support would probably be better suited to an RFC so that we can keep this thread on the topic of FTL. |
Beta Was this translation helpful? Give feedback.
-
|
@J-Rogmann thanks for the offer of a webinar. I'd be interested. I can be reached on obs discord. |
Beta Was this translation helpful? Give feedback.
-
|
Today is March 1st, which means we are now putting a hold on accepting new service submissions from FTL-based services or investing time into FTL output maintenance. We will continue to monitor the status of the FTL protocol in context of industry usage and demand, and will continue working toward adding WebRTC output support in OBS as the next major supported protocol. |
Beta Was this translation helpful? Give feedback.
-
|
Hi @dodgepong Is there any timeline about WebRTC output support? |
Beta Was this translation helpful? Give feedback.
-
|
Q3 2022 is our new target for an MVP of WebRTC output. |
Beta Was this translation helpful? Give feedback.
-
|
@dodgepong We are also using FTL but willing to move on to WebRTC output, just hope there's a grace period where FTL and WebRTC output are both available. Is there an active development branch somewhere to test? |
Beta Was this translation helpful? Give feedback.
-
|
There will be a grace period, yes. Most development related to WebRTC at the moment is being handled by @DDRBoxman. Our updated timeline for a minimum of WHIP-based WebRTC support is looking closer to Q3 2022. |
Beta Was this translation helpful? Give feedback.
-
|
Hi @aet I would love to help you with your migration. What resources can I provide to make it easier. |
Beta Was this translation helpful? Give feedback.
-
|
RE: OBS WebRTC support, do we have anymore details about this? Is there an issue to track it? Are you planning on generic WebRTC support or something like WHIP? |
Beta Was this translation helpful? Give feedback.
-
|
I think it was WHIP, we don't have an issue tracking it yet. Paging @DDRBoxman |
Beta Was this translation helpful? Give feedback.
-
|
https://lightspeed.tv is a new service that uses FTL, so it is still being used by some new platforms. |
Beta Was this translation helpful? Give feedback.
-
|
Checking lightspeed.tv, they appear to have maybe a few dozen users, and I'm not really concerned about small services. Our stance hasn't changed where we strongly discourage the adoption and support of FTL at this time, and we will not be accepting new FTL-based services. I've updated the main post to clarify the timetable a bit. The developer on our team who was working towards WebRTC output has been working on other projects that took precedence over this work on OBS, and we are currently looking for someone to take over the work and continue it to completion. So, at this time we no longer have an ETA on when WebRTC support will be available in OBS. Anyone interested in assisting with this work can certainly reach out to us and we can coordinate on it. You can either comment on this discussion, reach out via our Discord, or email me directly: [email protected] |
Beta Was this translation helpful? Give feedback.
-
|
In addition to still intending to adopt WHIP once possible, we are of course also keeping an eye on other developments such as the efforts of the "Media over QUIC" working group at IETF and projects with backing from companies involved in streaming applications such as QuicR, RUSH, Warp, etc. We are truly hoping that options to succeed FTL and RTMP will become available some time before the heat death of the universe. |
Beta Was this translation helpful? Give feedback.
-
|
I'm interested in knowing more about the present state of the work on WebRTC in OBS is and what's involved in completing it. Specifically looking towards addressing potentially limitations or concerns when it comes to replicating the latency reduction inherent from the design of FTL. Would you be able to go into more detail about that? |
Beta Was this translation helpful? Give feedback.
-
|
Hi @modwizcode I am actively working on this. I have a PR here I also created a prototype server here and would love your feedback! If this PR works for you would love a ACK that it meets your needs (or comments on how it does not) ETA wise this work is behind other work that the OBS team has scheduled. @Fenrirthviti might be able to speak to that more. |
Beta Was this translation helpful? Give feedback.
-
|
For those monitoring this thread for news about WebRTC support, here's a prototype/playground for how we're considering going about the process of adding WebRTC support to OBS: #7192 |
Beta Was this translation helpful? Give feedback.
-
|
For others following the PR has been moved here I also created a prototype server here and would love your feedback! If this PR works for you would love a ACK that it meets your needs (or comments on how it does not) |
Beta Was this translation helpful? Give feedback.
-
|
Is the opening post supposed to contain a list of currently supported streaming protocols? It doesn't seem up to date because https://obsproject.com/wiki/Streaming-With-SRT-Or-RIST-Protocols says RIST is supported while this post says it isn't. |
Beta Was this translation helpful? Give feedback.
-
|
Correct, the OP was written before RIST support was implemented in OBS. |
Beta Was this translation helpful? Give feedback.
-
|
I found this thread through google. Is there any other place that documents the supported protocols? I feel like this should be on the main obs website somewhere. |
Beta Was this translation helpful? Give feedback.
-
|
I don't believe there is such a place at this time. As of this writing, OBS currently natively supports RTMP, SRT, RIST, and HLS network streaming outputs, as well as virtualcam, Decklink and AJA outputs natively. There is a plugin that adds support for NDI output. FTL output also still exists in OBS, but it is deprecated. |
Beta Was this translation helpful? Give feedback.
-
|
Now that WebRTC has been merged, we are moving forward with the proposed plan for removal of FTL in OBS. While we understand this step took a lot longer than original estimations, this now marks the last requirement. Removal will be considered in future releases of OBS as the timeline suggests. Current estimates for a public release with WebRTC available are later this year. |
Beta Was this translation helpful? Give feedback.
-
|
I'm excited about WebRTC support being made available! Thank you and congrats to the folks who worked hard to make this happen. Are there details on the timeline for when FTL will be removed? I have a small project or two that make use of it and would like to know how much time I have to migrate to WebRTC. |
Beta Was this translation helpful? Give feedback.
-
|
@haydenmc Anything I can do to help you migrate to WebRTC? |
Beta Was this translation helpful? Give feedback.
-
|
Thank you for the offer! I still need to do a little investigating to see what my options are. |
Beta Was this translation helpful? Give feedback.
-
Per the opening post:
At the current release cadence will likely mean you have until Q1 - maybe Q2 - of 2024. |
Beta Was this translation helpful? Give feedback.
-
|
D'oh, thank you, I missed that! |
Beta Was this translation helpful? Give feedback.
-
|
As a note, our removal may be escalated as issues such as #9105 prevent us from building, and would require either workarounds or custom patches to fix, neither of which is appealing to us. No decision has been made at this time, but those still relying on FTL in the remaining grace period are invited to comment any challenges or roadblocks. |
Beta Was this translation helpful? Give feedback.
-
|
Simple to set up and use WebRTC-based streaming solution by Glimesh developers: https://github.com/Glimesh/broadcast-box |
Beta Was this translation helpful? Give feedback.
-
|
I am heavily biased @elfeiin, but this is my take!
That would be great! It will have to be something a little higher level though. With WebTransport we don't have a URI + StreamKey so will need to be something like MOQT
This isn't true. Companies like Phenix RTS do this today. At Twitch we also had 'WebRTC servers at scale' you can use them today with IVS Stages. At my current job LiveKit the CTO wrote about it here
I always put the signaling server + media server in the same host. What problem are you worried about?
I agree that WebRTC is intimidating. It covers so many more use cases then 'low latency broadcast'. My belief is that a protocol that ONLY covers 'low latency broadcast' (like FTL) won't have enough usage to warrant existence. Check out WebRTC for the Curious it is one attempt to make things easier (via education) I hope that Broadcast Box can make it easier for people that just want to use low latency OBS quickly. If users want to build something themselves WebRTC support exists in most major languages! |
Beta Was this translation helpful? Give feedback.
-
Forgive me for my ignorance, but is path the same as URI? WebTransport requests do include a path parameter. You're right that it's not a media protocol unto itself, but OBS could expose an API where routing info indicates the desired stream types, and OBS either does or doesn't supply them. I don't mean to bikeshed too much, but WT is very versatile.
I'll check this out. Thank you.
I just wanted UDP with good browser integration (as we all have, for years). Instead I got something way overkill for my usecase, which was frustrating. But to answer your question, the simplest implementation I can think of for serving a stream from one client to multiple other clients (broadcast) is to simply open a connection on the host for each client. This would overwhelm the host for large audiences, but that's the default setup for WebRTC. At the time I was working on my project I hadn't thought of chaining WebRTC connections to relay the broadcast or just implement deferred broadcast using WebRTC and relegate the server to a WebRTC client. I guess past me isn't that creative. But I think it's still a valid point that WebRTC is overkill, or at least nontrivial to build atop.
I believe in having options, but I am not a maintainer of OBS. Nor am I a user (yet) but my users will be your users, if I get any. It's frustrating to look at all these technologies that exactly meet my requirement and see that nobody uses them (big LOL). Perhaps I will reach enlightenment the more I explore the implementations. Thank you for your time. |
Beta Was this translation helpful? Give feedback.
-
WebRTC had this same problem pre-WHIP. WebRTC was 'too versatile' and made it hard to use. You would have to add lots of APIs and UIs to make it work.
Totally agree! For your use case WebRTC is an overkill. You can't shave features down because other people depend on those features (and the features important to OBS don't matter to you). I work with users that dislike that WebRTC does audio+video, they just want DataChannels! Join the real-time broadcast discord https://discord.gg/An5jjhNUE3 I would love to talk about this more. Would also love to help you build your idea! |
Beta Was this translation helpful? Give feedback.
-
|
Just to confirm from an official statement, we consider the current inclusion of WHIP/WebRTC output sufficient enough of a replacement for us to remove FTL support. The decision is largely motivated by the lack of maintenance and adoption of FTL, and it isn't worth the project's time to lug it along any further for the tiny handful of people who are still using it. That said, those people are more than welcome to keep using versions of OBS with it included, or keep support in their own builds. I have yet to see a compelling argument for us continuing to support it that doesn't amount to "we don't want to change". We strongly encourage adoption of something that is actively maintained, supported, and adopted (such as WebRTC) over FTL for the proposed use cases. :) |
Beta Was this translation helpful? Give feedback.
-
|
#10019 is now merged, so this is complete. |
Beta Was this translation helpful? Give feedback.
-
|
Thank you all for the open communication and generous timeline for migrating away from FTL. WebRTC/WHIP support is more than adequate of a replacement. Much appreciated! |
Beta Was this translation helpful? Give feedback.
and others
-
Overview
As previously mentioned in #3816, #3834, and #4018, the OBS team has internally discussed deprecating FTL and eventually removing it from OBS. I'd like this discussion to serve as a means of communicating intent and setting expectations, helping us see if our expectations for timing are realistic, evaluate what alternatives the community would like to see put in place the most, and hear out what reasons we might want to reconsider the deprecation of FTL support.
Rationale
We would like to remove FTL support from OBS for the following reasons:
Timetable
At present, our plans and timetable are as follows:
We plan to deprecate FTL on March 1, 2021. Until March 1, 2021, we plan to continue to allow services to add entries using FTL (
"output": "ftl_output",) to theplugins/rtmp-services/data/services.jsonfile via the usual pull request process, though we highly discourage choosing FTL at this point. Starting March 1, 2021, we will no longer accept new additions for entries that use FTL to our services list.To provide an alternative to current FTL users, we will begin work on implementing WebRTC support in OBS. This currently does not have a set delivery date. Current work on this has progressed slower than we like, as the individuals who were working on it have been pulled towards other projects/jobs outside OBS.
After 9 months of WebRTC support being available in OBS, we plan to remove FTL support (all code using FTL and all services entries for FTL ingests) from the OBS codebase, meaning the first major or minor OBS release that comes 9 months after WebRTC release will not support FTL.
Alternatives to FTL
The primary advantage of FTL that advocates point to is its low latency (often less than a second screen-to-screen) compared to RTMP (often 2-3 seconds or more screen-to-screen).
Currently Supported by OBS
Currently, OBS supports the following output protocols:
Of these options, the protocol that most closely meets the needs of people wishing to stream at low latency is SRT, which is our current recommendation for services currently using FTL.
Under Consideration for OBS Support
WebRTC
WebRTC support is not currently in OBS. While support could be implemented, the main issue is that signaling methods between the broadcaster and the server are as varied as there are server implementations. Ideally, we could implement a pure WebRTC implementation and allow services to provide SDP signaling plugins that allow them to communicate with OBS using whatever system they want. All that said, none of this has progressed beyond the idea phase for official inclusion in OBS.
WISH
The primary downside to WebRTC is that implementation depends on the server that you are communicating with, such as Janus, Wowza, Millicast, Mediasoup, Evercast, etc.
There have recently been some efforts to create a standardized WebRTC signaling/ingest using HTTPS called WISH. While this project is not complete, it is something we are keeping our eye on for future developments.
More on WISH:
RIST
RIST is also not currently supported by OBS (EDIT: This is no longer true, OBS now supports RIST output as of OBS Studio 27,2), and so far we have not received a great deal of demand for it. That said, the development of the protocol has been done in a much more open manner, and there are many users of the protocol in production. Furthermore, it is RTP/UDP-based, like FTL, so it could be a possible alternative for FTL users.
More on RIST:
Stakeholders
YouNow (@rsiv) currently uses FTL. Glimesh (@haydenmc) appears to have been planning to use FTL. @GRVYDEV maintains Project Lightspeed, which ingests FTL. @NCommander appears to have recently started an effort to standardize FTL development.
Beta Was this translation helpful? Give feedback.
All reactions