WebSocket streaming API

[om26er]

I have a websocket reverse proxy that forwards websocket messages from clients to a backend server. The front-facing server is based on NBIO WebSocket AND the backend instance is based on Yamux (https://github.com/hashicorp/yamux).

Due to the message oriented nature of websocket, it doesn't really have any backpressure, so it becomes quite easy to exhaust the server. It also results in increase memory footprint because a full websocket message has to be buffered first and only then we can forward that message to the backend server.

NBIO should expose an optional API, that exposes length of the websocket message upfront AND provides a "reader" object that can be read from. This would allow to translate any incoming websocket message into a length prefixed messaging scheme that I can use to forward to the yamux server, this will obviously reduce the memory footprint and due to the streaming nature, it will be possible to apply TCP backpressure as well

OnMessageStream(messageType int, messageLength int, reader io.Reader) {
}

[lesismal]

Is OnDataFrame satisfy your need? Please try:
https://github.com/lesismal/nbio-examples/blob/master/websocket/data_uploader/server/server.go#L27

[om26er]

It doesn't because it does not expose the length of the message upfront. For any effective length prefixed messaging, we need to know the length of the message upfront, also exposing a reader object would mean that OnMessageStream is only called once per message.

So I could essentially do

OnMessageStream(messageType int, messageLength int, reader io.Reader) {
    yamuxStream.Write(intToBytes(messageLength))
    io.Copy(yamuxStream, reader)
}

[lesismal]

The Websocket protocol itself doesn't provide a way to get the message length before the last data frame arrived with fin. So, it's impossible for frameworks to provide the message length before receiving the last frame.
To avoid a large-size buffer used for a single message, we can cache every data frame using a [][]byte, then we use not one large-size buffer but multi small-size buffers and can get the message size when the last frame arrives.

[lesismal]

And maybe another way: since your backend is handling stream, how about adding a length field in your application layer protocol header?
Then OnDataFrame works.

[om26er]

The Websocket protocol itself doesn't provide a way to get the message length before the last data frame arrived with fin.

I am not sure if that's true, a websocket message does contain the length of the message in the first few bytes, see https://www.openmymind.net/WebSocket-Framing-Masking-Fragmentation-and-More/ -- we just need to find a way to expose that in the API

[lesismal]

a websocket message does contain the length of the message in the first few bytes,

That's not message length, that's the frame length. A message will be separated into multi frames to avoid large-size buffer usage:
https://github.com/lesismal/nbio/blob/master/nbhttp/websocket/conn.go#L184

[om26er]

So I found an alternative: Just forward the wire level bytes from OnData handler to a Yamux stream. On Yamux side, I use helpers from https://github.com/gobwas/ws to parse websocket messages.

[lesismal]

I don't think it's a good idea to use gobwas/ws in production, for more details:
gobwas/ws#121
gobwas/ws#143
gobwas/ws-examples#18

One more thing, OnDataFrame may be better.

[om26er]

Do note I only use gobwas to parse websocket wire-level messages which are sent from front-facing proxy over a multiplexed yamux transport, this essentially allows me run websocket on top of Yamux. The front-facing proxy is still NBIO

[lesismal]

Do note I only use gobwas to parse websocket wire-level messages which are sent from front-facing proxy over a multiplexed yamux transport, this essentially allows me run websocket on top of Yamux. The front-facing proxy is still NBIO

That's fine.

[lesismal]

To reduce the total memory usage, SetMemoryLimit may help, you can try it.

Related:
#279 (comment)
#279 (comment)