go-libp2p-spdy has problems - Can not send objects to go, if they are larger than 64k

[dignifiedquire]

I've been tracking this issue for some time now, and it looks like this is due to spdystream (go implementation of spdy being used) not support flow control.

But lets start from the beginning.

1. Sending files from js to go ipfs works fine for files less than 63k, but starts suddenly failing at anything larger.
2. Looking into the logs, go-ipfs bitswap never receives the block we are sending
3. Looking into spdy it turns out it never emits the message we sent to the layer above.
4. It turns out the INITIAL_FLOW_CONTROL_WINDOW_SIZE is set at 64k, which spdy-stream (js impl) follows. But with larger buffers it starts using the flow control features of spdy.
5. It turns out there is an open issue to support flow control on spdystream, and as it does not implement it is not able to handle the frames spdy-transport is sending. Implement flow control moby/spdystream#2

[dignifiedquire]

cc @diasdavid @whyrusleeping

[dignifiedquire]

The diff of the logs for everyone interested (left fails, right suceeds), where the left side sends 64k data and right side 62k

https://www.diffchecker.com/Ulm69gxT

[dignifiedquire]

After the suggestions from @whyrusleeping I tried using multiplex. With a custom go ipfs build to enable it and updated libp2p-multiplex I got made some progress

The following sizes in bytes for a single file I was able to transfer from

• go -> js
• js -> js

For the transfer from js -> go see the comments for failure starting

const sizes = [
  1024,
  1024 * 62,
  // starts failing with spdy, errors as described above
  1024 * 64,
  1024 * 512,
  1024 * 768,
  1024 * 1023,
  // starts failing with multiplex, tcp disconnect, no error shown anywhere
  1024 * 1024,
  1024 * 1024 * 4,
  1024 * 1024 * 8
]

[daviddias]

Note: issue is in the go implementation