faster OS image download

[iwanbk]

Is your feature request related to a problem? Please describe

The full VM OS image download is slow from Australia, it took around 20 mins

Feature Request: Faster or Local OS load for (big) images

threefoldtech/test_feedback#419

Describe the solution you'd like

There are two kind of solutions:

1. from hub side
   It is already described on https://git.ourworld.tf/tfgrid/circle_engineering/issues/33: create replica of flist hub that are closer to the users

2. speedup the download from zos side
   rfs download the block on read operation. Because the download triggered by io.Copy (

   zos/pkg/storage/disk.go

   Lines 148 to 152 in 0ea6170

   	_, err = io.Copy(file, source)
   	if err != nil {
   	return errors.Wrap(err, "failed to write disk image")
   	}

   ), the flow become something like this:
   - OS try to read some block
   - rfs call remote hub to download the block
   - repeat until finish
   The process happens sequentially, hence very slow

   Some alternatives:
   a. Do the download on open, it is what happens in 0-fs
   IMO It is quite a big change from architecture point of view, i currently don't know the impact of this change
   b. paralelize the io.Copy
   It is a simple solution, but if the slowness happen on other usecase, we have to apply the same technique, which might be error prone.

While making hub replica would help to speedup the issue, i think solution 2.b also worth to do because it would further speedup the process and also quite easy to do.

[iwanbk]

I've tried to paralelize the io.Copy with this environment:

• zos node: In Indonesia, run on my old 2016 PC on qemu. (indonesia is close to Australia)
• os image: nixos

the results:

• original io.Copy: 1 hour (much worse than the reported 20 mins :) )
• 5 goroutines: 20 mins
• 10 goroutines: 20 mins
• 15 goroutines: 24 mins

While the number of speedup (in regards to number of goroutines) might be different for different locations, the goroutines is indeed speedup the process.

[iwanbk]

cc @xmonader @rawdaGastan @ashraffouda