Only download changed files

[dlech]

When your project starts to get big, downloading can take a while. We could try to be smart and only download files that have changed locally since the last time we downloaded.

[dlech]

This should include deleting files remotely that were deleted locally.

[WasabiFan]

@dlech is there a particular way you imagined this implemented (e.g., external library to use)? Or was the idea to simply compare file modification timestamps?

[dlech]

The vscode extension api has workspace.createFileSystemWatcher that provides create, change and delete events. I was thinking that we could maintain a list of new/changed files and a list of deleted files.

• When the extension loads, all files in the project are added to the new/changed list so that all are downloaded on the first download.
• When we download, we only download the files on the new/changed list and then clear that list.
• After the download, any deleted files are deleted remotely and the deleted list is cleared.

An optimized implementation could use the ev3devBrowser.download.include and ev3devBrowser.download.exclude configuration settings, but that would require watching for configuration setting changes are tearing down and setting up the file watcher again when the settings change. Given that the typical project will probably only be a handful of files, trying to filter the file watcher probably isn't worth it.

[WasabiFan]

Hmmm, interesting. I'm going to take a look at this and see what I can implement. Hopefully PR is incoming!

[WasabiFan]

FWIW, I have an implementation of this which works in almost all cases but ran into a bunch of minor quirks with the file watcher API that I haven't yet gone back and handled. I'm a bit worried that this implementation will be inconsistent and unpredictable, but we'll see. I am also unsure how to handle the case where the user deletes a file while the editor is closed.

[dlech]

I'll be glad to take a look at whatever you have. I really could use this feature lately!

I think the best we just start fresh whenever the extension is loaded and download all files. We can add a command to delete remote folders so that users can wipe out everything on the EV3 and re-download if things are acting funny.

[ndward]

Moving between my project folder in VS Code and the corresponding folder on the EV3 today (so that I could launch the same scripts from Brickman that I had launched from VS Code) I noticed that the folders' contents were very different. There were 69 scripts in the folder on the PC and more than 140 in the corresponding folder on the EV3. The huge size of the folder on the EV3 means that it takes forever to scroll through the folder in the Brickman file browser, maybe 3s to move from one file to the next or 3x140= 420s=7min just to scroll through the entire folder! The reason why the folders are different is clear enough: when you rename, move or delete a file in VS Code that works fine for the Windows folder but on the EV3 the original file remains with the same name in the same location so the folder on the EV3 begins to accumulate files that should not be there. And there is no way of deleting files or folders on the EV3 without using SSH, which I am determined that my beginner users should not have to use and which is the main advantage of the VS Code workflow.

My suggestion: when the contents of the project folder are downloaded from VS Code to the EV3 this should be systematically preceded by the deletion or emptying of the corresponding folder on the EV3 (if there is one). Assuming that would not take a significant amount of time then it would be worth doing for it would keep the contents of the two folders in sync, which is currently not happening. And I think it would be easier to implement than other approaches discussed on this page, though it would not solve the problem of it taking a long time to download a large folder (that probably contains many files that have not been modified since they were last downloaded).

[dlech]

I noticed that there are quite a few "sync" type VS Code extensions now. Perhaps we could borrow from one of them - or even better, find one that has a public API that we can just use (or contribute a public API to one of the projects if there isn't one).