Remove Tar.Make and Archive modules

[reynir]

The Tar.Make functor does not seem to be used much if at all in other code bases. Neither seems Tar_unix or Tar_lwt_unix to be used.

Aside from the low or lack of usage, the Tar.Make functor's IO argument differes from the arguments to Tar.HeaderWriter and Tar.HeaderReader in that the IO module works on bytes buffers while the other ones work on Cstruct.t buffers, and IO is not parameterized over an async module. Internally, the Tar.Make functor creates HeaderWriters and HeaderReaders, too. A number of the functions in Tar.Make are as well marked as deprecated.

Futhermore, the Tezos codebase according to this comment opted to use their own implementation over Tar_lwt_unix as the performance of some of the functions are poor.

In order to ease maintenance I would like to remove these modules that are not seeing much usage and perform poorly.

[reynir]

Tar.Make is used in Cuekeeper (CC @talex5) https://github.com/talex5/cuekeeper/blob/master/lib/utils/git_storage.ml#L21. Not much is used, and I can submit a PR to help migrate if we decide to remove Tar.Make.

[talex5]

CueKeeper is using very out-of-date versions of things anyway, so doesn't need to block this. Looks like it's only using Header (which already got moved out of the functor) and Archive.create_gen.

[0xrotense]

can I work on this?

[reynir]

@0x0god can you first reply in reynir/mirage-block-partition#7 my questions about the code? It is okay if you want to work on this instead.

[0xrotense]

Yes okay, i will try this instead

[reynir]

@dianaoigo I will assign you this issue. I think you need to comment here before I can do it.

[dianaoigo]

@reynir, yes. please do. already started working on it

[MisterDA]

I would argue for the opposite: iterating on tar archives is super useful. I like the Archive module, I'm using the lwt backend using file descriptors (a buffered backend with Lwt_io would also be cool). I'd rather have a more unified interface rather than removing features, because then I find myself reimplement (often poorly) features.

A recent use-case I'm having is examining a tar.gz archive using lwt as a backend. It feels like I'm writing a lot of complicated boiler-plate code that should really be somewhere in this package.

[hannesm]

@MisterDA interesting that you're using the Archive module. I'm all in for providing nice abstractions and functionality, but to me it looks like the Archive and Tar.Make do not have any users in the open source, and the tar-mirage implements a different way on accomplish very similar things. Sharing code in between lwt and mirage would be super-useful.

I'm still not sure what the best way is here, one path is to make the tar-mirage interface the one to use (and provide a lwt-only version thereof -- maybe with functors pre-initiated)?

[MisterDA]

Sorry, I've double-checked: I'm not using Archive (yet). In obuilder, we're using an strange overlay of Tar_lwt_unix, which to me indicates that what's the library is exposing doesn't fit right.
https://github.com/ocurrent/obuilder/blob/master/lib/tar_transfer.ml

I've been hacking using tar today, and I think what I wanted was Tar_gz.Archive.
ocurrent/ocaml-ci#791

I'm still not sure what the best way is here, one path is to make the tar-mirage interface the one to use (and provide a lwt-only version thereof -- maybe with functors pre-initiated)?

I haven't looked at tar-mirage yet! Will report.

[hannesm]

@MisterDA thanks for your comment. I think it would be nice to then (a) remove as outlined in the subject of this issue and (b) if you (or someone else) could PR the stuff you use in "obuilder" (is it similar to what tezos uses?), so that can be integrated!?

[MisterDA]

After a bit of hack on ocaml-tar, I've changed my mind. I think simplifying it is good.
I'd like to have Archive.with_next_file stay in some form.

[hannesm]

Done in #127