Absence of time lead to 1.1.1970 in tar archieve which is incompatible with FAT32

[Scondo]

Describe the bug
When there is no time specified for file in ipfs record tar file produced for directory set time of this file to 1.1.1970 (start of Unix epoch). However extracting such archive to FAT32 leads to error as start of MS-DOS epoch is 1.1.1980 and no file in FAT32 could have time before that.

I would agree that if it's intentionally set file time to 1.1.1970 - this is a problem of FAT32, but if it's just absence of time - this could be any time in fact. So there is no reason to stick to start of Unix epoch- it may be time of archive creation as well.

To Reproduce
Get tar file constructed by

rust-ipfs/http/src/v0/root_files.rs

Line 96 in 4bce467

pub fn get<T: IpfsTypes>(

(I found it within very_long_file_and_symlink_names test) and try to extract it to FAT32.

At least crate tar used by rust-ipfs and 7z on windows produce an error.

Expected behavior
I don't know for any existing filesystem, may be some use even later date as limitation, so it's definitely not good idea to set default time to 1.1.1980.

Most obvious solution is to set time of file to time of archive creation. Another - to use 1.1.2001 as start of last notable epoch (https://en.wikipedia.org/wiki/Epoch_(computing))

It's not a big deal for now, so I expect discussion.

[koivunej]

I would agree that if it's intentionally set file time to 1.1.1970

I think this was either by looking at the tar archives produced (at then) by go-ipfs or looking at what gnu tar does by default. Or maybe it was a comment on hiding this information which isn't really needed. The only problem with using another timestamp (or a variable timestamp) is that the tar archives become "less canonical" and are even less hashable. This should had been all commented with the code obviously.

While the test (fix by you in #466) was using tempdir to just get the contents of a file, I think an extractor application should handle special cases like FAT32 not supporting unix epochs as datetimes... So I'm actually quite surprised of this:

At least crate tar used by rust-ipfs and 7z on windows produce an error.

I assume you used a quite recent 7z? For my own curiosity, what version of Windows did you test this on?

This issue as a whole was one of the reasons I didn't want to add any feature to write to filesystem as I couldn't see how could I test it on all systems. This is an important bug report for that front aside from the test fix as well, thanks for that!

[Scondo]

I assume you used a quite recent 7z?

Well, you right: this was a bug of some alpha verion I used... with latest 7z there is no problem: file extracted with time of archive creation.

Maybe it should be reported as bug for tar crate...

However as I can see from python tarfile module typical fix for this is just ignore error on setting time, so it still doesn't feel OK for me to just left problem for unpacker developers. At least not without discussion.

I'm not sure if it's so important byte-to-byte compatibility with go-ipfs. As for gnu tar... yes 1.1.1970 seems to be set with tar on mounted filesystem (IF absence of time will be encoded as 1.1.1970). But it anyway doesn't match with tar of original folder...

Anyway it's just feelings, not sure if that's really problem....

[koivunej]

I'm not sure if it's so important byte-to-byte compatibility with go-ipfs.

Agreed. I was thinking of more consistency and hashability than hashability to go-ipfs tar archives; after all the UnixFs structures are most likely always missing the ctime unless you really want to encode it there (UnixFs 1.1 or similar, memories are a bit fuzzy right now). Trying to remember this more, I think go's default tar impl used by go-ipfs at least used to create STAR archives/headers, which are different from GNU tar because of reasons, standardization bodies or lack of them.

Anyway it's just feelings, not sure if that's really problem....

Thanks for reporting this either way and your fix in #466. If we ever get to writing a filesystem extractor running it on different filesystems -- I was initially thinking of FAT32 on a ramfs if that's allowed on github CI, but that is of course dependent on linux erroring out when setting these timestamps on FAT32.