Check if HDF5 "file" is a DAOS object

[brtnfld]

PR to skip the POXIS calls used in reading magic number. These POSIX calls can not be used for a DAOS object.

Intel DAOS team feedback is requested as we might be able to remove the HDF5 calls.

This solution assumes Unified Namespace and getfattr is available.

[brtnfld]

We might refrain from using getfattr if we modify H5Fis_accessible to return the VOL access type instead. Investigating.

[christgau]

Wouldn't it be better to directly call getxattr(2) from sys/xattr.h here instead of calling a command which has to be in the path (sure, it is very likely the case) and could potentially do something else than returning file attributes? In the end, getfattr does same call to retrieve the attributes.... at least on a CentOS 7 installation.

[DennisHeimbigner]

I do not think this code is in the proper location. I think it should be part of the later
file reading checks.

[brtnfld]

I do not think this code is in the proper location. I think it should be part of the later
file reading checks.

I think it has to be before phase 7 because that looks like the first place you start to read the magic number. Where do you suggest?

[brtnfld]

I do not think this code is in the proper location. I think it should be part of the later
file reading checks.

Wouldn't it be better to directly call getxattr(2) from sys/xattr.h here instead of calling a command which has to be in the path (sure, it is very likely the case) and could potentially do something else than returning file attributes? In the end, getfattr does same call to retrieve the attributes.... at least on a CentOS 7 installation.

Yes, that would be better.

[DennisHeimbigner]

FYI, I am working on this again.

[DennisHeimbigner]

I see numerous issues with this PR. Most importantly, the DAOS check
appears to be in the wrong place in the code. Should it not be in openmagic?
Oh well, we can fix it later.

[WardF]

@DennisHeimbigner it has not been merged, so now would be a good time to fix it too :)

[DennisHeimbigner]

Well let me take a look.

[DennisHeimbigner]

This is coming back to me. It makes me uneasy from a security point of view to use popen
because it in effect is executing a shell script of sorts as part of the netcdf-c library.
I really wish there was another, more direct, way to test for DAOS.

[DennisHeimbigner]

Also, there is not test case for this. Is there any open DAOS based server against we can test?

[dopplershift]

There's a getxattr() api that can (and probably should) be used instead.

[brtnfld]

I think this only needs to be done now if dfuse is not being used, as a dfuse netcdf file should appear as a POSIX file. This addressed the case of unified name space being used, which seems less common now.

[christgau]

I think this only needs to be done now if dfuse is not being used, as a dfuse netcdf file should appear as a POSIX file.

True.

This addressed the case of unified name space being used, which seems less common now.

But still is a valid use case. Supporting this appears to be lightweight in terms of code. I would highly appreciate it to be available (checking extended file attributes directly, instead via an popen call).

[christgau]

Is there any indicator which DAOS version this branch is developed against?

[brtnfld]

At the time, it would have been DAOS 2.0

[DennisHeimbigner]

Attached is an revised version of dinfermodel.c for you to test. It isolates the DAOS code
and makes it a bit more conformant to the dinfermodel.c coding style.
It also implements the daos check using the API calls listxattr and getxattr
(see dinfermodel.c#isdaoscontainer.c).

Please see it works for your tests.
dinfermodel.c.txt

[brtnfld]

Your patch works and correctly detects the daos file with the daos-vol, which was tested on Google Cloud. I see other testing errors, but I don't think they relate to this issue.

[WardF]

Ok, this test is now passing but taking 25 minutes, which feels longer than it was. I'm looking at that now. The work @DennisHeimbigner did to include xattr is great, but there are also missing checks for the required #define in configure.ac and CMakeLists.txt. Fixing that as well. But this should be good, with any luck. Thanks for your extreme patience!

[WardF]

Ok. On systems without sys/xattr.h or getfattr, standard output is spammed with 'getfattr not found`; this slows the tests CONSIDERABLY. Adding appropriate checks to mitigate this.

[WardF]

Thanks!