How to convert the NodeArg type property into numpy dtype?

[cansik]

I would like to check with onnxruntime what the input type of the tensors are. This can be done by using get_inputs() on the session object. This method returns a list of NodeArg which contain the name, shape and type of the tensor. However, the type is in string format, not usable for being used programmatically (f.e. tensor(float)).

Openvino for example has a neat function to convert from openvino format into numpy dtype. Is there somethig similar in onnxruntime to convert tensor(float) into np.float32? Or how is it possible to read the tensor type?

[skottmckay]

Not aware of anything being defined in the ORT python library as it seems like a slightly unusual use-case. Given you need to provide valid input to run the model, why would you not know the type of that data and/or the details of the model inputs?

FWIW models can be viewed in Netron and the input/output data types are shown there.

You can create a fairly simple lookup table like this:

onnxruntime/onnxruntime/python/tools/transformers/models/whisper/whisper_helper.py

Lines 371 to 378 in 430a086

	ort_to_np = {
	"tensor(float)": np.float32,
	"tensor(float16)": np.float16,
	"tensor(int64)": np.int64,
	"tensor(int32)": np.int32,
	"tensor(int8)": np.int8,
	"tensor(uint8)": np.uint8,
	}

Or alternatively use the ONNX python library to read the model input details.

import onnx
m = onnx.load('model.onnx')
for i in m.graph.input:
    if i.type.HasField('tensor_type'):
        print(i.type.tensor_type.elem_type)

Data type values are defined here: https://github.com/onnx/onnx/blob/c95a59c4754e40f0eb33174d7dfe5e2b9bd0ed7f/onnx/onnx.proto#L497-L542

[cansik]

I am currently writing a latency benchmark tool which benchmarks various backends and frontends (onnxruntime, openvino, ncnn and so on), where it is possible to load generic models and inference them with random data. For this purpose I need to know what the input type is.

I know this is not how the library is usually used. However, what is the purpose to return the type as a string instead of a type that can be used for further processing?

Thanks for the workaround with the lut and onnx, I think I'll do the same thing as whisper did. I just wanted to clarify if this is the way to go.

Update: the lookup table is not really great, since it is difficult to know, which types are used. For example this table does not contains the tensor(float32) type. So, it's better to always install onnx as well an parse the model. Do you think it would make sense to create a PR for onnxruntime to expose the actual type and not just a string?

[skottmckay]

The string is what ONNX uses. Not sure why.

https://github.com/onnx/onnx/blob/6ae260e42b993a62ec5c054f6c618879f36929d8/onnx/defs/data_type_utils.h#L17-L32