Is there cuda backend available for trying ?

[whatdhack]

Only see bindings/python/iree/runtime/system_api.py. Only see the following .

TARGET_BACKEND_TO_DRIVER = {
"dylib-llvm-aot": "dylib",
"vmvx": "vmvx",
"vulkan-spirv": "vulkan",
}

[stellaraccident]

We've had a short setback with the build setup for cuda and will be reenabling it shortly (within the week). In the meantime, official pypi wheels and nightly releases from last week do have it enabled.

That dict is just shortcuts and it is possible to specify it explicitly. @ThomasRaoux can you provide an example from the benchmark scripts?

[whatdhack]

@stellaraccident , thanks. I took the easy route - pip install . Would the next pip package have the cuda backend ? When is it expected to be available. Have another related question. I see that the list of options to iree.compiler.tf.compile_module () is at CompilerOptions. Is it still using grappler -> xla flow or it is possible to just use grappler now ?

[stellaraccident]

I believe we got the cuda build and bots fixed today, and we will find out when tonight's nightly builds (works locally).

There may be a terminology gap: we use a similar conversion path as is used for cloud TPUs. Internally, i think grappler is involved but isn't really a detail we care about. IREE's core compiler takes MHLO (when coming from TF).

Most of our new investment is in PyTorch, Jax, and TFLite and just rely on the TF team for their conversions and don't look that much at the details.

[whatdhack]

@stellaraccident , tried the newest release at https://github.com/google/iree/releases/tag/candidate-20220204.38 . Seems to fail with the following error in the MNIST example with vmvx and dylib-llvm-aot backend. Works with PyPI 20220109.9 .

tensorflow/python/distribute/merge_call_interim.py:51:0: error: 'util.global.store' op global type mismatch; global __iree_flow___sm_node19__optimizer.iter is 'tensor<i64>' but store is 'tensor<i32>'

What is backend string for cuda ? AFAIK, MHLO is similar to XLA emit, assuming M stands for meta. Just curious, what is the input in case of PyTorch, MHLO ?