using ORT from C++ with WebGPU or WebNN support

[ghost]

I would like to compile to WebAssembly a program written in C++ that uses ORT as a library. I have found a page that says I can build ORT using the --build_wasm_static_lib flag, but ideally I’d like to have either WebGPU or WebNN support.

I could find very little information on whether that is currently supported. Is it enough to compile it with both the --build_wasm_static_lib and the --use_jsep flags?

[tianleiwu]

There is a link in the page to build webgpu:
https://gist.github.com/fs-eire/a55b2c7e10a6864b9602c279b8b75dce

[ghost]

The problem I have is that it seems that page talks about building it for use from JavaScript, not as a C++ library.

[gyagp]

I don't think this is supported now.
Current ORT implementation to support WebGPU/WebNN includes 2 parts, C++ and TypeScript, which will be compiled to WASM and JS respectively. After compilation, JS API will be provided for you to program with.
In your case, the ORT implementation is better to be purely native, like many other EPs. WebNN doesn't support this, while WebGPU has native C/C++ API. Then for the WebGPU case, you can write any applications with ORT WebGPU native API, and it's your choice whether to convert it to a web version (via Emscripten) or not.

[ghost]

I’m confused about what you mean when you say “native” in this context. By “native”, do you just mean “accessible from C/C++”? (Regardless of whether it’s machine code or Wasm, I mean.)

I also don’t understand why you say that this isn’t supported, but then also say that it is possible to use ORT with Wasm and WebGPU from C++. (That is exactly what I’m looking for!)

If it is possible to compile a C++ program that uses ORT to Wasm and have it use WebGPU, that is great! I couldn’t figure out how to do it. (Though now it’s been a few months, so I can’t provide clear details on why.) I will keep trying it, though, and if I run into specific issues I can’t solve, I’ll provide more details. But hints and advice are appreciated regardless!

[gyagp]

I'm talking about the programming APIs for developers using ORT. Native is the opposite of the web, which means JS. Native means non-web, including C, C++, Rust, etc.
I just meant the second approach (WebGPU as a native EP of ORT) is technically possible, but it's not ORT's implementation now. This approach has other concerns, like debuggability, async handling, interaction with other web components, etc.
The second approach is like project wonnx (https://github.com/webonnx/wonnx), which is implemented in Rust.
To implement a Web AI framework with pure JS (like TensorFlow.js), or hybrid (like ORT WebGPU), or pure native, is still an open question.

[ghost]

Ah, I understand. You’re saying that it would be possible for ORT to allow C/C++ programs to be compiled to Wasm (with Emscripten) and use ORT with WebGPU, but that this is currently not supported nor implemented.

In my case, I’m trying to port to WebAssembly an existing program written in C++ that uses ORT. (More specifically, Lc0.) So that feature would be valuable for my specific use case. But I understand that it is not currently possible. Thank you for your answer!