How to run warp kernels in python scripts within other software?

[cadop]

I trying to resolve an issue with integrating warp in a program similar to omniverse. Essentially, scripts can be run within the program, it uses CPython (3.9). There seems to be an issue where paths are being requested but aren't accessible.

My guess here is that the kernel is being built somewhere and can't be found, or the source of the python code is not visible since it is being stored internally to the software (or some temp location).

Does this logic make any sense theoretically, or is totally not how things work? If it is, is there a way to specify in the python script where kernels should get stored to as a temporary/intermediary shared location?

This was a discussion on discord that was answered by @christophercrouzet . I thought it would be useful for tracking on github.

[cadop]

This was the answer from @christophercrouzet that solved the problem for me:

I'm assuming that you are trying to run eval() on a string representing Warp code? If so, due to a limitation with Python, Warp code requires to be evaluated from a file. We don't expose any API that handles that workaround but the it's fairly simple: save the code into a temporary file, use importlib to load the file as a Python module, and manually call the functions in there if needed. There's an implementation example available in warp/tests/test_transient_module.py.

Essentially you use a string to wrap the code, make a temporary file, and run that as the module. From the warp test mentioned,

warp/warp/tests/test_transient_module.py

Line 55 in 5dfdb52

def test_transient_module(test, device):

CODE = """# -*- coding: utf-8 -*-

import warp as wp

@wp.struct
class Data:
    x: wp.array(dtype=int)

@wp.func
def increment(x: int):
    # This shouldn't be picked up.
    return x + 123

@wp.func
def increment(x: int):
    return x + 1

@wp.kernel
def compute(data: Data):
    data.x[0] = increment(data.x[0])
"""


def load_code_as_module(code, name):
    file, file_path = tempfile.mkstemp(suffix=".py")

    try:
        with os.fdopen(file, "w") as f:
            f.write(code)

        spec = util.spec_from_file_location(name, file_path)
        module = util.module_from_spec(spec)
        spec.loader.exec_module(module)
    finally:
        os.remove(file_path)

    return module

Simply calling module = load_code_as_module(CODE, "") will execute the script, so if you have something like a print statement in the original code string, it will execute. The return of module lets you access elements though, so you can access the struct that was defined by data = module.Data().