pyembree ray casting acceleration

[ttsesm]

Hi @mikedh and thanks for the nice work with trimesh. I am quite new with it but it really gets in hand.

My question now is related with the ray casting approach based on the pyembree acceleration. I need to ray cast quite a few rays in a 3D scene, approx. 18060108 rays and maybe even more. Imagine that I have a scene with 16676 faces and from each face center I need to cast 1083 rays (i.e. 16676x1083 = 18060108).

As you understand I want to avoid for loops and I cast all the rays at once. Also I am using the pyembree based solution, as it states to accelerate the casting procedure in comparison to normal version, as follows:

import trimesh
import numpy as np

mesh = trimesh.Trimesh(vertices=vertices, faces=faces, face_normals=normals, process=False, use_embree=True) # create mesh from given data

intersection_points, index_ray, index_tri = mesh.ray.intersects_location(origins, drays, multiple_hits=False) # where origins are my triangle centers and drays my rays direction vectors

but this seems to perform quite slow as well, even though I am using pyembree acceleration. Is this the best I can get or I am doing something wrong?

[mikedh]

Hey, it could be the locatation calculation that's slow (happy to take speedup PR's!). I'd try running your code in a profiler like pyinstrument for details. Might also be worth checking to make sure it's actually using pyembree and not falling back to the native ray checks:

In [1]: import pyembree

In [2]: import trimesh

In [3]: m = trimesh.creation.icosphere

In [5]: m = trimesh.creation.icosphere()

# check for `ray_pyembree` instead of `ray_triangle`
In [6]: m.ray
Out[6]: <trimesh.ray.ray_pyembree.RayMeshIntersector at 0x7f8a8fc6e668>

[ttsesm]

@mikedh thanks for the prompt response. Hhmm, it seems that in my case it uses the ray_triangle instead of the ray_pyembree:

import pyembree
import trimesh
m = trimesh.creation.icosphere()
m.ray
<trimesh.ray.ray_triangle.RayMeshIntersector at 0x1a647739a30>

Any idea though why? Can I somehow force it to use the pyembree and not fall back to the native ray checking?

[mikedh]

It's probably wrapping an import error, what does this raise:

print(trimesh.ray.ray_pyembree.error)

[ttsesm]

it gives me the following:

>> print(trimesh.ray.ray_pyembree.error)
   Traceback (most recent call last):
  File "C:\Users\TTsesm\Desktop\dev\radiosity\r_venv\lib\site-packages\IPython\core\interactiveshell.py", line 3331, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)
  File "<ipython-input-10-d63e8e948013>", line 1, in <module>
    print(trimesh.ray.ray_pyembree.error)
  File "C:\Users\TTsesm\Desktop\dev\radiosity\r_venv\lib\site-packages\trimesh\exceptions.py", line 28, in __getattribute__
    raise super(ExceptionModule, self).__getattribute__('exc')
  File "C:\Users\TTsesm\Desktop\dev\radiosity\r_venv\lib\site-packages\trimesh\ray\__init__.py", line 5, in <module>
    from . import ray_pyembree
  File "C:\Program Files\JetBrains\PyCharm Community Edition 2020.1.1\plugins\python-ce\helpers\pydev\_pydev_bundle\pydev_import_hook.py", line 21, in do_import
    module = self._system_import(name, *args, **kwargs)
  File "C:\Users\TTsesm\Desktop\dev\radiosity\r_venv\lib\site-packages\trimesh\ray\ray_pyembree.py", line 10, in <module>
    from pyembree import rtcore_scene
  File "C:\Program Files\JetBrains\PyCharm Community Edition 2020.1.1\plugins\python-ce\helpers\pydev\_pydev_bundle\pydev_import_hook.py", line 21, in do_import
    module = self._system_import(name, *args, **kwargs)
ImportError: DLL load failed while importing rtcore_scene: The specified module could not be found.

I guess something is wrong with my pyembree installation?

[mikedh]

Yeah exactly. You can check the install with just from pyembree import rtcore_scene. Did you use their conda packages?

[ttsesm]

no I installed it manually with pip based on the following command scopatz/pyembree#23 (comment) and while it is installed without problems, apparently it seems it didn't get installed correctly. I am not sure though why.

Does it make any difference which version of embree I have installed? I know that it should be a v.2x thus in my case I installed the latest which is v2.17.

[aluo-x]

Also upgrading to embree 2.17.7 also significantly sped up my ray-casting, and the results look to be identical.
If someone gets around to it, might want to make a pull request.
See here: conda-forge/embree-feedstock#17

[ttsesm]

@aluo-x thanks for sharing this. Can you give some details how did you install embree v2.17.7 instead of v.2.14 on your conda environment (I guess you are using conda). In my case I managed to make my project to find pyembree by creating a conda environment (still though I could not figure out why it does not work with the pip installation). They way I've installed embree and pyembree in my conda environment is by following these steps scopatz/pyembree#14 (comment).

[ttsesm]

just to mention that now with the embree (v2.14) acceleration the ray casting is by far much faster in comparison to the fallback native solution. However, I wouldn't mind getting some more based on what @aluo-x mentioned and using embree v.2.17.7. Also it seems that there is a new wrapper which tries to provide direct access to the embree v3 api (https://github.com/sampotter/python-embree).

[aluo-x]

So basically, embree 2.17.7 is as far as I know the latest compatible version with the embree python package. To use it:

1. Start a conda environment without embree, pyembree, trimesh etc.
2. git clone --recursive https://github.com/conda-forge/embree-feedstock.git
3. Modify the meta.yaml file, take care to change the version and delete the sha256 (skipping that check) or fixing it to the correct one
4. cd into the recipe folder, type conda build .
5. Wait until it finishes, will appear frozen at times
6. Take note of the output path, and conda install NEW_PKG_PATH where you put in the location of the package
7. Install the pyembree package, and trimesh package. I usually install from source (git clone recursive, then python setup.py develop or conda build depending on if it has a recipe.

[ttsesm]

Thanks for sharing @aluo-x. In my case though using v2.17.7 instead of v2.14 did not provide any significant acceleration.

[aluo-x]

Sorry to hear that. I have a very modern CPU with 20+ cores. I was also surprised to see a speed increase.

[ttsesm]

I see, in my case I am applying this on a 6 core modern laptop :-p.

In any case, in comparison to my previous situation now I am in a much better situation. I am curious to see though if using Embree v3 with the other linked wrapper I'll see any difference.

[mikedh]

Yeah if you're on linux x86_64 you could also use the bash script we use to install in docker which uses embree 2.17.7:

wget https://github.com/mikedh/trimesh/blob/master/docker/builds/embree.bash
sudo bash embree.bash