importing a .glb file fails if only local path is provided and not served through HTTP

[f4z3k4s]

Hi!

First of all, thanks for the great work for all maintainers.

I've just started using this library in my project for rendering .glb files.

When the .glb file is served from a local static server, SceneLoader.ImportMeshAsync and other similar methods used for importing assets, such as Append, AppendAsync, etc.. work perfectly. However, when the file is not requested over HTTP, but rather the local path is provided to the above stated methods, the app just crashes without any easily traceable error (I wasn't looking so hard for the error, but it didn't appear in the terminal of the packager). Maybe it's my uneducatedness but from the documentation available here, I expected the import to work with just a local path.

I was able to solve the issue by using expo-asset to load the .glb file instead and provide a local uri, and then passing that local uri to Babylon, described in the below way:

import '@babylonjs/loaders';
import { Asset } from 'expo-asset';
import { SceneLoader, ... } from '@babylonjs/core';
 
const [{ localUri }] = await Asset.loadAsync(
  require('/assets/3d/tesla-high.glb')
);

const { meshes } = await SceneLoader.ImportMeshAsync(
  '',
  localUri as string,
  '',
  scene
);

However, I'd expect the library to do something similar internally, making the user to be able to only provide a path to a local .glb file and it load it without issues. If this work is purposefully left for the user, it would be nice to have it stated in the main documentation, since I think it's quite a common use-case.

[ryantrem]

Thanks @f4z3k4s for logging this issue, and thanks for the details about the workaround you found. This will be helpful in solving this issue.

@CoPrez do we have another issue tracking local file load problems? If so, maybe we can combine the two, and if not, will you add details from your recent investigation to this issue?

[CoPrez]

Thanks for the shout-out @ryantrem. I hadn't logged an issue for my prior investigations because it was actually an issue with how React-Native bundles assets from outside the project folder. Loading the files into Babylon was working as I expected.

@f4z3k4s would you mind sharing a snippet of code on how you would expect the ImportMesh call to work inside the package? Are you talking about loading local assets like you would for an image? Like this:

SceneLoader.ImportMeshAsync(
  '',
  require('/assets/3d/tesla-high.glb`),
  '',
  scene
);

We don't currently support loading local assets by just using the require('asset_path') syntax... but I think that would be a great improvement and is probably something many React-Native developers would expect.

[f4z3k4s]

Thanks for the quick response guys.

I was thinking of something similar that you pasted. I just realised that require can not be handled internally by your package because it is resolved at build time from string literals, rather than variables.

So your proposal seems React-Native-y in my opinion, as it is really similar to how Image works. :)

Inside the package you could handle this in several ways, the simplest being Image.resolveAssetSource(rootUrl).uri which doesn't have external dependencies, but might have some trouble in release builds via this thread but I haven't investigated deeper. A similar solution to expo-asset would be my safest bet. Also the documentation should include that users have to edit their metro.config.js and add the desired extension to resolver.assetExts in order to make this solution viable.

[ninjz]

This enhancement would be great to have since having to install expo-asset just to support local files seems unnecessary

Edit:
I've followed @f4z3k4s 's steps and for some reason I'm still unable to load my asset through expo-asset. This is the error I'm getting:

Error [Error: Unable to import meshes from file:///var/mobile/Containers/Data/Application/0BB61DBB-2AE6-4376-B7FE-A9179A530FB6/Library/Caches/ExponentAsset-dc277ce3e4e776fd608a3f702787dabd.glb: Unable to find file named ExponentAsset-dc277ce3e4e776fd608a3f702787dabd.glb]

[f4z3k4s]

@ninjz It seems from the uri you're doing it on iOS. As my project's scope covers only Android for now, I haven't tested the solution on iOS. You can try the below method just to check if it's working:

import { Image } from 'react-native';

// rootUrl being something like /assets/your-asset.glb 
const uri = Image.resolveAssetSource(rootUrl).uri;

SceneLoader.ImportMeshAsync(
  '',
  require(uri),
  '',
  scene
);

[ninjz]

@f4z3k4s Sorry for the delay. I just tested this on iOS and figured out how to get it working with the help of your method as a starting point. This is how I got this working on iOS:

import { Image } from 'react-native';

// rootUrl being something like /assets/your-asset.glb 
const file = require(rootUrl);

const uri = Image.resolveAssetSource(file).uri;

SceneLoader.ImportMeshAsync(
  '',
  uri,
  '',
  scene
);

[f4z3k4s]

@ninjz Awesome, glad you could make it work. :)

[282931]

@ninjz this not work on android & ios release builds.

[ninjz]

@ninjz this not work on android & ios release builds.

Yeah shoot, I ran into this while testing and forgot to mention this as I haven't been working with the lib recently. Would love to know how others have overcome this issue.

I believe the uri is still referencing the local file system causing the file to not be found.

[f4z3k4s]

@ninjz as I mentioned earlier, you could either use expo-asset or look into its source code and see how they handling this to get an idea. expo-asset does work in release.

@282931 so for you, you might be able to use expo-asset as a workaround until this gets handled by this repo.

[ninjz]

@f4z3k4s I just tried using expo-asset on iOS and am still unable to get it working. Are you using expo for your project or a bare React Native app? I'm asking because I'm wondering if it has to do with this.

Been trying all sort of things to get this working in release mode. Has there really been no one here that has gotten this running in release mode on iOS with a local file successfully?

Edit: This is the local URI that is being outputted:
file:///private/var/containers/Bundle/Application/7F92D2BC-75D7-4475-80c0-444113B-48A86B/<Redacted App Name>.app/assets/assets/3d/man+props2.glb

[f4z3k4s]

@ninjz sorry I have no experience on iOS with this yet but the app I am working on has to support iOS shortly (couple weeks) so I will be urged to figure this out. I will share my findings if I'll have any and it will be relevant at that time. FYI, on Android, the project is bare RN app. For the first glance, your URI seems correct.

[ryantrem]

For anyone else who might look at this issue in the future, see additional discussion here: https://forum.babylonjs.com/t/how-to-import-a-local-glb-file-in-babylonreactnative/18212

[f4z3k4s]

@ninjz I can confirm it is working on iOS in release mode as well with the above stated expo-asset example. As a consequence, your issue is likely with correctly setting up expo-asset rather babylon-react-native itself.

To summarise: I can confirm that the above stated expo-asset example does work on both iOS and Android in release mode.

[ryantrem]

Latest thoughts on this issue can be found here in the forum: https://forum.babylonjs.com/t/load-local-stl-file-from-assets-in-react-native/26245/8?u=ryantrem

Here is a copy/paste of the relevant part:

Another option yet would be to rely on the React Native method of embedding asset files in the app via require. You can see an example of this here: #165 (comment). In this example, expo-asset is used, but you should be able to achieve the same thing without taking a dependency on Expo by doing this:

import resolveAssetSource from 'react-native/Libraries/Image/resolveAssetSource';
const stlUri = resolveAssetSource(require('./SomeModel.stl')).uri;
SceneLoader.ImportMeshAsync('', stlUri);

Whether you use expo-asset or the React Native libs directly, you will need to tell the metro bundler to bundle .stl files as assets by modifying the metro.config.js file with these changes:

const defaultAssetExts = require("metro-config/src/defaults/defaults").assetExts;
...
module.exports = {
  ...
  assetExts: [...defaultAssetExts, 'stl'],
};

It would be great if someone could validate these steps and document them here: BabylonReactNative/README.md at master · BabylonJS/BabylonReactNative (github.com)

EDIT: As @f4z3k4s mentions, the above will have issues on Android in release builds. We probably need to dig deeper for this, and @f4z3k4s comment above is probably a good place to start: #165 (comment).

[f4z3k4s]

@ryantrem Please note that resolveAssetSource does not work in Android Release mode (see this thread), that's the reason why I've used expo-asset. If someone's developing for iOS only, that might be a solution though.

[ryantrem]

Thanks @f4z3k4s, I forgot you called this out already earlier in this thread. I will update my comment above to include this extra info.

[bghgary]

For now, let's just document this for now. We will figure out how to do this properly later.

[CedricGuillemet]

From tests I did:
Loading glb/jpg using require works fine. My metro.config is a bit different than above :

  resolver: {
    assetExts: [
        ...defaultAssetExts,
        'glb', 'jpg'
    ]
  }

But I'm running into an issue with .json. require returns the loaded object and I can't get the uri back from it using resolveAssetSource.
This is a problem for some methods from class like NodeMaterial because loader only accepts a URL or a snippet ID.

Is there a way to actually get the URL from a JSON or should we put extra care in any loading method that needs to URL to a json to also accept a string?

Instead of NodeMaterial.Parse(starsShaderURL, scene).then((nodeMaterial) => { ...
do:

const starsShader = require('./assets/starfieldShader.json'); // json is parsed, returns an object
var starsShaderMaterial = new NodeMaterial("starsShader", scene);
starsShaderMaterial.loadFromSerialization(starsShader, "");
starsShaderMaterial.build();

[CedricGuillemet]

Another solution with react-native-asset : https://forum.babylonjs.com/t/load-local-app-gltf-glb-mesh-in-babylon-react-native/30888/23

[CedricGuillemet]

An example with react-native-fs here https://forum.babylonjs.com/t/using-webxrimagetracking-in-a-bundled-react-native-app/40337/17
I've not tested yet.