GLMakie, relocatable with PackageCompiler ?

[AndreasKuhn-ak]

I created an interactive simulation/animation using GLMakie of the Vicsek Model. Everything runs fast and without any issues. Based on that, i wanted to make a standalone app with the create_app() function of the package PackageCompiler. This also works well on my machine, but when i try to run it on another machine (which is the entire purpose of an standalone app), i get the error:

I belive, this is due to the fact GLMakie works with absolute paths, which are only valid on the machine that created the app.
There is an package called "RelocatableFolders" which provides a workaround to that problem. The @path macro can be put infront of every @dir macro to handle absolute paths and provide an alternative local copy of the contents of the path.
I belive the function loadshader(name) = normpath(joinpath(@DIR, "..", "assets", "shader", name)) in the GLMakie.jl file is the root of all evil. So i tried to use the @path macro there. normpath(@path joinpath(@DIR, "..", "assets", "shader", name)). It still works on my machine but does not create a local copy of the objects in the given path, therefore it gives me an error on a different machine aswell:
ArgumentError: not a path: C:\Users\ank10ki\.julia\packages\GLMakie\pFGSp\assets\shader\postprocessing\fullscreen.vertnot a path: C:\Users\ank10ki\.julia\packages\GLMakie\pFGSp\assets\shader\postprocessing\fullscreen.vert
Therefore, i have a few questions at hand:

1. Am i doing something horribly wrong here ?
2. If not, is GLMakie relocatable at all, due to its apparent reliance on absolute paths ?
3. Is this the right place to ask this question, or should i aks the guys from PackageCompiler or RelocatableFolders ?

[MichaelHatherly]

Were you using @path within a function def or at toplevel? It'll only really do the right thing at toplevel in a module. You should be able to follow what's done with https://github.com/JuliaPlots/Makie.jl/blob/master/src/Makie.jl#L253 for the GLMakie assets I'd think.

[AndreasKuhn-ak]

Thanks for your suggestion, i tried it this way:

const ASSETS_DIR = RelocatableFolders.@path joinpath(@__DIR__, "..", "assets", "shader")
loadshader(name) = normpath(joinpath(ASSETS_DIR, name))
instead of 
loadshader(name) = normpath(joinpath(@DIR, "..", "assets", "shader", name)) 

but nothing changed. It works on my pc and on other pcs it throws the same error as in the picture above.

[MichaelHatherly]

Can you provide the output of the following after making the @path change (before you create a packagecompiler image):

julia> VERSION

julia> using GLMakie

julia> GLMakie.ASSETS_DIR.files

so we can see whether it's picking up the right files or not.

[AndreasKuhn-ak]

julia> VERSION
v"1.6.3"

julia> using GLMakie

julia> GLMakie.ASSETS_DIR.files
Dict{String, Vector{UInt8}} with 30 entries:
  "heatmap.vert"                     => [0x7b, 0x7b, 0x47, 0x4c, 0x53, 0x4c, 0x5f, 0x56, 0x45, 0x52  …  0x20, 0x30, 0x2c, 0x20, 0x31, 0x29, 0x3b, 0x0a, 0x7d, 0x0a]
  "postprocessing\\fxaa.frag"        => [0x7b, 0x7b, 0x47, 0x4c, 0x53, 0x4c, 0x5f, 0x56, 0x45, 0x52  …  0x20, 0x29, 0x2e, 0x72, 0x67, 0x62, 0x3b, 0x0a, 0x7d, 0x0a]
  "sprites.vert"                     => [0x7b, 0x7b, 0x47, 0x4c, 0x53, 0x4c, 0x5f, 0x56, 0x45, 0x52  …  0x64, 0x65, 0x78, 0x2b, 0x31, 0x29, 0x3b, 0x0a, 0x7d, 0x0a]
  "line_segment.vert"                => [0x7b, 0x7b, 0x47, 0x4c, 0x53, 0x4c, 0x5f, 0x56, 0x45, 0x52  …  0x65, 0x72, 0x74, 0x65, 0x78, 0x29, 0x3b, 0x0a, 0x7d, 0x0a]
  "parametric.frag"                  => [0x7b, 0x7b, 0x47, 0x4c, 0x53, 0x4c, 0x5f, 0x56, 0x45, 0x52  …  0x72, 0x74, 0x6f, 0x79, 0x2f, 0x0a, 0x0a, 0x2a, 0x2f, 0x0a]
  "intensity.frag"                   => [0x7b, 0x7b, 0x47, 0x4c, 0x53, 0x4c, 0x5f, 0x56, 0x45, 0x52  …  0x78, 0x2c, 0x20, 0x30, 0x29, 0x29, 0x3b, 0x0a, 0x7d, 0x0a]
  "volume.vert"                      => [0x7b, 0x7b, 0x47, 0x4c, 0x53, 0x4c, 0x5f, 0x56, 0x45, 0x52  …  0x64, 0x5f, 0x76, 0x65, 0x72, 0x74, 0x3b, 0x0a, 0x7d, 0x0a]
  "particles.vert"                   => [0x7b, 0x7b, 0x47, 0x4c, 0x53, 0x4c, 0x5f, 0x56, 0x45, 0x52  …  0x69, 0x74, 0x69, 0x6f, 0x6e, 0x29, 0x3b, 0x0a, 0x7d, 0x0a]
  "parametric.vert"                  => [0x7b, 0x7b, 0x47, 0x4c, 0x53, 0x4c, 0x5f, 0x56, 0x45, 0x52  …  0x20, 0x30, 0x2c, 0x20, 0x31, 0x29, 0x3b, 0x0a, 0x7d, 0x0a]
  "dots.vert"                        => [0x7b, 0x7b, 0x47, 0x4c, 0x53, 0x4c, 0x5f, 0x56, 0x45, 0x52  …  0x65, 0x72, 0x74, 0x65, 0x78, 0x29, 0x3b, 0x0a, 0x7d, 0x0a]
  "standard.vert"                    => [0x7b, 0x7b, 0x47, 0x4c, 0x53, 0x4c, 0x5f, 0x56, 0x45, 0x52  …  0x69, 0x74, 0x69, 0x6f, 0x6e, 0x29, 0x3b, 0x0a, 0x7d, 0x0a]
  "volume.frag"                      => [0x7b, 0x7b, 0x47, 0x4c, 0x53, 0x4c, 0x5f, 0x56, 0x45, 0x52  …  0x64, 0x2c, 0x20, 0x30, 0x29, 0x29, 0x3b, 0x0a, 0x7d, 0x0a]
  "postprocessing\\SSAO_blur.frag"   => [0x7b, 0x7b, 0x47, 0x4c, 0x53, 0x4c, 0x5f, 0x56, 0x45, 0x52  …  0x2c, 0x20, 0x31, 0x2e, 0x30, 0x29, 0x3b, 0x0a, 0x7d, 0x0a]
  "postprocessing\\fullscreen.vert"  => [0x7b, 0x7b, 0x47, 0x4c, 0x53, 0x4c, 0x5f, 0x56, 0x45, 0x52  …  0x32, 0x28, 0x30, 0x2c, 0x31, 0x29, 0x3b, 0x0a, 0x7d, 0x0a]
  "surface.vert"                     => [0x7b, 0x7b, 0x47, 0x4c, 0x53, 0x4c, 0x5f, 0x56, 0x45, 0x52  …  0x69, 0x74, 0x69, 0x6f, 0x6e, 0x29, 0x3b, 0x0a, 0x7d, 0x0a]
  "lines.geom"                       => [0x2f, 0x2f, 0x20, 0x2d, 0x2d, 0x2d, 0x2d, 0x2d, 0x2d, 0x2d  …  0x72, 0x61, 0x74, 0x69, 0x6f, 0x29, 0x3b, 0x0a, 0x7d, 0x0a]
  "postprocessing\\copy.frag"        => [0x7b, 0x7b, 0x47, 0x4c, 0x53, 0x4c, 0x5f, 0x56, 0x45, 0x52  …  0x20, 0x3d, 0x20, 0x31, 0x2e, 0x30, 0x3b, 0x0a, 0x7d, 0x0a]
  "sprites.geom"                     => [0x7b, 0x7b, 0x47, 0x4c, 0x53, 0x4c, 0x5f, 0x56, 0x45, 0x52  …  0x74, 0x69, 0x76, 0x65, 0x28, 0x29, 0x3b, 0x0a, 0x7d, 0x0a]
  "postprocessing\\postprocess.frag" => [0x7b, 0x7b, 0x47, 0x4c, 0x53, 0x4c, 0x5f, 0x56, 0x45, 0x52  …  0x74, 0x65, 0x20, 0x6c, 0x75, 0x6d, 0x61, 0x0a, 0x7d, 0x0a]
  "distance_shape.frag"              => [0x7b, 0x7b, 0x47, 0x4c, 0x53, 0x4c, 0x5f, 0x56, 0x45, 0x52  …  0x20, 0x66, 0x5f, 0x69, 0x64, 0x29, 0x3b, 0x0a, 0x7d, 0x0a]
  "standard.frag"                    => [0x7b, 0x7b, 0x47, 0x4c, 0x53, 0x4c, 0x5f, 0x56, 0x45, 0x52  …  0x20, 0x6f, 0x5f, 0x69, 0x64, 0x29, 0x3b, 0x0a, 0x7d, 0x0a]
  "line_segment.geom"                => [0x7b, 0x7b, 0x47, 0x4c, 0x53, 0x4c, 0x5f, 0x56, 0x45, 0x52  …  0x31, 0x29, 0x2c, 0x20, 0x31, 0x29, 0x3b, 0x0a, 0x7d, 0x0a]
  "image.vert"                       => [0x7b, 0x7b, 0x47, 0x4c, 0x53, 0x4c, 0x5f, 0x56, 0x45, 0x52  …  0x74, 0x69, 0x63, 0x65, 0x73, 0x29, 0x3b, 0x0a, 0x7d, 0x0a]
  "texture.frag"                     => [0x7b, 0x7b, 0x47, 0x4c, 0x53, 0x4c, 0x5f, 0x56, 0x45, 0x52  …  0x0a, 0x20, 0x20, 0x20, 0x20, 0x29, 0x3b, 0x0a, 0x7d, 0x0a]
  "dots.frag"                        => [0x7b, 0x7b, 0x47, 0x4c, 0x53, 0x4c, 0x5f, 0x56, 0x45, 0x52  …  0x65, 0x63, 0x74, 0x69, 0x64, 0x29, 0x3b, 0x0a, 0x7d, 0x0a]
  "lines.frag"                       => [0x7b, 0x7b, 0x47, 0x4c, 0x53, 0x4c, 0x5f, 0x56, 0x45, 0x52  …  0x20, 0x66, 0x5f, 0x69, 0x64, 0x29, 0x3b, 0x0a, 0x7d, 0x0a]
  "fragment_output.frag"             => [0x7b, 0x7b, 0x47, 0x4c, 0x53, 0x4c, 0x5f, 0x56, 0x45, 0x52  …  0x72, 0x69, 0x74, 0x65, 0x73, 0x7d, 0x7d, 0x0a, 0x7d, 0x0a]
  "lines.vert"                       => [0x7b, 0x7b, 0x47, 0x4c, 0x53, 0x4c, 0x5f, 0x56, 0x45, 0x52  …  0x65, 0x72, 0x74, 0x65, 0x78, 0x29, 0x3b, 0x0a, 0x7d, 0x0a]
  "util.vert"                        => [0x7b, 0x7b, 0x47, 0x4c, 0x53, 0x4c, 0x5f, 0x56, 0x45, 0x52  …  0x20, 0x6f, 0x66, 0x66, 0x34, 0x29, 0x3b, 0x0a, 0x7d, 0x0a]
  "postprocessing\\SSAO.frag"        => [0x7b, 0x7b, 0x47, 0x4c, 0x53, 0x4c, 0x5f, 0x56, 0x45, 0x52  …  0x3b, 0x0a, 0x20, 0x20, 0x20, 0x20, 0x7d, 0x0a, 0x7d, 0x0a]

[AndreasKuhn-ak]

Oh wait this was from an old session. After the change the output stays the same but i get an additional warning.

julia> VERSION
v"1.6.3"

julia> using GLMakie
[ Info: Precompiling GLMakie [e9467ef8-e4e7-5192-8a1a-b1aee30e663a]
┌ Warning: Package GLMakie does not have RelocatableFolders in its dependencies:
│ - If you have GLMakie checked out for development and have
│   added RelocatableFolders as a dependency but haven't updated your primary
│   environment's manifest file, try `Pkg.resolve()`.
│ - Otherwise you may need to report an issue with GLMakie
└ Loading RelocatableFolders into GLMakie from project dependency, future warnings for GLMakie are suppressed.

julia> GLMakie.ASSETS_DIR.files
Dict{String, Vector{UInt8}} with 30 entries:
  "heatmap.vert"                     => [0x7b, 0x7b, 0x47, 0x4c, 0x53, 0x4c, 0x5f, 0x56, 0x45, 0x52  …  0x20, 0x30, 0x2c, 0x20, 0x31, 0x29, 0x3b, 0x0a, 0x7d, 0x0a]
  "postprocessing\\fxaa.frag"        => [0x7b, 0x7b, 0x47, 0x4c, 0x53, 0x4c, 0x5f, 0x56, 0x45, 0x52  …  0x20, 0x29, 0x2e, 0x72, 0x67, 0x62, 0x3b, 0x0a, 0x7d, 0x0a]
  "sprites.vert"                     => [0x7b, 0x7b, 0x47, 0x4c, 0x53, 0x4c, 0x5f, 0x56, 0x45, 0x52  …  0x64, 0x65, 0x78, 0x2b, 0x31, 0x29, 0x3b, 0x0a, 0x7d, 0x0a]
  "line_segment.vert"                => [0x7b, 0x7b, 0x47, 0x4c, 0x53, 0x4c, 0x5f, 0x56, 0x45, 0x52  …  0x65, 0x72, 0x74, 0x65, 0x78, 0x29, 0x3b, 0x0a, 0x7d, 0x0a]
  "parametric.frag"                  => [0x7b, 0x7b, 0x47, 0x4c, 0x53, 0x4c, 0x5f, 0x56, 0x45, 0x52  …  0x72, 0x74, 0x6f, 0x79, 0x2f, 0x0a, 0x0a, 0x2a, 0x2f, 0x0a]
  "intensity.frag"                   => [0x7b, 0x7b, 0x47, 0x4c, 0x53, 0x4c, 0x5f, 0x56, 0x45, 0x52  …  0x78, 0x2c, 0x20, 0x30, 0x29, 0x29, 0x3b, 0x0a, 0x7d, 0x0a]
  "volume.vert"                      => [0x7b, 0x7b, 0x47, 0x4c, 0x53, 0x4c, 0x5f, 0x56, 0x45, 0x52  …  0x64, 0x5f, 0x76, 0x65, 0x72, 0x74, 0x3b, 0x0a, 0x7d, 0x0a]
  "particles.vert"                   => [0x7b, 0x7b, 0x47, 0x4c, 0x53, 0x4c, 0x5f, 0x56, 0x45, 0x52  …  0x69, 0x74, 0x69, 0x6f, 0x6e, 0x29, 0x3b, 0x0a, 0x7d, 0x0a]
  "parametric.vert"                  => [0x7b, 0x7b, 0x47, 0x4c, 0x53, 0x4c, 0x5f, 0x56, 0x45, 0x52  …  0x20, 0x30, 0x2c, 0x20, 0x31, 0x29, 0x3b, 0x0a, 0x7d, 0x0a]
  "dots.vert"                        => [0x7b, 0x7b, 0x47, 0x4c, 0x53, 0x4c, 0x5f, 0x56, 0x45, 0x52  …  0x65, 0x72, 0x74, 0x65, 0x78, 0x29, 0x3b, 0x0a, 0x7d, 0x0a]
  "standard.vert"                    => [0x7b, 0x7b, 0x47, 0x4c, 0x53, 0x4c, 0x5f, 0x56, 0x45, 0x52  …  0x69, 0x74, 0x69, 0x6f, 0x6e, 0x29, 0x3b, 0x0a, 0x7d, 0x0a]
  "volume.frag"                      => [0x7b, 0x7b, 0x47, 0x4c, 0x53, 0x4c, 0x5f, 0x56, 0x45, 0x52  …  0x64, 0x2c, 0x20, 0x30, 0x29, 0x29, 0x3b, 0x0a, 0x7d, 0x0a]
  "postprocessing\\SSAO_blur.frag"   => [0x7b, 0x7b, 0x47, 0x4c, 0x53, 0x4c, 0x5f, 0x56, 0x45, 0x52  …  0x2c, 0x20, 0x31, 0x2e, 0x30, 0x29, 0x3b, 0x0a, 0x7d, 0x0a]
  "postprocessing\\fullscreen.vert"  => [0x7b, 0x7b, 0x47, 0x4c, 0x53, 0x4c, 0x5f, 0x56, 0x45, 0x52  …  0x32, 0x28, 0x30, 0x2c, 0x31, 0x29, 0x3b, 0x0a, 0x7d, 0x0a]
  "surface.vert"                     => [0x7b, 0x7b, 0x47, 0x4c, 0x53, 0x4c, 0x5f, 0x56, 0x45, 0x52  …  0x69, 0x74, 0x69, 0x6f, 0x6e, 0x29, 0x3b, 0x0a, 0x7d, 0x0a]
  "lines.geom"                       => [0x2f, 0x2f, 0x20, 0x2d, 0x2d, 0x2d, 0x2d, 0x2d, 0x2d, 0x2d  …  0x72, 0x61, 0x74, 0x69, 0x6f, 0x29, 0x3b, 0x0a, 0x7d, 0x0a]
  "postprocessing\\copy.frag"        => [0x7b, 0x7b, 0x47, 0x4c, 0x53, 0x4c, 0x5f, 0x56, 0x45, 0x52  …  0x20, 0x3d, 0x20, 0x31, 0x2e, 0x30, 0x3b, 0x0a, 0x7d, 0x0a]
  "sprites.geom"                     => [0x7b, 0x7b, 0x47, 0x4c, 0x53, 0x4c, 0x5f, 0x56, 0x45, 0x52  …  0x74, 0x69, 0x76, 0x65, 0x28, 0x29, 0x3b, 0x0a, 0x7d, 0x0a]
  "postprocessing\\postprocess.frag" => [0x7b, 0x7b, 0x47, 0x4c, 0x53, 0x4c, 0x5f, 0x56, 0x45, 0x52  …  0x74, 0x65, 0x20, 0x6c, 0x75, 0x6d, 0x61, 0x0a, 0x7d, 0x0a]
  "distance_shape.frag"              => [0x7b, 0x7b, 0x47, 0x4c, 0x53, 0x4c, 0x5f, 0x56, 0x45, 0x52  …  0x20, 0x66, 0x5f, 0x69, 0x64, 0x29, 0x3b, 0x0a, 0x7d, 0x0a]
  "standard.frag"                    => [0x7b, 0x7b, 0x47, 0x4c, 0x53, 0x4c, 0x5f, 0x56, 0x45, 0x52  …  0x20, 0x6f, 0x5f, 0x69, 0x64, 0x29, 0x3b, 0x0a, 0x7d, 0x0a]
  "line_segment.geom"                => [0x7b, 0x7b, 0x47, 0x4c, 0x53, 0x4c, 0x5f, 0x56, 0x45, 0x52  …  0x31, 0x29, 0x2c, 0x20, 0x31, 0x29, 0x3b, 0x0a, 0x7d, 0x0a]
  "image.vert"                       => [0x7b, 0x7b, 0x47, 0x4c, 0x53, 0x4c, 0x5f, 0x56, 0x45, 0x52  …  0x74, 0x69, 0x63, 0x65, 0x73, 0x29, 0x3b, 0x0a, 0x7d, 0x0a]
  "texture.frag"                     => [0x7b, 0x7b, 0x47, 0x4c, 0x53, 0x4c, 0x5f, 0x56, 0x45, 0x52  …  0x0a, 0x20, 0x20, 0x20, 0x20, 0x29, 0x3b, 0x0a, 0x7d, 0x0a]
  "dots.frag"                        => [0x7b, 0x7b, 0x47, 0x4c, 0x53, 0x4c, 0x5f, 0x56, 0x45, 0x52  …  0x65, 0x63, 0x74, 0x69, 0x64, 0x29, 0x3b, 0x0a, 0x7d, 0x0a]
  "lines.frag"                       => [0x7b, 0x7b, 0x47, 0x4c, 0x53, 0x4c, 0x5f, 0x56, 0x45, 0x52  …  0x20, 0x66, 0x5f, 0x69, 0x64, 0x29, 0x3b, 0x0a, 0x7d, 0x0a]
  "fragment_output.frag"             => [0x7b, 0x7b, 0x47, 0x4c, 0x53, 0x4c, 0x5f, 0x56, 0x45, 0x52  …  0x72, 0x69, 0x74, 0x65, 0x73, 0x7d, 0x7d, 0x0a, 0x7d, 0x0a]
  "lines.vert"                       => [0x7b, 0x7b, 0x47, 0x4c, 0x53, 0x4c, 0x5f, 0x56, 0x45, 0x52  …  0x65, 0x72, 0x74, 0x65, 0x78, 0x29, 0x3b, 0x0a, 0x7d, 0x0a]
  "util.vert"                        => [0x7b, 0x7b, 0x47, 0x4c, 0x53, 0x4c, 0x5f, 0x56, 0x45, 0x52  …  0x20, 0x6f, 0x66, 0x66, 0x34, 0x29, 0x3b, 0x0a, 0x7d, 0x0a]
  "postprocessing\\SSAO.frag"        => [0x7b, 0x7b, 0x47, 0x4c, 0x53, 0x4c, 0x5f, 0x56, 0x45, 0x52  …  0x3b, 0x0a, 0x20, 0x20, 0x20, 0x20, 0x7d, 0x0a, 0x7d, 0x0a]

[bjarthur]

following the MWE in #2282 (comment) and despite #2282 being merged i still get the following error on MS Windows 10 / WSL2 Ubuntu 22.04 when relocating the system image to another WSL2 Ubuntu 22.04 box:

$ MakieExe/bin/MakieApp
SystemError: opening file "/home/username/.julia/packages/GLMakie/CjUwr/assets/shader/postprocessing/fullscreen.vert": No such file or directory

this is with GLMakie 0.9.2 and PackageCompiler 2.1.15

[bjarthur]

i can also still reproduce this error when relocating a packagecompiled makie app from native windows 10 to native windows 10.

[bjarthur]

this is the change that broke relocatibility with packagecompiler. i'm not sure how to fix it and preserve the speed up commented upon. can we please re-open this issue until we figure this out?

[bjarthur]

the workaround seems to be to add empty!(GLMakie.LOADED_SHADERS) to your code

[SimonDanisch]

This should fix it: #3490