findfont scans all fonts every time #67

ederag · 2021-12-31T16:45:15Z

findfont works well, but takes several seconds - which may be considered fast, considering - here (openSUSE Leap-15.3),
but led to the significant system-dependent overhead found in cesaraustralia/DynamicGrids.jl#194 (comment).

findfont is always scanning all fonts, opening them
to get their name and properties and find the best one.

The issue is that on my system there were over 10_000 fonts installed, most quick to load (median about 7.5 µs),
but some taking much longer (see the diagnosis histogram), so the average is about 270 µs.

diagnosis

using FreeTypeAbstraction: fontpaths, try_load, match_font
font_folders = copy(fontpaths())

t1_ = Float64[]
path_ = String[]
@time for folder in font_folders
           for font in readdir(folder)
               fpath = joinpath(folder, font)
               
               t1 = @elapsed face = try_load(fpath)
               push!(t1_, t1)
               push!(path_, fpath)
               face === nothing && continue
               finalize(face)
           end
       end

  2.855721 seconds (159.18 k allocations: 23.692 MiB)

julia> sum(t1_)
2.832251035

julia> using UnicodePlots
julia> histogram(t1_)
                  ┌                                        ┐ 
   [0.0  , 0.001) ┤█████████████████████████████████  10063  
   [0.001, 0.002) ┤▌ 151                                     
   [0.002, 0.003) ┤▍ 83                                      
   [0.003, 0.004) ┤▎ 32                                      
   [0.004, 0.005) ┤▏ 4                                       
   [0.005, 0.006) ┤▏ 14                                      
   [0.006, 0.007) ┤▏ 7                                       
   [0.007, 0.008) ┤▏ 15                                      
   [0.008, 0.009) ┤▏ 9                                       
   [0.009, 0.01 ) ┤▏ 1                                       
   [0.01 , 0.011) ┤▏ 3                                       
   [0.011, 0.012) ┤  0                                       
   [0.012, 0.013) ┤▏ 7                                       
   [0.013, 0.014) ┤▏ 1                                       
   [0.014, 0.015) ┤▏ 1                                       
                  └                                        ┘ 
                                   Frequency                

julia> using Statistics
julia> mean(t1_)
0.00027256770618804736
julia> median(t1_)
7.4702e-5

There are several solutions on the other side (e.g. removing slow fonts, using FTFont(font_path) to load the specific font directly).

But it might be nice to have a kind of cache so that findfont(font_name) would be even faster on subsequent calls ?

The text was updated successfully, but these errors were encountered:

rafaqz · 2022-01-04T07:40:45Z

Thanks for looking into this. DynamicGrids could allow passing in the FTFont directly, but a cache seems like a cleaner solution, and maybe better if we dont all implement that.

Can we just move the cache in Makie here? How easy would it be to just copy the code over? @SimonDanisch @jkrumbiegel

ederag · 2022-01-04T08:57:22Z

Agreed that centralizing would be better.
I opened a discussion on discourse because it might involve Fontconfig.jl as well (it's very fast),
while Fontconfig.jl and FreeTypeAbstraction.jl should probably not depend on one another.

SimonDanisch · 2022-01-04T10:20:45Z

Sounds like a good idea to make findfont as fast as possible ^^

rafaqz · 2022-01-04T13:34:05Z

Yes depending on FontConfig.jl here does look like the best solution. Im happy to review a PR for this.

ederag · 2022-01-04T14:24:30Z

If there is an agreement about making this package dependent on FontConfig.jl, then
I might be able to create a PR, hopefully this week-end.

fontconfig syntax differs from the current findfont one.
So it would be safer to add the new function load_font described on discourse,
and only after a while, deprecate the current findfont that has proven reliable ?

SimonDanisch · 2022-01-05T10:37:36Z

Yes depending on FontConfig.jl here does look like the best solution.

So I tried pretty hard to use FontConfig instead of rolling our own font search, but I couldn't get it to work reliable on all platforms.
To be honest, I don't really remember the problems anymore, I just remember that I gave up and thought, that a simple findfont would be easier and more reliable.

ederag · 2022-01-05T11:28:13Z

So I tried pretty hard to use FontConfig instead of rolling our own font search,
but I couldn't get it to work reliable on all platforms.

JuliaGraphics/Fontconfig.jl#21 (solved few months after the first findfont commit: 406441d) and JuliaGraphics/Fontconfig.jl#8
have been solved, but that remains a valid concern,
as there are two opened issues, both about installation (JuliaGraphics/Fontconfig.jl#12 and JuliaGraphics/Fontconfig.jl#30).
They looked specific to certain configurations, but still good to have in mind.

SimonDanisch · 2022-01-05T11:49:38Z

Seems like there is at least BinaryBuilder now...But:

JuliaGraphics/Fontconfig.jl#31 (comment)

And I guess someone will need to maintain Fontconfig.jl and update the CIs etc...

timholy · 2022-12-23T15:28:04Z

This seems relevant for JuliaLang/julia#47184 (comment). I inserted some debugging code:

$ git diff
diff --git a/src/findfonts.jl b/src/findfonts.jl
index 0b668a5..fabdc2e 100644
--- a/src/findfonts.jl
+++ b/src/findfonts.jl
@@ -136,9 +136,12 @@ function findfont(

     best_score_so_far = (0, 0, false, typemin(Int))
     best_font = nothing
+    @show font_folders
+    nfonts = 0

     for folder in font_folders
         for font in readdir(folder)
+            nfonts += 1
             fpath = joinpath(folder, font)
             face = try_load(fpath)
             face === nothing && continue
@@ -168,6 +171,7 @@ function findfont(
             end
         end
     end
+    @show nfonts best_font

     return best_font
 end

and got this output:

julia> @time using CairoMakie
 11.008582 seconds (18.42 M allocations: 1.153 GiB, 5.60% gc time, 0.54% compilation time)

julia> @time @eval scatter(0..1, rand(10), markersize=rand(10) .* 20)
font_folders = ["/home/tim/.julia/packages/Makie/Ggejq/assets/fonts", "/usr/share/fonts", "/usr/share/fonts/X11", "/usr/share/fonts/X11/Type1", "/usr/share/fonts/X11/encodings", "/usr/share/fonts/X11/encodings/large", "/usr/share/fonts/X11/misc", "/usr/share/fonts/X11/util", "/usr/share/fonts/cMap", "/usr/share/fonts/cmap", "/usr/share/fonts/cmap/adobe-cns1", "/usr/share/fonts/cmap/adobe-gb1", "/usr/share/fonts/cmap/adobe-japan1", "/usr/share/fonts/cmap/adobe-japan2", "/usr/share/fonts/cmap/adobe-korea1", "/usr/share/fonts/opentype", "/usr/share/fonts/opentype/urw-base35", "/usr/share/fonts/truetype", "/usr/share/fonts/truetype/ancient-scripts", "/usr/share/fonts/truetype/dejavu", "/usr/share/fonts/truetype/droid", "/usr/share/fonts/truetype/lato", "/usr/share/fonts/truetype/noto", "/usr/share/fonts/truetype/unifont", "/usr/share/fonts/type1", "/usr/share/fonts/type1/gsfonts", "/usr/share/fonts/type1/texlive-fonts-recommended", "/usr/share/fonts/type1/urw-base35", "/home/tim/.local/share/fonts", "/usr/local/share/fonts"]
nfonts = 1730
best_font = FTFont (family = TeX Gyre Heros Makie, style = Regular)
font_folders = ["/home/tim/.julia/packages/Makie/Ggejq/assets/fonts", "/usr/share/fonts", "/usr/share/fonts/X11", "/usr/share/fonts/X11/Type1", "/usr/share/fonts/X11/encodings", "/usr/share/fonts/X11/encodings/large", "/usr/share/fonts/X11/misc", "/usr/share/fonts/X11/util", "/usr/share/fonts/cMap", "/usr/share/fonts/cmap", "/usr/share/fonts/cmap/adobe-cns1", "/usr/share/fonts/cmap/adobe-gb1", "/usr/share/fonts/cmap/adobe-japan1", "/usr/share/fonts/cmap/adobe-japan2", "/usr/share/fonts/cmap/adobe-korea1", "/usr/share/fonts/opentype", "/usr/share/fonts/opentype/urw-base35", "/usr/share/fonts/truetype", "/usr/share/fonts/truetype/ancient-scripts", "/usr/share/fonts/truetype/dejavu", "/usr/share/fonts/truetype/droid", "/usr/share/fonts/truetype/lato", "/usr/share/fonts/truetype/noto", "/usr/share/fonts/truetype/unifont", "/usr/share/fonts/type1", "/usr/share/fonts/type1/gsfonts", "/usr/share/fonts/type1/texlive-fonts-recommended", "/usr/share/fonts/type1/urw-base35", "/home/tim/.local/share/fonts", "/usr/local/share/fonts"]
nfonts = 1730
best_font = FTFont (family = TeX Gyre Heros Makie, style = Bold)
 16.750444 seconds (185.45 k allocations: 11.756 MiB, 1.04% compilation time)
FigureAxisPlot()

I think you could add a fonts.jl to Makie that basically does this:

const best_regular = findfonts(...)
const best_bold = findfonts(...)

and then the choice would be precompiled. (You wouldn't call it at runtime at all.)

t-bltg · 2022-12-23T20:43:37Z

~~In addition to finding default fonts at compile time as proposed by @timholy, one could cache fonts instead of looking up the font on every to_font invocation.~~
~~This is the caching mechanism I wrote for UnicodePlots.~~

NVM, fonts are already cached.

jkrumbiegel · 2022-12-23T21:16:42Z

I guess a simple optimization could be just saving the list of font names for all found files in a text file. The font search as it is relies only on family and style name, as I have always found that to be the most reliable way to pick specific font variants. As opposed to trying to make the engine match a font whose name I already know by picking weight values etc. correctly. I used to fight with matplotlib a lot to make it match certain font variants back in the day.

I think most users really just want to select specific fonts and do not need a complicated matching engine. So we don't need to open each file just to read family and style name over and over again. The only thing to work out would be when to invalidate the cache.

t-bltg · 2022-12-23T21:37:23Z

I really don't see how we could reuse FONT_CACHE filled at precompile time, since a FTFont holds a pointer (and that is non-serializable).
However, we can cache the font paths as regular strings during precompilation, and avoid scanning > 1k font directories at runtime.

timholy · 2022-12-24T10:55:11Z

Right, it has to be something durable. Cache the choice, not the result. Reading a single font file will be much faster than reading all of them.

ederag mentioned this issue Jan 3, 2022

savegif slower than GifOutput cesaraustralia/DynamicGrids.jl#194

Closed

ederag mentioned this issue Jan 18, 2022

allow FTFont in TextConfig(font=...) cesaraustralia/DynamicGrids.jl#202

Merged

t-bltg mentioned this issue Jun 22, 2022

fix plot size in png - rework docs JuliaPlots/UnicodePlots.jl#269

Merged

t-bltg linked a pull request Dec 23, 2022 that will close this issue

add cache mechanism #76

Open

jkrumbiegel mentioned this issue Dec 27, 2022

Hardcode Makie's default fonts to avoid findfont latency MakieOrg/Makie.jl#2528

Closed

tecosaur mentioned this issue Jan 31, 2024

Better font-finding heuristics, with a shortcut and a few caches. #84

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

findfont scans all fonts every time #67

findfont scans all fonts every time #67

ederag commented Dec 31, 2021 •

edited

Loading

rafaqz commented Jan 4, 2022

ederag commented Jan 4, 2022 •

edited

Loading

SimonDanisch commented Jan 4, 2022

rafaqz commented Jan 4, 2022

ederag commented Jan 4, 2022

SimonDanisch commented Jan 5, 2022

ederag commented Jan 5, 2022 •

edited

Loading

SimonDanisch commented Jan 5, 2022

timholy commented Dec 23, 2022

t-bltg commented Dec 23, 2022 •

edited

Loading

jkrumbiegel commented Dec 23, 2022 •

edited

Loading

t-bltg commented Dec 23, 2022 •

edited

Loading

timholy commented Dec 24, 2022

findfont scans all fonts every time #67

findfont scans all fonts every time #67

Comments

ederag commented Dec 31, 2021 • edited Loading

rafaqz commented Jan 4, 2022

ederag commented Jan 4, 2022 • edited Loading

SimonDanisch commented Jan 4, 2022

rafaqz commented Jan 4, 2022

ederag commented Jan 4, 2022

SimonDanisch commented Jan 5, 2022

ederag commented Jan 5, 2022 • edited Loading

SimonDanisch commented Jan 5, 2022

timholy commented Dec 23, 2022

t-bltg commented Dec 23, 2022 • edited Loading

jkrumbiegel commented Dec 23, 2022 • edited Loading

t-bltg commented Dec 23, 2022 • edited Loading

timholy commented Dec 24, 2022

ederag commented Dec 31, 2021 •

edited

Loading

ederag commented Jan 4, 2022 •

edited

Loading

ederag commented Jan 5, 2022 •

edited

Loading

t-bltg commented Dec 23, 2022 •

edited

Loading

jkrumbiegel commented Dec 23, 2022 •

edited

Loading

t-bltg commented Dec 23, 2022 •

edited

Loading