Mark feature writer: more specific anchors should override less specific ones

[simoncozens]

Let's suppose you have pair of anchors "top"/"_top" which attach most top marks to most base characters. But in certain combinations, you want to override that general pairing. (I'm looking at Noto Sans Khmer, which has a top anchor in uni1787 but a separate anchor to be used only for uni17C6.)

When you compile such a font where a mark/base pair is attached by more than one anchor, ufo2ft will tell you that the "last" anchor will prevail, but which anchor ends up being considered "last" is essentially uncontrollable. This is because the anchors are grouped into non-overlapping sets first, by a very clever algorithm:

ufo2ft/Lib/ufo2ft/featureWriters/markFeatureWriter.py

Lines 454 to 462 in f4f49e4

	# To compute the number of lookups that we need to build, we want
	# the minimum number of lookups such that, whenever a mark glyph
	# belongs to several mark classes, these classes are not in the same
	# lookup. A trivial solution is to make 1 lookup per mark class
	# but that's a bit wasteful, we might be able to do better by grouping
	# mark classes that do not conflict.
	# This is a graph coloring problem: the graph nodes are mark classes,
	# edges are between classes that would conflict and the colors are
	# the lookups in which they can go.

Unfortunately, this means that lots of small groups of anchors might end up being put together into a single big group - or alternatively, they might end up being put together into a single medium-sized group, if there are only a few small groups of anchors.

Hence there is no way in this algorithm to reliable sort the groups by size to ensure that more-specific anchors override a set of general ones.

[anthrotype]

sort the groups by size to ensure that more-specific anchors override a set of general ones

I'm not sure exactly what you're proposing. In another related issue, @khaledhosny has proposed to disable this grouping altogether and go back to a simple one-lookup per mark class which in his particular case even leads to smaller file size (which was the point of grouping I suppose)

#762

[anthrotype]

Oh maybe your "more specific anchors should override less specific ones" could be implemented by following @behdad's suggestion below? #563 (comment)

if a mark can be attached via multiple anchors, then for any two such anchors a and b, (eg a = 'top' and b = 'top.alt'), then if b.startswith(a) and len(b) > len(a) + 1 and not b[len(a)].isalnum() remove a from the candidates.
If more than one anchor is left, warn. Output anchors in alphanumeric sorted order.

I like that actually

[moyogo]

Defaulting to whatever heuristic we have is fine, but it will still break in some cases. There really should be a lib key or a mark feature writer parameter for an order of anchors to use.

[belluzj]

I agree with Denis, @simoncozens you're the second person after @khaledhosny who says you rely on what I would call "ambiguous" anchor combos with the expectation that fontmake should be able to resolve the ambiguity using some heuristic that will give you "what you want".

IMO so far the only solution was to remove the ambiguity from the source files, in order to ensure that for each pair of glyphs there's at most one anchor/_anchor pair that matches between them.

But now that you're both actively relying on this, I think the new "proper" solution is to formalize by a new field on anchors in the sources, defining a list of priorities in case of ambiguity, either like Denis suggests with a lib key, or with some new data on the anchor element in the UFO standard.

[simoncozens]

It sounds like you're implying that I want some magical do-what-I-mean heuristic to get the anchors in the right order. But what I actually want is very clear and well-defined: narrower contexts override wider ones.

To fix this in the sources would mean that to override a single base-mark anchor pair with a font of 200 bases, I would have to add additional anchors to all 200 of the bases, 199 of which would replicate an existing anchor. To fix another pair with a different base would require another 200 additional anchors, 199 of which are replicating existing anchors. And have you ever tried editing a font file when three of the anchor marks sit on top of each other? It's quite frustrating. Now imagine doing that while, all the time, you are thinking "The only reason I have to deal with this madness is because the font compiler doesn't follow a very clear and well-defined mechanism for choosing the right anchor".

[anthrotype]

a very clear and well-defined mechanism for choosing the right anchor

sure, how does one chose the "right anchor" then? We need something more specific than "narrower contexts override wider ones". Let's start with a concrete example that currently can't be encoded in the way you want, and describe how you propose for it to be handled.

[belluzj]

@simoncozens sorry for my tone, I didn't mean to imply anything, I wanted to say that I've now realized that considering this ambiguous and asking the designer to fix it in the source is not the solution, and instead we need to formalize the desired behaviour with data (and like you say, a formal description of the mechanism you have in mind).

[khaledhosny]

My code currently depends on the lookups being sorted based on anchor names, which is not very flexible but at least predictable. Any other predictable algorithm is (like Behdad’s one mentioned above in #591 (comment)) is also fine by me.