Function in non-type template param causes undefined reference with debug build

[seanbaxter]

Bugzilla Link	45804
Version	trunk
OS	Linux
CC	@adrian-prantl,@dwblaikie,@DougGregor,@JDevlieghere,@pogo59,@zygoloid

Extended Description

void func1(int x);

template<void(&f)(int)>
void func2() { }

int main() {
  func2<func1>();
}

clang++-9 ref.cxx -g
/tmp/ref-970157.o:(.debug_info+0x67): undefined reference to `func1(int)'
clang: error: linker command failed with exit code 1 (use -v to see invocation

The LLVM function symbol in DITemplateValueParameter apparently ODR uses that function:

!DITemplateValueParameter(name: "f", type: !18, value: void (i32)* @_Z5func1i)

For non-type template params that bind to a declaration, it seems you can just pass an appropriately typed nullptr as the Val argument of DIBuilder::createTemplateValueParameter:

!DITemplateValueParameter(name: "f", type: !18, value: void (i32)* null)

Of course when you pass null you lose introspection of the parameter in the debugger.

gcc manages to get the best of both worlds somehow:
If func1 is undefined the program still links, but there's no introspection of f in the debugger.
If func2 is externally defined.. or its defined as inline and ODR used inside the TU, it both links and gives you introspection in the debugger.

[dwblaikie]

If func2 is externally defined.. or its defined as inline and ODR used inside the TU, it both links and gives you introspection in the debugger.

From my testing it looks like that only works if func1 is externally defined in the same translation unit as the template? If func1 is defined in some other translation unit - GCC's debug info won't describe the DW_AT_location of the non-type template parameter, right?

Yeah, it's possible to implement that in Clang if someone wants to, but it does reduce the debuggability in cases like that ^.

[seanbaxter]

Is this something where we can set the linkage of the locally-undefined function to extern_weak and assume it'll either get filled in or set to null?

[dwblaikie]

Is this something where we can set the linkage of the locally-undefined
function to extern_weak and assume it'll either get filled in or set to null?

Maybe? You could see if you could write that assembly by hand & make it work? (taking assembly output from Clang to start with)

Though I'd be concerned that it'd resolve to null in some cases which would give the user/debugger incorrect information. It's not null, it's actually not present at all.

And I'm not sure DWARF gives us a way to use a DWARF location expression that returns a "no result" result. (ie: we couldn't even wrap the address in a DWARF expression that tests for nullness, and if null, says "actually, there's no location here")

[zygoloid]

The reference to func1 in comment#0 does constitute an odr-use, so it's not strictly-speaking wrong for us to reject this. And it seems like it's not possible in general to avoid -g changing the linking behavior here: even if func1 is in fact defined, the reference to it from the debug info will presumably prevent LLVM and the linker from discarding it (and the former could presumably also change linking behavior in some cases).

So it seems to me that we have a choice: if we want to avoid the debug info from changing linking behavior even for incorrect programs, then it can't refer to symbols unless the debug info reference is somehow discarded whenever the symbol is (eg, same comdat group). And if we're OK with the debug info changing linking behavior, then this is just a case where that happens to cause us to start rejecting an invalid program.

As an example of what I'm talking about with -g changing linking behavior, consider this related example:

// tu1.cpp
template<typename T> void f() {}
template<void()> struct X {};
X<&f<int>> x;

// tu2.cpp
template<typename T> void f();
int main() { f<int>(); }

This fails to link without -g, because it violates the requirement to define the template function f in every TU where it's implicitly instantiated. But it does link with -g, because the reference to f<int> from the debug info is enough to cause Clang to emit it, and to cause LLVM to retain it.

[dwblaikie]

Admittedly, the "this could change the behavior of the final program" is pretty scary (eg: if the object file that defines the address-taken function is in a library and has global ctors... ) - possibility of heisenbugs and such & might sufficiently motivate the extra loss in debug info even in the good cases (eg: the function is defined, but in another TU)

[zygoloid]

OK, so presumably this amounts to: debug info can only contain a reference to a symbol if either

a) the debug info and the symbol are connected by comdat or similar (eg, debug info for a function can refer to that same function)

or

b) the symbol is a non-discardable symbol defined in the same translation unit as the debug information

... and in particular for (b) we're not saying that it's sufficient for us to know that there must be a definition somewhere in the program, because if there actually isn't then we'd be risking changing the behavior of a (admittedly invalid) program under -g.

It sounds like we have a from-first-principles argument that the GCC behavior described in comment#1 is the right behavior :)

[seanbaxter]

For my own education, do you know where the standard says that passing a function pointer/reference as a template argument counts as ODR usage? Mechanically there's no need to do this if the corresponding template parameter isn't ODR used.

#include <cstdio>

template<typename type_t>
void func() {
  // Make an array of void to break the build during instantiation.
  type_t array[3] { };
}

template<void(&func_ref)()>
void foo() {
  // func_ref isn't used here, so do we really need to instantiate the
  // template parameter? func_ref is really a decl-ref to func, not 
  // a pointer to it.
}

int main() {
  foo<func<void> >();
}

This breaks the major compilers so I'm not doubting it's rightness, but I can't find anything about it in [basic.def.odr].

[zygoloid]

For my own education, do you know where the standard says that passing a
function pointer/reference as a template argument counts as ODR usage?

http://eel.is/c++draft/basic.def.odr#7.sentence-2

[dwblaikie]

Adrian, Paul - you folks have any ideas/thoughts on this - especially whether there's a way to have a DWARF location expression describe "no location" as a result.

It'd seem a bit unfortunate to make non-type template parameters of pointer type only describe their target when the target is in the same translation unit. But may be necessary.

[seanbaxter]

I'm declaring the locally-undefined function named in the template argument as extern_weak. So far, that works great. If it's defined in another TU the symbol gets set and the debugger sees it as you'd hope. Is there a hazard I should be aware of when doing this?

[dwblaikie]

I'm declaring the locally-undefined function named in the template argument
as extern_weak. So far, that works great. If it's defined in another TU the
symbol gets set and the debugger sees it as you'd hope. Is there a hazard I
should be aware of when doing this?

My concern as mentioned in comment #​3, is:

Though I'd be concerned that it'd resolve to null in some cases which would give the user/debugger incorrect information. It's not null, it's actually not present at all.

If you use extern_weak, and the function isn't available, what happens? I assume the address gets resolved to zero - now what happens if there are two instantiations of the template, one with an actual null parameter, one with a non-null-but-undefined function? a user might be confused by the parameter printing out null, even though they're in a non-null specialization of their template perhaps, etc.

[zygoloid]

What happens if one DSO contains an extern_weak reference to a symbol plus a discardable definition of the symbol, and another DSO contains an unresolved reference to the symbol? Does the extern_weak reference at static link time prevent the discardable symbol from being discarded, or is it still discarded if there are no other references?

[seanbaxter]

DSO-2 can find the linkonce_odr function defined in TU2 of DSO-1 that's named by a template argument's extern_weak reference in TU1 of DSO-1. But the linkonce_odr function is kept alive by a non-discardable function in TU2 of DSO-1. During linking, a U can find a W, in other words.

I don't know how to create (in C++) a linkonce_odr function that's not ultimately named by some other non-discardable function in the same TU.

As far as discards go, -O2 gets to those, in most cases, before the static linker sees them.