-
Notifications
You must be signed in to change notification settings - Fork 11.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Clang] Fix parsing of reversible type traits in template arguments #95969
Conversation
@llvm/pr-subscribers-clang Author: cor3ntin (cor3ntin) ChangesConstructs like This matches usages in libstdc++, and we can hope Fixes #95598 Full diff: https://github.com/llvm/llvm-project/pull/95969.diff 5 Files Affected:
diff --git a/clang/docs/ReleaseNotes.rst b/clang/docs/ReleaseNotes.rst
index 0f958d213172a..e4cd93a3635d9 100644
--- a/clang/docs/ReleaseNotes.rst
+++ b/clang/docs/ReleaseNotes.rst
@@ -872,7 +872,7 @@ Bug Fixes to C++ Support
between the addresses of two labels (a GNU extension) to a pointer within a constant expression. (#GH95366).
- Fix immediate escalation bugs in the presence of dependent call arguments. (#GH94935)
- Clang now diagnoses explicit specializations with storage class specifiers in all contexts.
-
+- Fix parsing of built-in type-traits such as ``__is_pointer`` in libstdc++ headers. (#GH95598)
Bug Fixes to AST Handling
^^^^^^^^^^^^^^^^^^^^^^^^^
diff --git a/clang/include/clang/Parse/Parser.h b/clang/include/clang/Parse/Parser.h
index d054b8cf0d240..b84d59605cc45 100644
--- a/clang/include/clang/Parse/Parser.h
+++ b/clang/include/clang/Parse/Parser.h
@@ -1877,6 +1877,10 @@ class Parser : public CodeCompletionHandler {
UnaryExprOnly,
PrimaryExprOnly
};
+
+ bool isRevertibleTypeTrait(const IdentifierInfo *Id,
+ clang::tok::TokenKind *Kind = nullptr);
+
ExprResult ParseCastExpression(CastParseKind ParseKind,
bool isAddressOfOperand,
bool &NotCastExpr,
diff --git a/clang/lib/Parse/ParseExpr.cpp b/clang/lib/Parse/ParseExpr.cpp
index eb7447fa038e4..132bf283158d0 100644
--- a/clang/lib/Parse/ParseExpr.cpp
+++ b/clang/lib/Parse/ParseExpr.cpp
@@ -760,6 +760,87 @@ class CastExpressionIdValidator final : public CorrectionCandidateCallback {
};
}
+bool Parser::isRevertibleTypeTrait(const IdentifierInfo *II,
+ tok::TokenKind *Kind) {
+ if (RevertibleTypeTraits.empty()) {
+#define RTT_JOIN(X, Y) X##Y
+#define REVERTIBLE_TYPE_TRAIT(Name) \
+ RevertibleTypeTraits[PP.getIdentifierInfo(#Name)] = RTT_JOIN(tok::kw_, Name)
+
+ REVERTIBLE_TYPE_TRAIT(__is_abstract);
+ REVERTIBLE_TYPE_TRAIT(__is_aggregate);
+ REVERTIBLE_TYPE_TRAIT(__is_arithmetic);
+ REVERTIBLE_TYPE_TRAIT(__is_array);
+ REVERTIBLE_TYPE_TRAIT(__is_assignable);
+ REVERTIBLE_TYPE_TRAIT(__is_base_of);
+ REVERTIBLE_TYPE_TRAIT(__is_bounded_array);
+ REVERTIBLE_TYPE_TRAIT(__is_class);
+ REVERTIBLE_TYPE_TRAIT(__is_complete_type);
+ REVERTIBLE_TYPE_TRAIT(__is_compound);
+ REVERTIBLE_TYPE_TRAIT(__is_const);
+ REVERTIBLE_TYPE_TRAIT(__is_constructible);
+ REVERTIBLE_TYPE_TRAIT(__is_convertible);
+ REVERTIBLE_TYPE_TRAIT(__is_convertible_to);
+ REVERTIBLE_TYPE_TRAIT(__is_destructible);
+ REVERTIBLE_TYPE_TRAIT(__is_empty);
+ REVERTIBLE_TYPE_TRAIT(__is_enum);
+ REVERTIBLE_TYPE_TRAIT(__is_floating_point);
+ REVERTIBLE_TYPE_TRAIT(__is_final);
+ REVERTIBLE_TYPE_TRAIT(__is_function);
+ REVERTIBLE_TYPE_TRAIT(__is_fundamental);
+ REVERTIBLE_TYPE_TRAIT(__is_integral);
+ REVERTIBLE_TYPE_TRAIT(__is_interface_class);
+ REVERTIBLE_TYPE_TRAIT(__is_layout_compatible);
+ REVERTIBLE_TYPE_TRAIT(__is_literal);
+ REVERTIBLE_TYPE_TRAIT(__is_lvalue_expr);
+ REVERTIBLE_TYPE_TRAIT(__is_lvalue_reference);
+ REVERTIBLE_TYPE_TRAIT(__is_member_function_pointer);
+ REVERTIBLE_TYPE_TRAIT(__is_member_object_pointer);
+ REVERTIBLE_TYPE_TRAIT(__is_member_pointer);
+ REVERTIBLE_TYPE_TRAIT(__is_nothrow_assignable);
+ REVERTIBLE_TYPE_TRAIT(__is_nothrow_constructible);
+ REVERTIBLE_TYPE_TRAIT(__is_nothrow_destructible);
+ REVERTIBLE_TYPE_TRAIT(__is_nullptr);
+ REVERTIBLE_TYPE_TRAIT(__is_object);
+ REVERTIBLE_TYPE_TRAIT(__is_pod);
+ REVERTIBLE_TYPE_TRAIT(__is_pointer);
+ REVERTIBLE_TYPE_TRAIT(__is_polymorphic);
+ REVERTIBLE_TYPE_TRAIT(__is_reference);
+ REVERTIBLE_TYPE_TRAIT(__is_referenceable);
+ REVERTIBLE_TYPE_TRAIT(__is_rvalue_expr);
+ REVERTIBLE_TYPE_TRAIT(__is_rvalue_reference);
+ REVERTIBLE_TYPE_TRAIT(__is_same);
+ REVERTIBLE_TYPE_TRAIT(__is_scalar);
+ REVERTIBLE_TYPE_TRAIT(__is_scoped_enum);
+ REVERTIBLE_TYPE_TRAIT(__is_sealed);
+ REVERTIBLE_TYPE_TRAIT(__is_signed);
+ REVERTIBLE_TYPE_TRAIT(__is_standard_layout);
+ REVERTIBLE_TYPE_TRAIT(__is_trivial);
+ REVERTIBLE_TYPE_TRAIT(__is_trivially_assignable);
+ REVERTIBLE_TYPE_TRAIT(__is_trivially_constructible);
+ REVERTIBLE_TYPE_TRAIT(__is_trivially_copyable);
+ REVERTIBLE_TYPE_TRAIT(__is_unbounded_array);
+ REVERTIBLE_TYPE_TRAIT(__is_union);
+ REVERTIBLE_TYPE_TRAIT(__is_unsigned);
+ REVERTIBLE_TYPE_TRAIT(__is_void);
+ REVERTIBLE_TYPE_TRAIT(__is_volatile);
+ REVERTIBLE_TYPE_TRAIT(__reference_binds_to_temporary);
+#define TRANSFORM_TYPE_TRAIT_DEF(_, Trait) \
+ REVERTIBLE_TYPE_TRAIT(RTT_JOIN(__, Trait));
+#include "clang/Basic/TransformTypeTraits.def"
+#undef REVERTIBLE_TYPE_TRAIT
+#undef RTT_JOIN
+ }
+ llvm::SmallDenseMap<IdentifierInfo *, tok::TokenKind>::iterator Known =
+ RevertibleTypeTraits.find(II);
+ if (Known != RevertibleTypeTraits.end()) {
+ if (Kind)
+ *Kind = Known->second;
+ return true;
+ }
+ return false;
+}
+
/// Parse a cast-expression, or, if \pisUnaryExpression is true, parse
/// a unary-expression.
///
@@ -1103,85 +1184,9 @@ ExprResult Parser::ParseCastExpression(CastParseKind ParseKind,
else if (Next.is(tok::l_paren) && Tok.is(tok::identifier) &&
Tok.getIdentifierInfo()->hasRevertedTokenIDToIdentifier()) {
IdentifierInfo *II = Tok.getIdentifierInfo();
- // Build up the mapping of revertible type traits, for future use.
- if (RevertibleTypeTraits.empty()) {
-#define RTT_JOIN(X,Y) X##Y
-#define REVERTIBLE_TYPE_TRAIT(Name) \
- RevertibleTypeTraits[PP.getIdentifierInfo(#Name)] \
- = RTT_JOIN(tok::kw_,Name)
-
- REVERTIBLE_TYPE_TRAIT(__is_abstract);
- REVERTIBLE_TYPE_TRAIT(__is_aggregate);
- REVERTIBLE_TYPE_TRAIT(__is_arithmetic);
- REVERTIBLE_TYPE_TRAIT(__is_array);
- REVERTIBLE_TYPE_TRAIT(__is_assignable);
- REVERTIBLE_TYPE_TRAIT(__is_base_of);
- REVERTIBLE_TYPE_TRAIT(__is_bounded_array);
- REVERTIBLE_TYPE_TRAIT(__is_class);
- REVERTIBLE_TYPE_TRAIT(__is_complete_type);
- REVERTIBLE_TYPE_TRAIT(__is_compound);
- REVERTIBLE_TYPE_TRAIT(__is_const);
- REVERTIBLE_TYPE_TRAIT(__is_constructible);
- REVERTIBLE_TYPE_TRAIT(__is_convertible);
- REVERTIBLE_TYPE_TRAIT(__is_convertible_to);
- REVERTIBLE_TYPE_TRAIT(__is_destructible);
- REVERTIBLE_TYPE_TRAIT(__is_empty);
- REVERTIBLE_TYPE_TRAIT(__is_enum);
- REVERTIBLE_TYPE_TRAIT(__is_floating_point);
- REVERTIBLE_TYPE_TRAIT(__is_final);
- REVERTIBLE_TYPE_TRAIT(__is_function);
- REVERTIBLE_TYPE_TRAIT(__is_fundamental);
- REVERTIBLE_TYPE_TRAIT(__is_integral);
- REVERTIBLE_TYPE_TRAIT(__is_interface_class);
- REVERTIBLE_TYPE_TRAIT(__is_layout_compatible);
- REVERTIBLE_TYPE_TRAIT(__is_literal);
- REVERTIBLE_TYPE_TRAIT(__is_lvalue_expr);
- REVERTIBLE_TYPE_TRAIT(__is_lvalue_reference);
- REVERTIBLE_TYPE_TRAIT(__is_member_function_pointer);
- REVERTIBLE_TYPE_TRAIT(__is_member_object_pointer);
- REVERTIBLE_TYPE_TRAIT(__is_member_pointer);
- REVERTIBLE_TYPE_TRAIT(__is_nothrow_assignable);
- REVERTIBLE_TYPE_TRAIT(__is_nothrow_constructible);
- REVERTIBLE_TYPE_TRAIT(__is_nothrow_destructible);
- REVERTIBLE_TYPE_TRAIT(__is_nullptr);
- REVERTIBLE_TYPE_TRAIT(__is_object);
- REVERTIBLE_TYPE_TRAIT(__is_pod);
- REVERTIBLE_TYPE_TRAIT(__is_pointer);
- REVERTIBLE_TYPE_TRAIT(__is_polymorphic);
- REVERTIBLE_TYPE_TRAIT(__is_reference);
- REVERTIBLE_TYPE_TRAIT(__is_referenceable);
- REVERTIBLE_TYPE_TRAIT(__is_rvalue_expr);
- REVERTIBLE_TYPE_TRAIT(__is_rvalue_reference);
- REVERTIBLE_TYPE_TRAIT(__is_same);
- REVERTIBLE_TYPE_TRAIT(__is_scalar);
- REVERTIBLE_TYPE_TRAIT(__is_scoped_enum);
- REVERTIBLE_TYPE_TRAIT(__is_sealed);
- REVERTIBLE_TYPE_TRAIT(__is_signed);
- REVERTIBLE_TYPE_TRAIT(__is_standard_layout);
- REVERTIBLE_TYPE_TRAIT(__is_trivial);
- REVERTIBLE_TYPE_TRAIT(__is_trivially_assignable);
- REVERTIBLE_TYPE_TRAIT(__is_trivially_constructible);
- REVERTIBLE_TYPE_TRAIT(__is_trivially_copyable);
- REVERTIBLE_TYPE_TRAIT(__is_unbounded_array);
- REVERTIBLE_TYPE_TRAIT(__is_union);
- REVERTIBLE_TYPE_TRAIT(__is_unsigned);
- REVERTIBLE_TYPE_TRAIT(__is_void);
- REVERTIBLE_TYPE_TRAIT(__is_volatile);
- REVERTIBLE_TYPE_TRAIT(__reference_binds_to_temporary);
-#define TRANSFORM_TYPE_TRAIT_DEF(_, Trait) \
- REVERTIBLE_TYPE_TRAIT(RTT_JOIN(__, Trait));
-#include "clang/Basic/TransformTypeTraits.def"
-#undef REVERTIBLE_TYPE_TRAIT
-#undef RTT_JOIN
- }
-
- // If we find that this is in fact the name of a type trait,
- // update the token kind in place and parse again to treat it as
- // the appropriate kind of type trait.
- llvm::SmallDenseMap<IdentifierInfo *, tok::TokenKind>::iterator Known
- = RevertibleTypeTraits.find(II);
- if (Known != RevertibleTypeTraits.end()) {
- Tok.setKind(Known->second);
+ tok::TokenKind Kind;
+ if (isRevertibleTypeTrait(II, &Kind)) {
+ Tok.setKind(Kind);
return ParseCastExpression(ParseKind, isAddressOfOperand,
NotCastExpr, isTypeCast,
isVectorLiteral, NotPrimaryExpression);
diff --git a/clang/lib/Parse/ParseTentative.cpp b/clang/lib/Parse/ParseTentative.cpp
index ea17c3e3252ec..dac7a6a662f6e 100644
--- a/clang/lib/Parse/ParseTentative.cpp
+++ b/clang/lib/Parse/ParseTentative.cpp
@@ -1385,6 +1385,12 @@ Parser::isCXXDeclarationSpecifier(ImplicitTypenameContext AllowImplicitTypename,
if (!getLangOpts().ObjC && Next.is(tok::identifier))
return TPResult::True;
+ if (Next.is(tok::l_paren) &&
+ Tok.getIdentifierInfo()->hasRevertedTokenIDToIdentifier() &&
+ isRevertibleTypeTrait(Tok.getIdentifierInfo())) {
+ return TPResult::False;
+ }
+
if (Next.isNot(tok::coloncolon) && Next.isNot(tok::less)) {
// Determine whether this is a valid expression. If not, we will hit
// a parse error one way or another. In that case, tell the caller that
diff --git a/clang/test/Parser/cxx-template-argument.cpp b/clang/test/Parser/cxx-template-argument.cpp
index 667dd8698435b..3c2169f86d6e7 100644
--- a/clang/test/Parser/cxx-template-argument.cpp
+++ b/clang/test/Parser/cxx-template-argument.cpp
@@ -141,3 +141,15 @@ namespace r360308_regression {
return a == b;
}
}
+
+namespace GH95598 {
+template<typename _Tp, bool _IsPtr = __is_pointer(_Tp)>
+struct __is_pointer {};
+// expected-warning@-1 {{keyword '__is_pointer' will be made available as an identifier for the remainder of the translation unit}}
+
+template<bool>
+struct ts{};
+
+template<typename _Tp>
+ struct is_pointer : ts<__is_pointer(_Tp)> {};
+}
|
namespace GH95598 { | ||
template<typename _Tp, bool _IsPtr = __is_pointer(_Tp)> | ||
struct __is_pointer {}; | ||
// expected-warning@-1 {{keyword '__is_pointer' will be made available as an identifier for the remainder of the translation unit}} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This diagnostic is very confusing because it implies the name is only available as an identifier when in reality it's available as either an identifier or a keyword. This seems to be the original functionality though: https://godbolt.org/z/WzTs4T31j (based on the changes in 47642d2 which added this diagnostic).
So I think your changes are aligned with the original intent of the diagnostic, but I've CCed @zygoloid and @rnk just in case others from this era recall differently.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think we've incrementally ended up in a bad place. Here's my memory of what happened:
Originally, the idea was: we provide builtins such as __is_pointer
for use by the standard library. But for a bunch of those, libstdc++ provides a type with the same name that works it out for itself. OK, we can work around that: if we see a declaration with that name, then the standard library isn't using our builtin, so we treat it as a non-keyword so that libstdc++ doesn't break.
But then it turns out that boost uses these things as keywords directly, rather than using the standard library functionality. To keep both that and libstdc++ working at the same time, we treat the keyword followed by an open paren as having the builtin meaning even when the keyword has been reverted to being a non-keyword.
And now we're noticing that we also need special behavior in other parts of the parser to treat the keyword followed by an open paren as a special case.
I don't think we should be extending this hack further. I've suggested on the libstdc++ bug report that this be fixed in libstdc++ instead, by stopping using our keywords as their type names. Maybe one day we can then remove it entirely.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
One thing we could consider doing would be removing the entire "revertible keyword" mechanism, and instead, in the lexer, classifying these tokens as keywords only if the next pp-token is an lparen
. That would probably simplify the handling of these keywords everywhere, and would happen to give the outcome that libstdc++ is expecting in this case too. (It's a breaking change for some weird corner cases, such as producing the keyword via macro expansion and things like bool __is_pointer(true);
, but hopefully those things don't happen in practice!)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@zygoloid https://github.com/itanium-cxx-abi/cxx-abi/blob/07427ac5b6ff0799c47e8c754609cf8220c05cc0/gcc-tinfo2.cc#L373 (never mind, this turns out to have never compiled with Clang in the first place. GCC seems to support __is_pointer
-as-an-identifier in a lot more positions than Clang though: https://godbolt.org/z/KGTWa1hqY )
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@zygoloid I prototyped something #96097
One of the test has a default constructor declaration, ie is_pod()
, which the lexer approach would struggle to cope with. Let me know what you think
The alternative is to keep status quo or go with the current PR and change the warning to a depreciation warning.
My concern is that until we support older libstdc++ (which will be the case for a long time), we won't be able to rip the hack out
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for the historical information @zygoloid!
I don't think we should be extending this hack further. I've suggested on the libstdc++ bug report that this be fixed in libstdc++ instead, by stopping using our keywords as their type names. Maybe one day we can then remove it entirely.
I tend to agree; if we can avoid continuing to extend this hack, I think we should. That said, I'd be curious whether deprecation is highly disruptive or not; my intuition is that we'd struggle with this one because people tend to use boost (and certainly libstdc++) as system headers, and we suppress diagnostics by default in system headers. So I'd expect this deprecation to take a long while before we could remove the hacks due to the long tail of not wanting to break system headers.
The ctor situation would be annoying, but I think the ctor and dtor are the only places we'd have to put in a special hack for "got a keyword where we expected an identifier", so perhaps the blast radius is sufficiently limited for that to be a reasonable approach?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There are now patches posted for libstdc++ to avoid using our type trait keywords as identifiers.
if (Next.is(tok::l_paren) && | ||
Tok.getIdentifierInfo()->hasRevertedTokenIDToIdentifier() && | ||
isRevertibleTypeTrait(Tok.getIdentifierInfo())) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You can set the correct tok::TokenKind
here too since it will be set later anyway when this is re-parsed as an expression
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I would rather not mutate anything in this function
@@ -1385,6 +1385,12 @@ Parser::isCXXDeclarationSpecifier(ImplicitTypenameContext AllowImplicitTypename, | |||
if (!getLangOpts().ObjC && Next.is(tok::identifier)) | |||
return TPResult::True; | |||
|
|||
if (Next.is(tok::l_paren) && |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We might need a comment above this like:
// If this identifier was reverted from a token ID, and the next token
// is a '(', this is likely to be a use of a type trait. If it is, this
// can never be a type name.
Stumbled upon a related bug #29631 |
I think we should go with this patch and independently make sure to fix the warning |
Constructs like `__is_pointer(Foo)` are never considered to be functions declarations. This matches usages in libstdc++, and we can hope no one else redefine these reserved identifiers. Fixes llvm#95598
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Presuming precommit CI comes back without surprises, this LGTM!
…lvm#95969) Constructs like `__is_pointer(Foo)` are never considered to be functions declarations. This matches usages in libstdc++, and we can hope no one else redefine these reserved identifiers. Fixes llvm#95598
…lvm#95969) Constructs like `__is_pointer(Foo)` are never considered to be functions declarations. This matches usages in libstdc++, and we can hope no one else redefine these reserved identifiers. Fixes llvm#95598
…ist (#100572) `__is_layout_compatible` was added in Clang 19 (#81506), and at that time it wasn't entirely clear whether it should be a revertible type trait or not. We decided to follow the example of other type traits. Since then #95969 happened, and now we know that we don't want new revertible type traits. This patch removes `__is_layout_compatible` from revertible type traits list, and leaves a comment what revertible type traits are, and that new type traits should not be added there. The intention is to also cherry-pick this to 19 branch.
…ist (llvm#100572) `__is_layout_compatible` was added in Clang 19 (llvm#81506), and at that time it wasn't entirely clear whether it should be a revertible type trait or not. We decided to follow the example of other type traits. Since then llvm#95969 happened, and now we know that we don't want new revertible type traits. This patch removes `__is_layout_compatible` from revertible type traits list, and leaves a comment what revertible type traits are, and that new type traits should not be added there. The intention is to also cherry-pick this to 19 branch. (cherry picked from commit 3295d37)
…ist (#100572) Summary: `__is_layout_compatible` was added in Clang 19 (#81506), and at that time it wasn't entirely clear whether it should be a revertible type trait or not. We decided to follow the example of other type traits. Since then #95969 happened, and now we know that we don't want new revertible type traits. This patch removes `__is_layout_compatible` from revertible type traits list, and leaves a comment what revertible type traits are, and that new type traits should not be added there. The intention is to also cherry-pick this to 19 branch. Test Plan: Reviewers: Subscribers: Tasks: Tags: Differential Revision: https://phabricator.intern.facebook.com/D60250602
…ist (llvm#100572) `__is_layout_compatible` was added in Clang 19 (llvm#81506), and at that time it wasn't entirely clear whether it should be a revertible type trait or not. We decided to follow the example of other type traits. Since then llvm#95969 happened, and now we know that we don't want new revertible type traits. This patch removes `__is_layout_compatible` from revertible type traits list, and leaves a comment what revertible type traits are, and that new type traits should not be added there. The intention is to also cherry-pick this to 19 branch. (cherry picked from commit 3295d37)
Constructs like
__is_pointer(Foo)
are never considered to be functions declarations.This matches usages in libstdc++, and we can hope
no one else redefine these reserved identifiers.
Fixes #95598