-
Notifications
You must be signed in to change notification settings - Fork 12
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
parser: workaround cursor.get_tokens() issue with macro expansions
There's a bug in clang that causes cursor.get_tokens() to sometimes fail when there are macro expansions in the cursor's extent. Just having a 'bool x' variable or struct member fails to tokenize, because bool is a macro rather than a typedef in stdbool.h. I haven't been able to come up with a reproducer that would hit this with our current usage of cursor.get_tokens(). However, it's obvious when trying to tokenize said 'bool x'. Rather than wait for us hitting subtle bugs, defensively work around the issue. The issue seems to be related to something going awry in the cursor's extent. Simply recreating the extent works. The resulting new extent has the same __repr__, but they differ under the hood. The new one works, the old one doesn't. Try the regular cursor.get_tokens() first, and if that produces no tokens, fall back to recreating the extent, and getting the tokens from the translation unit. This is likely caused by clang issue [1], or it's closely related. See also [2]. [1] llvm/llvm-project#43451 [2] https://stackoverflow.com/questions/16786767/obtain-original-unexpanded-macro-text-using-libclang
- Loading branch information
Showing
1 changed file
with
34 additions
and
8 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters