-
Notifications
You must be signed in to change notification settings - Fork 11.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Coro] Amortize debug info processing cost in CoroSplit #109032
Open
artempyanykh
wants to merge
15
commits into
llvm:main
Choose a base branch
from
artempyanykh:fast-coro-upstream
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from 1 commit
Commits
Show all changes
15 commits
Select commit
Hold shift + click to select a range
a987688
[NFC][Coro] Add helpers for coro cloning with a TimeTraceScope
artempyanykh 3cb662c
[NFC][Utils] Extract CloneFunctionAttributesInto from CloneFunctionInto
artempyanykh 7b6c39f
[Utils] Extract ProcessSubprogramAttachment from CloneFunctionInto
artempyanykh a611c5e
[NFC][Utils] Remove DebugInfoFinder parameter from CloneBasicBlock
artempyanykh 06c10b1
[NFC][Utils] Clone basic blocks after we're done with metadata in Clo…
artempyanykh fad2d5e
[NFC][Utils] Extract BuildDebugInfoMDMap from CloneFunctionInto
artempyanykh 53d8285
[NFC][Utils] Extract CloneFunctionMetadataInto from CloneFunctionInto
artempyanykh b268200
[NFC][Utils] Extract CloneFunctionBodyInto from CloneFunctionInto
artempyanykh a6cba59
[Utils] Eliminate DISubprogram set from BuildDebugInfoMDMap
artempyanykh 0eefa6a
[NFC] Remove adhoc definition of MDMapT in IRMover
artempyanykh d7f598c
[Utils] Identity map global debug info on first use in CloneFunction*
artempyanykh e14a46f
[Coro] Prebuild a global debug info set and share it between all coro…
artempyanykh d1479b9
[Analysis] Add DebugInfoCache analysis
artempyanykh 4c76540
[Coro] Use DebugInfoCache to speed up cloning in CoroSplitPass
artempyanykh f2ec90a
[Analysis] Add a doxygen comment to DebugInfoCache
artempyanykh File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -135,6 +135,29 @@ void llvm::CloneFunctionAttributesInto(Function *NewFunc, | |
OldAttrs.getRetAttrs(), NewArgAttrs)); | ||
} | ||
|
||
DISubprogram *llvm::ProcessSubprogramAttachment(const Function &F, | ||
CloneFunctionChangeType Changes, | ||
DebugInfoFinder &DIFinder) { | ||
DISubprogram *SPClonedWithinModule = nullptr; | ||
if (Changes < CloneFunctionChangeType::DifferentModule) { | ||
SPClonedWithinModule = F.getSubprogram(); | ||
} | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. LLVM's coding guidelines prescribe not using |
||
if (SPClonedWithinModule) | ||
DIFinder.processSubprogram(SPClonedWithinModule); | ||
|
||
const Module *M = F.getParent(); | ||
if (Changes != CloneFunctionChangeType::ClonedModule && M) { | ||
// Inspect instructions to process e.g. DILexicalBlocks of inlined functions | ||
for (const auto &BB : F) { | ||
for (const auto &I : BB) { | ||
DIFinder.processInstruction(*M, I); | ||
} | ||
} | ||
} | ||
|
||
return SPClonedWithinModule; | ||
} | ||
|
||
// Clone OldFunc into NewFunc, transforming the old arguments into references to | ||
// VMap values. | ||
void llvm::CloneFunctionInto(Function *NewFunc, const Function *OldFunc, | ||
|
@@ -167,23 +190,19 @@ void llvm::CloneFunctionInto(Function *NewFunc, const Function *OldFunc, | |
// duplicate instructions and then freeze them in the MD map. We also record | ||
// information about dbg.value and dbg.declare to avoid duplicating the | ||
// types. | ||
std::optional<DebugInfoFinder> DIFinder; | ||
DebugInfoFinder DIFinder; | ||
|
||
// Track the subprogram attachment that needs to be cloned to fine-tune the | ||
// mapping within the same module. | ||
DISubprogram *SPClonedWithinModule = nullptr; | ||
if (Changes < CloneFunctionChangeType::DifferentModule) { | ||
// Need to find subprograms, types, and compile units. | ||
|
||
assert((NewFunc->getParent() == nullptr || | ||
NewFunc->getParent() == OldFunc->getParent()) && | ||
"Expected NewFunc to have the same parent, or no parent"); | ||
|
||
// Need to find subprograms, types, and compile units. | ||
DIFinder.emplace(); | ||
|
||
SPClonedWithinModule = OldFunc->getSubprogram(); | ||
if (SPClonedWithinModule) | ||
DIFinder->processSubprogram(SPClonedWithinModule); | ||
} else { | ||
// Need to find all the compile units. | ||
|
||
assert((NewFunc->getParent() == nullptr || | ||
NewFunc->getParent() != OldFunc->getParent()) && | ||
"Expected NewFunc to have different parents, or no parent"); | ||
|
@@ -192,19 +211,20 @@ void llvm::CloneFunctionInto(Function *NewFunc, const Function *OldFunc, | |
assert(NewFunc->getParent() && | ||
"Need parent of new function to maintain debug info invariants"); | ||
|
||
// Need to find all the compile units. | ||
DIFinder.emplace(); | ||
} | ||
} | ||
|
||
DISubprogram *SPClonedWithinModule = | ||
ProcessSubprogramAttachment(*OldFunc, Changes, DIFinder); | ||
|
||
// Loop over all of the basic blocks in the function, cloning them as | ||
// appropriate. Note that we save BE this way in order to handle cloning of | ||
// recursive functions into themselves. | ||
for (const BasicBlock &BB : *OldFunc) { | ||
|
||
// Create a new basic block and copy instructions into it! | ||
BasicBlock *CBB = CloneBasicBlock(&BB, VMap, NameSuffix, NewFunc, CodeInfo, | ||
DIFinder ? &*DIFinder : nullptr); | ||
BasicBlock *CBB = | ||
CloneBasicBlock(&BB, VMap, NameSuffix, NewFunc, CodeInfo, nullptr); | ||
|
||
// Add basic block mapping. | ||
VMap[&BB] = CBB; | ||
|
@@ -227,7 +247,7 @@ void llvm::CloneFunctionInto(Function *NewFunc, const Function *OldFunc, | |
} | ||
|
||
if (Changes < CloneFunctionChangeType::DifferentModule && | ||
DIFinder->subprogram_count() > 0) { | ||
DIFinder.subprogram_count() > 0) { | ||
// Turn on module-level changes, since we need to clone (some of) the | ||
// debug info metadata. | ||
// | ||
|
@@ -242,24 +262,24 @@ void llvm::CloneFunctionInto(Function *NewFunc, const Function *OldFunc, | |
|
||
// Avoid cloning types, compile units, and (other) subprograms. | ||
SmallPtrSet<const DISubprogram *, 16> MappedToSelfSPs; | ||
for (DISubprogram *ISP : DIFinder->subprograms()) { | ||
for (DISubprogram *ISP : DIFinder.subprograms()) { | ||
if (ISP != SPClonedWithinModule) { | ||
mapToSelfIfNew(ISP); | ||
MappedToSelfSPs.insert(ISP); | ||
} | ||
} | ||
|
||
// If a subprogram isn't going to be cloned skip its lexical blocks as well. | ||
for (DIScope *S : DIFinder->scopes()) { | ||
for (DIScope *S : DIFinder.scopes()) { | ||
auto *LScope = dyn_cast<DILocalScope>(S); | ||
if (LScope && MappedToSelfSPs.count(LScope->getSubprogram())) | ||
mapToSelfIfNew(S); | ||
} | ||
|
||
for (DICompileUnit *CU : DIFinder->compile_units()) | ||
for (DICompileUnit *CU : DIFinder.compile_units()) | ||
mapToSelfIfNew(CU); | ||
|
||
for (DIType *Type : DIFinder->types()) | ||
for (DIType *Type : DIFinder.types()) | ||
mapToSelfIfNew(Type); | ||
} else { | ||
assert(!SPClonedWithinModule && | ||
|
@@ -313,7 +333,7 @@ void llvm::CloneFunctionInto(Function *NewFunc, const Function *OldFunc, | |
SmallPtrSet<const void *, 8> Visited; | ||
for (auto *Operand : NMD->operands()) | ||
Visited.insert(Operand); | ||
for (auto *Unit : DIFinder->compile_units()) { | ||
for (auto *Unit : DIFinder.compile_units()) { | ||
MDNode *MappedUnit = | ||
MapMetadata(Unit, VMap, RF_None, TypeMapper, Materializer); | ||
if (Visited.insert(MappedUnit).second) | ||
|
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Could you elaborate on "Process"? This is more about collecting debug info, right?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That's right. This function hydrates the passed DebugInfoFinder by visiting relevant components of the function's subprogram attachment. I'll update the name/comment when we get to extracting that commit in a separate PR.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sounds good, now that the other two PRs are open & approved, let's proceed with this commit!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Great, yes! @felipepiovezan a silly question -- now that those PRs are approved, how/when do they get merged? Stacking unmerged PRs on top of each other doesn't have the best UX in Github.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
So I think the best option here is to merge the two PRs that are approved (#112976 and #112948) . Both of those stand on their own and don't depend on each other, so it's fine to merge in w/e order, right?
And then you can grab the third commit (or more, if the subsequent commits are also independent of each other) of this PR and open a new PR (or multiple PRs, if you did the () before), so it will look fine on top of
main
. My understanding is that the fourth commit depends on the third though.Regarding what to do about this umbrella PR in the mean time, you have two options: leave this as is (with the understanding that it is not meant to be merged, merely serve as a reference for all the work in the proposal), or rebase on top of main, which will get rid of the two-already merged commits and force push to your branch
artempyanykh:fast-coro-upstream
, which will mean this PR serves a reference of the work that's left to be merged, also with the understand that we're not merging it.With all that stack, we'd never have to stack unmerged PRs on top of each other, does that make sense?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
By the way, do you have commit access?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That's right, yes.
After the first 2 are merged, extracting the third one in a separate PR is straightforward. With the rest of the stack there's quite a bit of sequencing, so we may need to rinse and repeat.
+1 to this. I was planning to keep rebasing this stack anyway to make sure things keep working end-to-end while the chunks are being extracted and merged.
No, not sure what the process for this is. Why?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sorry for the delay, the LLVM dev conference happened last week and we were mostly out.
Just wondering if you needed help merging the open PRs. I've merged the first two for you.
The process for getting commit access is described here: https://llvm.org/docs/DeveloperPolicy.html#obtaining-commit-access
If you don't want to do that, you can always let the reviewers know you need someone to press the merge button for you