[BOLT] Hash-based function matching

[shawbyoung]

Using the hashes of binary and profiled functions
to recover functions with changed names.

Test Plan: added
hashing-based-function-matching.test.

[llvmbot]

@llvm/pr-subscribers-bolt

Author: shaw young (shawbyoung)

Changes

Using the hashes of binary and profiled functions
to recover functions with changed names.

Test Plan: tbd

---

Full diff: https://github.com/llvm/llvm-project/pull/95821.diff

1 Files Affected:

• (modified) bolt/lib/Profile/YAMLProfileReader.cpp (+17)

diff --git a/bolt/lib/Profile/YAMLProfileReader.cpp b/bolt/lib/Profile/YAMLProfileReader.cpp
index f25f59201f1cd..f0fcb1c130002 100644
--- a/bolt/lib/Profile/YAMLProfileReader.cpp
+++ b/bolt/lib/Profile/YAMLProfileReader.cpp
@@ -415,6 +415,23 @@ Error YAMLProfileReader::readProfile(BinaryContext &BC) {
     if (!YamlBF.Used && BF && !ProfiledFunctions.count(BF))
       matchProfileToFunction(YamlBF, *BF);
 
+  // Uses the strict hash of profiled and binary functions to match functions
+  // that are not matched by name or common name.
+  std::unordered_map<size_t, BinaryFunction*> StrictBinaryFunctionHashes;
+  StrictBinaryFunctionHashes.reserve(BC.getBinaryFunctions().size());
+
+  for (auto& [_, BF] : BC.getBinaryFunctions()) {
+    StrictBinaryFunctionHashes[BF.getHash()] = &BF;
+  }
+
+  for (auto YamlBF : YamlBP.Functions) {
+    auto It = StrictBinaryFunctionHashes.find(YamlBF.Hash);
+    if (It != StrictBinaryFunctionHashes.end() && !ProfiledFunctions.count(It->second)) {
+      auto *BF = It->second;
+      matchProfileToFunction(YamlBF, *BF);
+    }
+  }
+
   for (yaml::bolt::BinaryFunctionProfile &YamlBF : YamlBP.Functions)
     if (!YamlBF.Used && opts::Verbosity >= 1)
       errs() << "BOLT-WARNING: profile ignored for function " << YamlBF.Name

[aaupov]

Looks very good, but please address some minor issues.

[WenleiHe]

What is captured in strict hash?

For reference, function hash for compiler PGO isn't very strict, and it's not strong enough for small functions. So we use comprehensive graph matching in addition to hash matching to recover renamed function/profile pair. Change in #95135 cc @wlei-llvm

[shawbyoung]

In BOLT, a strict hash captures a binary function's block order as well as each blocks' instructions & opcodes. So, the goal of this PR is to recover block-identical functions that have been simply been renamed.

[aaupov]

LG overall, with a couple of nits.

[llvm-ci]

LLVM Buildbot has detected a new failure on builder bolt-x86_64-ubuntu-shared running on bolt-worker while building bolt at step 5 "build-bolt".

Full details are available at: https://lab.llvm.org/buildbot/#/builders/151/builds/367

Here is the relevant piece of the build log for the reference:

Step 5 (build-bolt) failure: build (failure)
...
1.012 [10/6/12] Linking CXX shared library lib/libLLVMBOLTTargetX86.so.19.0git
1.018 [9/6/13] Creating library symlink lib/libLLVMBOLTTargetAArch64.so
1.018 [9/5/14] Creating library symlink lib/libLLVMBOLTTargetX86.so
1.019 [9/4/15] Creating library symlink lib/libLLVMBOLTTargetRISCV.so
1.026 [9/3/16] Linking CXX shared library lib/libLLVMBOLTPasses.so.19.0git
1.032 [8/3/17] Creating library symlink lib/libLLVMBOLTPasses.so
1.058 [7/3/18] Linking CXX shared library lib/libLLVMBOLTRuntimeLibs.so.19.0git
1.064 [6/3/19] Creating library symlink lib/libLLVMBOLTRuntimeLibs.so
6.552 [6/2/20] Building CXX object tools/bolt/lib/Profile/CMakeFiles/LLVMBOLTProfile.dir/YAMLProfileReader.cpp.o
6.586 [5/2/21] Linking CXX shared library lib/libLLVMBOLTProfile.so.19.0git
FAILED: lib/libLLVMBOLTProfile.so.19.0git 
: && /usr/bin/c++ -fPIC -fPIC -fno-semantic-interposition -fvisibility-inlines-hidden -Werror=date-time -fno-lifetime-dse -Wall -Wextra -Wno-unused-parameter -Wwrite-strings -Wcast-qual -Wno-missing-field-initializers -pedantic -Wno-long-long -Wimplicit-fallthrough -Wno-uninitialized -Wno-nonnull -Wno-class-memaccess -Wno-redundant-move -Wno-pessimizing-move -Wno-noexcept-type -Wdelete-non-virtual-dtor -Wsuggest-override -Wno-comment -Wno-misleading-indentation -Wctad-maybe-unsupported -fdiagnostics-color -ffunction-sections -fdata-sections -O3 -DNDEBUG  -Wl,-z,defs -Wl,-z,nodelete -fuse-ld=lld -Wl,--color-diagnostics   -Wl,--gc-sections -shared -Wl,-soname,libLLVMBOLTProfile.so.19.0git -o lib/libLLVMBOLTProfile.so.19.0git tools/bolt/lib/Profile/CMakeFiles/LLVMBOLTProfile.dir/BoltAddressTranslation.cpp.o tools/bolt/lib/Profile/CMakeFiles/LLVMBOLTProfile.dir/DataAggregator.cpp.o tools/bolt/lib/Profile/CMakeFiles/LLVMBOLTProfile.dir/DataReader.cpp.o tools/bolt/lib/Profile/CMakeFiles/LLVMBOLTProfile.dir/Heatmap.cpp.o tools/bolt/lib/Profile/CMakeFiles/LLVMBOLTProfile.dir/StaleProfileMatching.cpp.o tools/bolt/lib/Profile/CMakeFiles/LLVMBOLTProfile.dir/YAMLProfileReader.cpp.o tools/bolt/lib/Profile/CMakeFiles/LLVMBOLTProfile.dir/YAMLProfileWriter.cpp.o  -Wl,-rpath,"\$ORIGIN/../lib:/home/worker/bolt-worker2/bolt-x86_64-ubuntu-shared/build/lib:"  lib/libLLVMBOLTCore.so.19.0git  lib/libLLVMBOLTUtils.so.19.0git  lib/libLLVMTransformUtils.so.19.0git  lib/libLLVMSupport.so.19.0git  -Wl,-rpath-link,/home/worker/bolt-worker2/bolt-x86_64-ubuntu-shared/build/lib && :
ld.lld: error: undefined symbol: opts::Lite
>>> referenced by YAMLProfileReader.cpp
>>>               tools/bolt/lib/Profile/CMakeFiles/LLVMBOLTProfile.dir/YAMLProfileReader.cpp.o:(llvm::bolt::YAMLProfileReader::readProfile(llvm::bolt::BinaryContext&) (.localalias))
collect2: error: ld returned 1 exit status
19.429 [5/1/22] Building CXX object tools/bolt/lib/Rewrite/CMakeFiles/LLVMBOLTRewrite.dir/RewriteInstance.cpp.o
ninja: build stopped: subcommand failed.