[HLSL] set alwaysinline on HLSL functions #106588
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
HLSL inlines all its functions by default. This uses the alwaysinline attribute to force that in the corresponding pass for user functions by default and overrides the default noinline of some implicit functions. This makes an instance of explicit inlining for buffer subscripts unnecessary. Adds tests for function and constructor inlining and augments some existing tests to verify correct inlining of implicitly created functions as well. incidentally restore RUN line that I believe was mistakenly removed as part of llvm#88918 fixes llvm#89282
llvmbot
added
clang
|
@llvm/pr-subscribers-clang Author: Greg Roth (pow2clk) ChangesHLSL inlines all its functions by default. This uses the alwaysinline attribute to force that in the corresponding pass for user functions by default and overrides the default noinline of some implicit functions. This makes an instance of explicit inlining for buffer subscripts unnecessary. Adds tests for function and constructor inlining and augments some existing tests to verify correct inlining of implicitly created functions as well. incidentally restore RUN line that I believe was mistakenly removed as part of #88918 fixes #89282 Full diff: https://github.com/llvm/llvm-project/pull/106588.diff 10 Files Affected:
diff --git a/clang/lib/CodeGen/CGHLSLRuntime.cpp b/clang/lib/CodeGen/CGHLSLRuntime.cpp
index 4bd7b6ba58de0d..24d126ced0d9f7 100644
--- a/clang/lib/CodeGen/CGHLSLRuntime.cpp
+++ b/clang/lib/CodeGen/CGHLSLRuntime.cpp
@@ -414,9 +414,20 @@ void CGHLSLRuntime::emitEntryFunction(const FunctionDecl *FD,
void CGHLSLRuntime::setHLSLFunctionAttributes(const FunctionDecl *FD,
llvm::Function *Fn) {
- if (FD->isInExportDeclContext()) {
- const StringRef ExportAttrKindStr = "hlsl.export";
- Fn->addFnAttr(ExportAttrKindStr);
+ if (FD) { // "explicit" functions with declarations
+ if (FD->isInExportDeclContext()) {
+ const StringRef ExportAttrKindStr = "hlsl.export";
+ Fn->addFnAttr(ExportAttrKindStr);
+ }
+ // Respect noinline if the explicit functions use it
+ // otherwise default to alwaysinline
+ if (!Fn->hasFnAttribute(Attribute::NoInline))
+ Fn->addFnAttr(llvm::Attribute::AlwaysInline);
+ } else { // "implicit" autogenerated functions with no declaration
+ // Implicit functions might get marked as noinline by default
+ // but we override that for HLSL
+ Fn->removeFnAttr(Attribute::NoInline);
+ Fn->addFnAttr(Attribute::AlwaysInline);
}
}
diff --git a/clang/lib/CodeGen/CodeGenFunction.cpp b/clang/lib/CodeGen/CodeGenFunction.cpp
index a5747283e98058..aceeed0e66d130 100644
--- a/clang/lib/CodeGen/CodeGenFunction.cpp
+++ b/clang/lib/CodeGen/CodeGenFunction.cpp
@@ -1239,9 +1239,9 @@ void CodeGenFunction::StartFunction(GlobalDecl GD, QualType RetTy,
if (getLangOpts().OpenMP && CurCodeDecl)
CGM.getOpenMPRuntime().emitFunctionProlog(*this, CurCodeDecl);
- if (FD && getLangOpts().HLSL) {
+ if (getLangOpts().HLSL) {
// Handle emitting HLSL entry functions.
- if (FD->hasAttr<HLSLShaderAttr>()) {
+ if (FD && FD->hasAttr<HLSLShaderAttr>()) {
CGM.getHLSLRuntime().emitEntryFunction(FD, Fn);
}
CGM.getHLSLRuntime().setHLSLFunctionAttributes(FD, Fn);
diff --git a/clang/lib/Sema/HLSLExternalSemaSource.cpp b/clang/lib/Sema/HLSLExternalSemaSource.cpp
index 9aacbe4ad9548e..0a534d94192560 100644
--- a/clang/lib/Sema/HLSLExternalSemaSource.cpp
+++ b/clang/lib/Sema/HLSLExternalSemaSource.cpp
@@ -290,8 +290,6 @@ struct BuiltinTypeDeclBuilder {
SourceLocation()));
MethodDecl->setLexicalDeclContext(Record);
MethodDecl->setAccess(AccessSpecifier::AS_public);
- MethodDecl->addAttr(AlwaysInlineAttr::CreateImplicit(
- AST, SourceRange(), AlwaysInlineAttr::CXX11_clang_always_inline));
Record->addDecl(MethodDecl);
return *this;
diff --git a/clang/test/CodeGenHLSL/GlobalConstructorFunction.hlsl b/clang/test/CodeGenHLSL/GlobalConstructorFunction.hlsl
index f954c9d2f029f2..b39311ad67cd62 100644
--- a/clang/test/CodeGenHLSL/GlobalConstructorFunction.hlsl
+++ b/clang/test/CodeGenHLSL/GlobalConstructorFunction.hlsl
@@ -1,4 +1,5 @@
-// RUN: %clang_cc1 -triple dxil-pc-shadermodel6.0-compute -emit-llvm -disable-llvm-passes %s -o - | FileCheck %s
+// RUN: %clang_cc1 -triple dxil-pc-shadermodel6.0-compute -emit-llvm -disable-llvm-passes %s -o - | FileCheck %s --check-prefixes=CHECK,NOINLINE
+// RUN: %clang_cc1 -triple dxil-pc-shadermodel6.0-compute -emit-llvm -O0 %s -o - | FileCheck %s --check-prefixes=CHECK,INLINE
int i;
@@ -7,7 +8,7 @@ __attribute__((constructor)) void call_me_first(void) {
}
__attribute__((constructor)) void then_call_me(void) {
- i = 12;
+ i = 13;
}
__attribute__((destructor)) void call_me_last(void) {
@@ -21,11 +22,21 @@ void main(unsigned GI : SV_GroupIndex) {}
// CHECK-NOT:@llvm.global_ctors
// CHECK-NOT:@llvm.global_dtors
-//CHECK: define void @main()
-//CHECK-NEXT: entry:
-//CHECK-NEXT: call void @"?call_me_first@@YAXXZ"()
-//CHECK-NEXT: call void @"?then_call_me@@YAXXZ"()
-//CHECK-NEXT: %0 = call i32 @llvm.dx.flattened.thread.id.in.group()
-//CHECK-NEXT: call void @"?main@@YAXI@Z"(i32 %0)
-//CHECK-NEXT: call void @"?call_me_last@@YAXXZ"(
-//CHECK-NEXT: ret void
+// CHECK: define void @main()
+// CHECK-NEXT: entry:
+// Verify function constructors are emitted
+// NOINLINE-NEXT: call void @"?call_me_first@@YAXXZ"()
+// NOINLINE-NEXT: call void @"?then_call_me@@YAXXZ"()
+// NOINLINE-NEXT: %0 = call i32 @llvm.dx.flattened.thread.id.in.group()
+// NOINLINE-NEXT: call void @"?main@@YAXI@Z"(i32 %0)
+// NOINLINE-NEXT: call void @"?call_me_last@@YAXXZ"(
+// NOINLINE-NEXT: ret void
+
+// Verify constructor calls are inlined when AlwaysInline is run
+// INLINE-NEXT: alloca
+// INLINE-NEXT: store i32 12
+// INLINE-NEXT: store i32 13
+// INLINE-NEXT: %0 = call i32 @llvm.dx.flattened.thread.id.in.group()
+// INLINE-NEXT: store i32 %
+// INLINE-NEXT: store i32 0
+// INLINE: ret void
diff --git a/clang/test/CodeGenHLSL/GlobalConstructorLib.hlsl b/clang/test/CodeGenHLSL/GlobalConstructorLib.hlsl
index 2c5c4e19c3296d..78f6475462bc47 100644
--- a/clang/test/CodeGenHLSL/GlobalConstructorLib.hlsl
+++ b/clang/test/CodeGenHLSL/GlobalConstructorLib.hlsl
@@ -1,4 +1,5 @@
-// RUN: %clang_cc1 -triple dxil-pc-shadermodel6.3-library -emit-llvm -disable-llvm-passes %s -o - | FileCheck %s
+// RUN: %clang_cc1 -triple dxil-pc-shadermodel6.3-library -emit-llvm -disable-llvm-passes %s -o - | FileCheck %s --check-prefixes=CHECK,NOINLINE
+// RUN: %clang_cc1 -triple dxil-pc-shadermodel6.3-library -emit-llvm -O0 %s -o - | FileCheck %s --check-prefixes=CHECK,INLINE
// Make sure global variable for ctors exist for lib profile.
// CHECK:@llvm.global_ctors
@@ -11,7 +12,11 @@ void FirstEntry() {}
// CHECK: define void @FirstEntry()
// CHECK-NEXT: entry:
-// CHECK-NEXT: call void @_GLOBAL__sub_I_GlobalConstructorLib.hlsl()
+// NOINLINE-NEXT: call void @_GLOBAL__sub_I_GlobalConstructorLib.hlsl()
+// NOINLINE-NEXT: call void @"?FirstEntry@@YAXXZ"()
+// Verify inlining leaves only calls to "llvm." intrinsics
+// INLINE-NOT: call {{[^@]*}} @{{[^l][^l][^v][^m][^\.]}}
+// CHECK: ret void
[shader("compute")]
[numthreads(1,1,1)]
@@ -19,5 +24,15 @@ void SecondEntry() {}
// CHECK: define void @SecondEntry()
// CHECK-NEXT: entry:
-// CHECK-NEXT: call void @_GLOBAL__sub_I_GlobalConstructorLib.hlsl()
-// CHECK-NEXT: call void @"?SecondEntry@@YAXXZ"()
+// NOINLINE-NEXT: call void @_GLOBAL__sub_I_GlobalConstructorLib.hlsl()
+// NOINLINE-NEXT: call void @"?SecondEntry@@YAXXZ"()
+// Verify inlining leaves only calls to "llvm." intrinsics
+// INLINE-NOT: call {{[^@]*}} @{{[^l][^l][^v][^m][^\.]}}
+// CHECK: ret void
+
+
+// Verify the constructor is alwaysinline
+// NOINLINE: ; Function Attrs: {{.*}}alwaysinline
+// NOINLINE-NEXT: define internal void @_GLOBAL__sub_I_GlobalConstructorLib.hlsl() [[IntAttr:\#[0-9]+]]
+
+// NOINLINE: attributes [[IntAttr]] = {{.*}} alwaysinline
diff --git a/clang/test/CodeGenHLSL/GlobalDestructors.hlsl b/clang/test/CodeGenHLSL/GlobalDestructors.hlsl
index 24c3c039fc6192..ea28354222f885 100644
--- a/clang/test/CodeGenHLSL/GlobalDestructors.hlsl
+++ b/clang/test/CodeGenHLSL/GlobalDestructors.hlsl
@@ -1,10 +1,18 @@
-// RUN: %clang_cc1 -triple dxil-pc-shadermodel6.0-compute -std=hlsl202x -emit-llvm -disable-llvm-passes %s -o - | FileCheck %s --check-prefixes=CS,CHECK
-// RUN: %clang_cc1 -triple dxil-pc-shadermodel6.3-library -std=hlsl202x -emit-llvm -disable-llvm-passes %s -o - | FileCheck %s --check-prefixes=LIB,CHECK
+// RUN: %clang_cc1 -triple dxil-pc-shadermodel6.0-compute -std=hlsl202x -emit-llvm -disable-llvm-passes %s -o - | FileCheck %s --check-prefixes=CS,NOINLINE,CHECK
+// RUN: %clang_cc1 -triple dxil-pc-shadermodel6.3-library -std=hlsl202x -emit-llvm -disable-llvm-passes %s -o - | FileCheck %s --check-prefixes=LIB,NOINLINE,CHECK
+// RUN: %clang_cc1 -triple dxil-pc-shadermodel6.0-compute -std=hlsl202x -emit-llvm -O0 %s -o - | FileCheck %s --check-prefixes=INLINE,CHECK
+// RUN: %clang_cc1 -triple dxil-pc-shadermodel6.3-library -std=hlsl202x -emit-llvm -O0 %s -o - | FileCheck %s --check-prefixes=INLINE,CHECK
-// Make sure global variable for dtors exist for lib profile.
+// Tests that constructors and destructors are appropriately generated for globals
+// and that their calls are inlined when AlwaysInline is run
+// but global variables are retained for the library profiles
+
+// Make sure global variable for ctors/dtors exist for lib profile.
+// LIB:@llvm.global_ctors
// LIB:@llvm.global_dtors
-// Make sure global variable for dtors removed for compute profile.
-// CS-NOT:llvm.global_dtors
+// Make sure global variable for ctors/dtors removed for compute profile.
+// CS-NOT:@llvm.global_ctors
+// CS-NOT:@llvm.global_dtors
struct Tail {
Tail() {
@@ -46,22 +54,25 @@ void main(unsigned GI : SV_GroupIndex) {
Wag();
}
-// Make sure global variable for ctors/dtors removed.
-// CHECK-NOT:@llvm.global_ctors
-// CHECK-NOT:@llvm.global_dtors
-//CHECK: define void @main()
-//CHECK-NEXT: entry:
-//CHECK-NEXT: call void @_GLOBAL__sub_I_GlobalDestructors.hlsl()
-//CHECK-NEXT: %0 = call i32 @llvm.dx.flattened.thread.id.in.group()
-//CHECK-NEXT: call void @"?main@@YAXI@Z"(i32 %0)
-//CHECK-NEXT: call void @_GLOBAL__D_a()
-//CHECK-NEXT: ret void
+// CHECK: define void @main()
+// CHECK-NEXT: entry:
+// Verify destructor is emitted
+// NOINLINE-NEXT: call void @_GLOBAL__sub_I_GlobalDestructors.hlsl()
+// NOINLINE-NEXT: %0 = call i32 @llvm.dx.flattened.thread.id.in.group()
+// NOINLINE-NEXT: call void @"?main@@YAXI@Z"(i32 %0)
+// NOINLINE-NEXT: call void @_GLOBAL__D_a()
+// NOINLINE-NEXT: ret void
+// Verify inlining leaves only calls to "llvm." intrinsics
+// INLINE-NOT: call {{[^@]*}} @{{[^l][^l][^v][^m][^\.]}}
+// INLINE: ret void
// This is really just a sanity check I needed for myself to verify that
// function scope static variables also get destroyed properly.
-//CHECK: define internal void @_GLOBAL__D_a()
-//CHECK-NEXT: entry:
-//CHECK-NEXT: call void @"??1Tail@@QAA@XZ"(ptr @"?T@?1??Wag@@YAXXZ@4UTail@@A")
-//CHECK-NEXT: call void @"??1Pupper@@QAA@XZ"(ptr @"?GlobalPup@@3UPupper@@A")
-//CHECK-NEXT: ret void
+// NOINLINE: define internal void @_GLOBAL__D_a() [[IntAttr:\#[0-9]+]]
+// NOINLINE-NEXT: entry:
+// NOINLINE-NEXT: call void @"??1Tail@@QAA@XZ"(ptr @"?T@?1??Wag@@YAXXZ@4UTail@@A")
+// NOINLINE-NEXT: call void @"??1Pupper@@QAA@XZ"(ptr @"?GlobalPup@@3UPupper@@A")
+// NOINLINE-NEXT: ret void
+
+// NOINLINE: attributes [[IntAttr]] = {{.*}} alwaysinline
diff --git a/clang/test/CodeGenHLSL/builtins/RWBuffer-constructor.hlsl b/clang/test/CodeGenHLSL/builtins/RWBuffer-constructor.hlsl
index baddfcf2cf1d52..174f4c3eaaad26 100644
--- a/clang/test/CodeGenHLSL/builtins/RWBuffer-constructor.hlsl
+++ b/clang/test/CodeGenHLSL/builtins/RWBuffer-constructor.hlsl
@@ -1,3 +1,4 @@
+// RUN: %clang_cc1 -triple dxil-pc-shadermodel6.3-library -x hlsl -emit-llvm -disable-llvm-passes -o - %s | FileCheck %s
// RUN: %clang_cc1 -triple spirv-vulkan-library -x hlsl -emit-llvm -disable-llvm-passes -o - %s | FileCheck %s --check-prefix=CHECK-SPIRV
RWBuffer<float> Buf;
diff --git a/clang/test/CodeGenHLSL/builtins/RWBuffer-subscript.hlsl b/clang/test/CodeGenHLSL/builtins/RWBuffer-subscript.hlsl
index da8a1e538ec5e7..2a350c1619bd6e 100644
--- a/clang/test/CodeGenHLSL/builtins/RWBuffer-subscript.hlsl
+++ b/clang/test/CodeGenHLSL/builtins/RWBuffer-subscript.hlsl
@@ -11,6 +11,7 @@ void main(unsigned GI : SV_GroupIndex) {
// Even at -O0 the subscript operators get inlined. The -O0 IR is a bit messy
// and confusing to follow so the match here is pretty weak.
-// CHECK: define internal void @"?main@@YAXI@Z"
-// CHECK-NOT: call
+// CHECK: define void @main()
+// Verify inlining leaves only calls to "llvm." intrinsics
+// CHECK-NOT: call {{[^@]*}} @{{[^l][^l][^v][^m][^\.]}}
// CHECK: ret void
diff --git a/clang/test/CodeGenHLSL/inline-constructors.hlsl b/clang/test/CodeGenHLSL/inline-constructors.hlsl
new file mode 100644
index 00000000000000..40c7a42bdc262e
--- /dev/null
+++ b/clang/test/CodeGenHLSL/inline-constructors.hlsl
@@ -0,0 +1,74 @@
+// RUN: %clang_cc1 -triple dxil-pc-shadermodel6.0-compute -std=hlsl202x -emit-llvm -o - -disable-llvm-passes %s | FileCheck %s --check-prefixes=CHECK,NOINLINE
+// RUN: %clang_cc1 -triple dxil-pc-shadermodel6.3-library -std=hlsl202x -emit-llvm -o - -disable-llvm-passes %s | FileCheck %s --check-prefixes=CHECK,NOINLINE
+// RUN: %clang_cc1 -triple dxil-pc-shadermodel6.0-compute -std=hlsl202x -emit-llvm -o - -O0 %s | FileCheck %s --check-prefixes=CHECK,INLINE
+// RUN: %clang_cc1 -triple dxil-pc-shadermodel6.3-library -std=hlsl202x -emit-llvm -o - -O0 %s | FileCheck %s --check-prefixes=CHECK,INLINE
+
+// Tests that implicit contstructor calls for user classes will always be inlined.
+
+struct Weed {
+ Weed() {Count += 1;}
+ [[maybe_unused]] void pull() {Count--;}
+ static int weedCount() { return Count; }
+private:
+ static int Count;
+
+} YardWeeds;
+
+int Weed::Count = 1; // It begins. . .
+
+struct Kitty {
+ unsigned burrsInFur;
+
+ Kitty() {
+ burrsInFur = 0;
+ }
+
+ void wanderInYard(int hours) {
+ burrsInFur = hours*Weed::weedCount()/8;
+ }
+
+ void lick() {
+ if(burrsInFur) {
+ burrsInFur--;
+ Weed w;
+ }
+ }
+
+} Nion;
+
+void NionsDay(int hours) {
+ static Kitty Nion;
+ Nion.wanderInYard(hours);
+ while(Nion.burrsInFur) Nion.lick();
+}
+
+// CHECK: define void @main()
+// CHECK-NEXT: entry:
+// Verify constructor is emitted
+// NOINLINE-NEXT: call void @_GLOBAL__sub_I_inline_constructors.hlsl()
+// NOINLINE-NEXT: %0 = call i32 @llvm.dx.flattened.thread.id.in.group()
+// NOINLINE-NEXT: call void @"?main@@YAXI@Z"(i32 %0)
+// Verify inlining leaves only calls to "llvm." intrinsics
+// INLINE-NOT: call {{[^@]*}} @{{[^l][^l][^v][^m][^\.]}}
+// CHECK: ret void
+[shader("compute")]
+[numthreads(1,1,1)]
+void main(unsigned GI : SV_GroupIndex) {
+ NionsDay(10);
+}
+
+
+// CHECK: define void @rainyMain()
+// CHECK-NEXT: entry:
+// Verify constructor is emitted
+// NOINLINE-NEXT: call void @_GLOBAL__sub_I_inline_constructors.hlsl()
+// NOINLINE-NEXT: call void @"?rainyMain@@YAXXZ"()
+// Verify inlining leaves only calls to "llvm." intrinsics
+// INLINE-NOT: call {{[^@]*}} @{{[^l][^l][^v][^m][^\.]}}
+// CHECK: ret void
+[shader("compute")]
+[numthreads(1,1,1)]
+void rainyMain() {
+ NionsDay(1);
+}
+
diff --git a/clang/test/CodeGenHLSL/inline-functions.hlsl b/clang/test/CodeGenHLSL/inline-functions.hlsl
new file mode 100644
index 00000000000000..9c00e096a9eef2
--- /dev/null
+++ b/clang/test/CodeGenHLSL/inline-functions.hlsl
@@ -0,0 +1,114 @@
+// RUN: %clang_cc1 -x hlsl -triple dxil-pc-shadermodel6.3-library %s -emit-llvm -disable-llvm-passes -o - | FileCheck %s --check-prefixes=CHECK,NOINLINE
+// RUN: %clang_cc1 -x hlsl -triple dxil-pc-shadermodel6.3-library %s -emit-llvm -O0 -o - | FileCheck %s --check-prefixes=CHECK,INLINE
+// RUN: %clang_cc1 -x hlsl -triple dxil-pc-shadermodel6.0-compute %s -emit-llvm -disable-llvm-passes -o - | FileCheck %s --check-prefixes=CHECK,NOINLINE
+// RUN: %clang_cc1 -x hlsl -triple dxil-pc-shadermodel6.0-compute %s -emit-llvm -O0 -o - | FileCheck %s --check-prefixes=CHECK,INLINE
+
+// Tests that user functions will always be inlined.
+// This includes exported functions and mangled entry point implementation functions.
+// The unmangled entry functions must not be alwaysinlined.
+
+#define MAX 100
+
+float nums[MAX];
+
+// Verify that all functions have the alwaysinline attribute
+// CHECK: Function Attrs: alwaysinline
+// CHECK: define void @"?swap@@YAXY0GE@III@Z"(ptr noundef byval([100 x i32]) align 4 %Buf, i32 noundef %ix1, i32 noundef %ix2) [[IntAttr:\#[0-9]+]]
+// CHECK: ret void
+// Swap the values of Buf at indices ix1 and ix2
+void swap(unsigned Buf[MAX], unsigned ix1, unsigned ix2) {
+ float tmp = Buf[ix1];
+ Buf[ix1] = Buf[ix2];
+ Buf[ix2] = tmp;
+}
+
+// CHECK: Function Attrs: alwaysinline
+// CHECK: define void @"?BubbleSort@@YAXY0GE@II@Z"(ptr noundef byval([100 x i32]) align 4 %Buf, i32 noundef %size) [[IntAttr]]
+// CHECK: ret void
+// Inefficiently sort Buf in place
+void BubbleSort(unsigned Buf[MAX], unsigned size) {
+ bool swapped = true;
+ while (swapped) {
+ swapped = false;
+ for (unsigned i = 1; i < size; i++) {
+ if (Buf[i] < Buf[i-1]) {
+ swap(Buf, i, i-1);
+ swapped = true;
+ }
+ }
+ }
+}
+
+// Note ExtAttr is the inlined export set of attribs
+// CHECK: Function Attrs: alwaysinline
+// CHECK: define noundef i32 @"?RemoveDupes@@YAIY0GE@II@Z"(ptr noundef byval([100 x i32]) align 4 %Buf, i32 noundef %size) [[ExtAttr:\#[0-9]+]]
+// CHECK: ret i32
+// Sort Buf and remove any duplicate values
+// returns the number of values left
+export
+unsigned RemoveDupes(unsigned Buf[MAX], unsigned size) {
+ BubbleSort(Buf, size);
+ unsigned insertPt = 0;
+ for (unsigned i = 1; i < size; i++) {
+ if (Buf[i] == Buf[i-1])
+ insertPt++;
+ else
+ Buf[insertPt] = Buf[i];
+ }
+ return insertPt;
+}
+
+
+RWBuffer<unsigned> Indices;
+
+// The mangled version of main only remains without inlining
+// because it has internal linkage from the start
+// Note main functions get the norecurse attrib, which IntAttr reflects
+// NOINLINE: Function Attrs: alwaysinline
+// NOINLINE: define internal void @"?main@@YAXI@Z"(i32 noundef %GI) [[IntAttr]]
+// NOINLINE: ret void
+
+// The unmangled version is not inlined, EntryAttr reflects that
+// CHECK: Function Attrs: convergent norecurse
+// CHECK: define void @main() [[EntryAttr:\#[0-9]+]]
+// Make sure function calls are inlined when AlwaysInline is run
+// This only leaves calls to llvm. intrinsics
+// INLINE-NOT: call {{[^@]*}} @{{[^l][^l][^v][^m][^\.]}}
+// CHECK: ret void
+
+[numthreads(1,1,1)]
+[shader("compute")]
+void main(unsigned int GI : SV_GroupIndex) {
+ unsigned tmpIndices[MAX];
+ if (GI > MAX) return;
+ for (unsigned i = 1; i < GI; i++)
+ tmpIndices[i] = Indices[i];
+ RemoveDupes(tmpIndices, GI);
+ for (unsigned i = 1; i < GI; i++)
+ tmpIndices[i] = Indices[i];
+}
+
+// The mangled version of main only remains without inlining
+// because it has internal linkage from the start
+// Note main functions get the norecurse attrib, which IntAttr reflects
+// NOINLINE: Function Attrs: alwaysinline
+// NOINLINE: define internal void @"?main10@@YAXXZ"() [[IntAttr]]
+// NOINLINE: ret void
+
+// The unmangled version is not inlined, EntryAttr reflects that
+// CHECK: Function Attrs: convergent norecurse
+// CHECK: define void @main10() [[EntryAttr]]
+// Make sure function calls are inlined when AlwaysInline is run
+// This only leaves calls to llvm. intrinsics
+// INLINE-NOT: call {{[^@]*}} @{{[^l][^l][^v][^m][^\.]}}
+// CHECK: ret void
+
+[numthreads(1,1,1)]
+[shader("compute")]
+void main10() {
+ main(10);
+}
+
+// CHECK: attributes [[IntAttr]] = {{.*}} alwaysinline
+// CHECK: attributes [[ExtAttr]] = {{.*}} alwaysinline
+// CHECK-NOT: attributes [[EntryAttr]] = {{.*}} alwaysinline
|
|
@llvm/pr-subscribers-hlsl Author: Greg Roth (pow2clk) ChangesHLSL inlines all its functions by default. This uses the alwaysinline attribute to force that in the corresponding pass for user functions by default and overrides the default noinline of some implicit functions. This makes an instance of explicit inlining for buffer subscripts unnecessary. Adds tests for function and constructor inlining and augments some existing tests to verify correct inlining of implicitly created functions as well. incidentally restore RUN line that I believe was mistakenly removed as part of #88918 fixes #89282 Full diff: https://github.com/llvm/llvm-project/pull/106588.diff 10 Files Affected:
diff --git a/clang/lib/CodeGen/CGHLSLRuntime.cpp b/clang/lib/CodeGen/CGHLSLRuntime.cpp
index 4bd7b6ba58de0d..24d126ced0d9f7 100644
--- a/clang/lib/CodeGen/CGHLSLRuntime.cpp
+++ b/clang/lib/CodeGen/CGHLSLRuntime.cpp
@@ -414,9 +414,20 @@ void CGHLSLRuntime::emitEntryFunction(const FunctionDecl *FD,
void CGHLSLRuntime::setHLSLFunctionAttributes(const FunctionDecl *FD,
llvm::Function *Fn) {
- if (FD->isInExportDeclContext()) {
- const StringRef ExportAttrKindStr = "hlsl.export";
- Fn->addFnAttr(ExportAttrKindStr);
+ if (FD) { // "explicit" functions with declarations
+ if (FD->isInExportDeclContext()) {
+ const StringRef ExportAttrKindStr = "hlsl.export";
+ Fn->addFnAttr(ExportAttrKindStr);
+ }
+ // Respect noinline if the explicit functions use it
+ // otherwise default to alwaysinline
+ if (!Fn->hasFnAttribute(Attribute::NoInline))
+ Fn->addFnAttr(llvm::Attribute::AlwaysInline);
+ } else { // "implicit" autogenerated functions with no declaration
+ // Implicit functions might get marked as noinline by default
+ // but we override that for HLSL
+ Fn->removeFnAttr(Attribute::NoInline);
+ Fn->addFnAttr(Attribute::AlwaysInline);
}
}
diff --git a/clang/lib/CodeGen/CodeGenFunction.cpp b/clang/lib/CodeGen/CodeGenFunction.cpp
index a5747283e98058..aceeed0e66d130 100644
--- a/clang/lib/CodeGen/CodeGenFunction.cpp
+++ b/clang/lib/CodeGen/CodeGenFunction.cpp
@@ -1239,9 +1239,9 @@ void CodeGenFunction::StartFunction(GlobalDecl GD, QualType RetTy,
if (getLangOpts().OpenMP && CurCodeDecl)
CGM.getOpenMPRuntime().emitFunctionProlog(*this, CurCodeDecl);
- if (FD && getLangOpts().HLSL) {
+ if (getLangOpts().HLSL) {
// Handle emitting HLSL entry functions.
- if (FD->hasAttr<HLSLShaderAttr>()) {
+ if (FD && FD->hasAttr<HLSLShaderAttr>()) {
CGM.getHLSLRuntime().emitEntryFunction(FD, Fn);
}
CGM.getHLSLRuntime().setHLSLFunctionAttributes(FD, Fn);
diff --git a/clang/lib/Sema/HLSLExternalSemaSource.cpp b/clang/lib/Sema/HLSLExternalSemaSource.cpp
index 9aacbe4ad9548e..0a534d94192560 100644
--- a/clang/lib/Sema/HLSLExternalSemaSource.cpp
+++ b/clang/lib/Sema/HLSLExternalSemaSource.cpp
@@ -290,8 +290,6 @@ struct BuiltinTypeDeclBuilder {
SourceLocation()));
MethodDecl->setLexicalDeclContext(Record);
MethodDecl->setAccess(AccessSpecifier::AS_public);
- MethodDecl->addAttr(AlwaysInlineAttr::CreateImplicit(
- AST, SourceRange(), AlwaysInlineAttr::CXX11_clang_always_inline));
Record->addDecl(MethodDecl);
return *this;
diff --git a/clang/test/CodeGenHLSL/GlobalConstructorFunction.hlsl b/clang/test/CodeGenHLSL/GlobalConstructorFunction.hlsl
index f954c9d2f029f2..b39311ad67cd62 100644
--- a/clang/test/CodeGenHLSL/GlobalConstructorFunction.hlsl
+++ b/clang/test/CodeGenHLSL/GlobalConstructorFunction.hlsl
@@ -1,4 +1,5 @@
-// RUN: %clang_cc1 -triple dxil-pc-shadermodel6.0-compute -emit-llvm -disable-llvm-passes %s -o - | FileCheck %s
+// RUN: %clang_cc1 -triple dxil-pc-shadermodel6.0-compute -emit-llvm -disable-llvm-passes %s -o - | FileCheck %s --check-prefixes=CHECK,NOINLINE
+// RUN: %clang_cc1 -triple dxil-pc-shadermodel6.0-compute -emit-llvm -O0 %s -o - | FileCheck %s --check-prefixes=CHECK,INLINE
int i;
@@ -7,7 +8,7 @@ __attribute__((constructor)) void call_me_first(void) {
}
__attribute__((constructor)) void then_call_me(void) {
- i = 12;
+ i = 13;
}
__attribute__((destructor)) void call_me_last(void) {
@@ -21,11 +22,21 @@ void main(unsigned GI : SV_GroupIndex) {}
// CHECK-NOT:@llvm.global_ctors
// CHECK-NOT:@llvm.global_dtors
-//CHECK: define void @main()
-//CHECK-NEXT: entry:
-//CHECK-NEXT: call void @"?call_me_first@@YAXXZ"()
-//CHECK-NEXT: call void @"?then_call_me@@YAXXZ"()
-//CHECK-NEXT: %0 = call i32 @llvm.dx.flattened.thread.id.in.group()
-//CHECK-NEXT: call void @"?main@@YAXI@Z"(i32 %0)
-//CHECK-NEXT: call void @"?call_me_last@@YAXXZ"(
-//CHECK-NEXT: ret void
+// CHECK: define void @main()
+// CHECK-NEXT: entry:
+// Verify function constructors are emitted
+// NOINLINE-NEXT: call void @"?call_me_first@@YAXXZ"()
+// NOINLINE-NEXT: call void @"?then_call_me@@YAXXZ"()
+// NOINLINE-NEXT: %0 = call i32 @llvm.dx.flattened.thread.id.in.group()
+// NOINLINE-NEXT: call void @"?main@@YAXI@Z"(i32 %0)
+// NOINLINE-NEXT: call void @"?call_me_last@@YAXXZ"(
+// NOINLINE-NEXT: ret void
+
+// Verify constructor calls are inlined when AlwaysInline is run
+// INLINE-NEXT: alloca
+// INLINE-NEXT: store i32 12
+// INLINE-NEXT: store i32 13
+// INLINE-NEXT: %0 = call i32 @llvm.dx.flattened.thread.id.in.group()
+// INLINE-NEXT: store i32 %
+// INLINE-NEXT: store i32 0
+// INLINE: ret void
diff --git a/clang/test/CodeGenHLSL/GlobalConstructorLib.hlsl b/clang/test/CodeGenHLSL/GlobalConstructorLib.hlsl
index 2c5c4e19c3296d..78f6475462bc47 100644
--- a/clang/test/CodeGenHLSL/GlobalConstructorLib.hlsl
+++ b/clang/test/CodeGenHLSL/GlobalConstructorLib.hlsl
@@ -1,4 +1,5 @@
-// RUN: %clang_cc1 -triple dxil-pc-shadermodel6.3-library -emit-llvm -disable-llvm-passes %s -o - | FileCheck %s
+// RUN: %clang_cc1 -triple dxil-pc-shadermodel6.3-library -emit-llvm -disable-llvm-passes %s -o - | FileCheck %s --check-prefixes=CHECK,NOINLINE
+// RUN: %clang_cc1 -triple dxil-pc-shadermodel6.3-library -emit-llvm -O0 %s -o - | FileCheck %s --check-prefixes=CHECK,INLINE
// Make sure global variable for ctors exist for lib profile.
// CHECK:@llvm.global_ctors
@@ -11,7 +12,11 @@ void FirstEntry() {}
// CHECK: define void @FirstEntry()
// CHECK-NEXT: entry:
-// CHECK-NEXT: call void @_GLOBAL__sub_I_GlobalConstructorLib.hlsl()
+// NOINLINE-NEXT: call void @_GLOBAL__sub_I_GlobalConstructorLib.hlsl()
+// NOINLINE-NEXT: call void @"?FirstEntry@@YAXXZ"()
+// Verify inlining leaves only calls to "llvm." intrinsics
+// INLINE-NOT: call {{[^@]*}} @{{[^l][^l][^v][^m][^\.]}}
+// CHECK: ret void
[shader("compute")]
[numthreads(1,1,1)]
@@ -19,5 +24,15 @@ void SecondEntry() {}
// CHECK: define void @SecondEntry()
// CHECK-NEXT: entry:
-// CHECK-NEXT: call void @_GLOBAL__sub_I_GlobalConstructorLib.hlsl()
-// CHECK-NEXT: call void @"?SecondEntry@@YAXXZ"()
+// NOINLINE-NEXT: call void @_GLOBAL__sub_I_GlobalConstructorLib.hlsl()
+// NOINLINE-NEXT: call void @"?SecondEntry@@YAXXZ"()
+// Verify inlining leaves only calls to "llvm." intrinsics
+// INLINE-NOT: call {{[^@]*}} @{{[^l][^l][^v][^m][^\.]}}
+// CHECK: ret void
+
+
+// Verify the constructor is alwaysinline
+// NOINLINE: ; Function Attrs: {{.*}}alwaysinline
+// NOINLINE-NEXT: define internal void @_GLOBAL__sub_I_GlobalConstructorLib.hlsl() [[IntAttr:\#[0-9]+]]
+
+// NOINLINE: attributes [[IntAttr]] = {{.*}} alwaysinline
diff --git a/clang/test/CodeGenHLSL/GlobalDestructors.hlsl b/clang/test/CodeGenHLSL/GlobalDestructors.hlsl
index 24c3c039fc6192..ea28354222f885 100644
--- a/clang/test/CodeGenHLSL/GlobalDestructors.hlsl
+++ b/clang/test/CodeGenHLSL/GlobalDestructors.hlsl
@@ -1,10 +1,18 @@
-// RUN: %clang_cc1 -triple dxil-pc-shadermodel6.0-compute -std=hlsl202x -emit-llvm -disable-llvm-passes %s -o - | FileCheck %s --check-prefixes=CS,CHECK
-// RUN: %clang_cc1 -triple dxil-pc-shadermodel6.3-library -std=hlsl202x -emit-llvm -disable-llvm-passes %s -o - | FileCheck %s --check-prefixes=LIB,CHECK
+// RUN: %clang_cc1 -triple dxil-pc-shadermodel6.0-compute -std=hlsl202x -emit-llvm -disable-llvm-passes %s -o - | FileCheck %s --check-prefixes=CS,NOINLINE,CHECK
+// RUN: %clang_cc1 -triple dxil-pc-shadermodel6.3-library -std=hlsl202x -emit-llvm -disable-llvm-passes %s -o - | FileCheck %s --check-prefixes=LIB,NOINLINE,CHECK
+// RUN: %clang_cc1 -triple dxil-pc-shadermodel6.0-compute -std=hlsl202x -emit-llvm -O0 %s -o - | FileCheck %s --check-prefixes=INLINE,CHECK
+// RUN: %clang_cc1 -triple dxil-pc-shadermodel6.3-library -std=hlsl202x -emit-llvm -O0 %s -o - | FileCheck %s --check-prefixes=INLINE,CHECK
-// Make sure global variable for dtors exist for lib profile.
+// Tests that constructors and destructors are appropriately generated for globals
+// and that their calls are inlined when AlwaysInline is run
+// but global variables are retained for the library profiles
+
+// Make sure global variable for ctors/dtors exist for lib profile.
+// LIB:@llvm.global_ctors
// LIB:@llvm.global_dtors
-// Make sure global variable for dtors removed for compute profile.
-// CS-NOT:llvm.global_dtors
+// Make sure global variable for ctors/dtors removed for compute profile.
+// CS-NOT:@llvm.global_ctors
+// CS-NOT:@llvm.global_dtors
struct Tail {
Tail() {
@@ -46,22 +54,25 @@ void main(unsigned GI : SV_GroupIndex) {
Wag();
}
-// Make sure global variable for ctors/dtors removed.
-// CHECK-NOT:@llvm.global_ctors
-// CHECK-NOT:@llvm.global_dtors
-//CHECK: define void @main()
-//CHECK-NEXT: entry:
-//CHECK-NEXT: call void @_GLOBAL__sub_I_GlobalDestructors.hlsl()
-//CHECK-NEXT: %0 = call i32 @llvm.dx.flattened.thread.id.in.group()
-//CHECK-NEXT: call void @"?main@@YAXI@Z"(i32 %0)
-//CHECK-NEXT: call void @_GLOBAL__D_a()
-//CHECK-NEXT: ret void
+// CHECK: define void @main()
+// CHECK-NEXT: entry:
+// Verify destructor is emitted
+// NOINLINE-NEXT: call void @_GLOBAL__sub_I_GlobalDestructors.hlsl()
+// NOINLINE-NEXT: %0 = call i32 @llvm.dx.flattened.thread.id.in.group()
+// NOINLINE-NEXT: call void @"?main@@YAXI@Z"(i32 %0)
+// NOINLINE-NEXT: call void @_GLOBAL__D_a()
+// NOINLINE-NEXT: ret void
+// Verify inlining leaves only calls to "llvm." intrinsics
+// INLINE-NOT: call {{[^@]*}} @{{[^l][^l][^v][^m][^\.]}}
+// INLINE: ret void
// This is really just a sanity check I needed for myself to verify that
// function scope static variables also get destroyed properly.
-//CHECK: define internal void @_GLOBAL__D_a()
-//CHECK-NEXT: entry:
-//CHECK-NEXT: call void @"??1Tail@@QAA@XZ"(ptr @"?T@?1??Wag@@YAXXZ@4UTail@@A")
-//CHECK-NEXT: call void @"??1Pupper@@QAA@XZ"(ptr @"?GlobalPup@@3UPupper@@A")
-//CHECK-NEXT: ret void
+// NOINLINE: define internal void @_GLOBAL__D_a() [[IntAttr:\#[0-9]+]]
+// NOINLINE-NEXT: entry:
+// NOINLINE-NEXT: call void @"??1Tail@@QAA@XZ"(ptr @"?T@?1??Wag@@YAXXZ@4UTail@@A")
+// NOINLINE-NEXT: call void @"??1Pupper@@QAA@XZ"(ptr @"?GlobalPup@@3UPupper@@A")
+// NOINLINE-NEXT: ret void
+
+// NOINLINE: attributes [[IntAttr]] = {{.*}} alwaysinline
diff --git a/clang/test/CodeGenHLSL/builtins/RWBuffer-constructor.hlsl b/clang/test/CodeGenHLSL/builtins/RWBuffer-constructor.hlsl
index baddfcf2cf1d52..174f4c3eaaad26 100644
--- a/clang/test/CodeGenHLSL/builtins/RWBuffer-constructor.hlsl
+++ b/clang/test/CodeGenHLSL/builtins/RWBuffer-constructor.hlsl
@@ -1,3 +1,4 @@
+// RUN: %clang_cc1 -triple dxil-pc-shadermodel6.3-library -x hlsl -emit-llvm -disable-llvm-passes -o - %s | FileCheck %s
// RUN: %clang_cc1 -triple spirv-vulkan-library -x hlsl -emit-llvm -disable-llvm-passes -o - %s | FileCheck %s --check-prefix=CHECK-SPIRV
RWBuffer<float> Buf;
diff --git a/clang/test/CodeGenHLSL/builtins/RWBuffer-subscript.hlsl b/clang/test/CodeGenHLSL/builtins/RWBuffer-subscript.hlsl
index da8a1e538ec5e7..2a350c1619bd6e 100644
--- a/clang/test/CodeGenHLSL/builtins/RWBuffer-subscript.hlsl
+++ b/clang/test/CodeGenHLSL/builtins/RWBuffer-subscript.hlsl
@@ -11,6 +11,7 @@ void main(unsigned GI : SV_GroupIndex) {
// Even at -O0 the subscript operators get inlined. The -O0 IR is a bit messy
// and confusing to follow so the match here is pretty weak.
-// CHECK: define internal void @"?main@@YAXI@Z"
-// CHECK-NOT: call
+// CHECK: define void @main()
+// Verify inlining leaves only calls to "llvm." intrinsics
+// CHECK-NOT: call {{[^@]*}} @{{[^l][^l][^v][^m][^\.]}}
// CHECK: ret void
diff --git a/clang/test/CodeGenHLSL/inline-constructors.hlsl b/clang/test/CodeGenHLSL/inline-constructors.hlsl
new file mode 100644
index 00000000000000..40c7a42bdc262e
--- /dev/null
+++ b/clang/test/CodeGenHLSL/inline-constructors.hlsl
@@ -0,0 +1,74 @@
+// RUN: %clang_cc1 -triple dxil-pc-shadermodel6.0-compute -std=hlsl202x -emit-llvm -o - -disable-llvm-passes %s | FileCheck %s --check-prefixes=CHECK,NOINLINE
+// RUN: %clang_cc1 -triple dxil-pc-shadermodel6.3-library -std=hlsl202x -emit-llvm -o - -disable-llvm-passes %s | FileCheck %s --check-prefixes=CHECK,NOINLINE
+// RUN: %clang_cc1 -triple dxil-pc-shadermodel6.0-compute -std=hlsl202x -emit-llvm -o - -O0 %s | FileCheck %s --check-prefixes=CHECK,INLINE
+// RUN: %clang_cc1 -triple dxil-pc-shadermodel6.3-library -std=hlsl202x -emit-llvm -o - -O0 %s | FileCheck %s --check-prefixes=CHECK,INLINE
+
+// Tests that implicit contstructor calls for user classes will always be inlined.
+
+struct Weed {
+ Weed() {Count += 1;}
+ [[maybe_unused]] void pull() {Count--;}
+ static int weedCount() { return Count; }
+private:
+ static int Count;
+
+} YardWeeds;
+
+int Weed::Count = 1; // It begins. . .
+
+struct Kitty {
+ unsigned burrsInFur;
+
+ Kitty() {
+ burrsInFur = 0;
+ }
+
+ void wanderInYard(int hours) {
+ burrsInFur = hours*Weed::weedCount()/8;
+ }
+
+ void lick() {
+ if(burrsInFur) {
+ burrsInFur--;
+ Weed w;
+ }
+ }
+
+} Nion;
+
+void NionsDay(int hours) {
+ static Kitty Nion;
+ Nion.wanderInYard(hours);
+ while(Nion.burrsInFur) Nion.lick();
+}
+
+// CHECK: define void @main()
+// CHECK-NEXT: entry:
+// Verify constructor is emitted
+// NOINLINE-NEXT: call void @_GLOBAL__sub_I_inline_constructors.hlsl()
+// NOINLINE-NEXT: %0 = call i32 @llvm.dx.flattened.thread.id.in.group()
+// NOINLINE-NEXT: call void @"?main@@YAXI@Z"(i32 %0)
+// Verify inlining leaves only calls to "llvm." intrinsics
+// INLINE-NOT: call {{[^@]*}} @{{[^l][^l][^v][^m][^\.]}}
+// CHECK: ret void
+[shader("compute")]
+[numthreads(1,1,1)]
+void main(unsigned GI : SV_GroupIndex) {
+ NionsDay(10);
+}
+
+
+// CHECK: define void @rainyMain()
+// CHECK-NEXT: entry:
+// Verify constructor is emitted
+// NOINLINE-NEXT: call void @_GLOBAL__sub_I_inline_constructors.hlsl()
+// NOINLINE-NEXT: call void @"?rainyMain@@YAXXZ"()
+// Verify inlining leaves only calls to "llvm." intrinsics
+// INLINE-NOT: call {{[^@]*}} @{{[^l][^l][^v][^m][^\.]}}
+// CHECK: ret void
+[shader("compute")]
+[numthreads(1,1,1)]
+void rainyMain() {
+ NionsDay(1);
+}
+
diff --git a/clang/test/CodeGenHLSL/inline-functions.hlsl b/clang/test/CodeGenHLSL/inline-functions.hlsl
new file mode 100644
index 00000000000000..9c00e096a9eef2
--- /dev/null
+++ b/clang/test/CodeGenHLSL/inline-functions.hlsl
@@ -0,0 +1,114 @@
+// RUN: %clang_cc1 -x hlsl -triple dxil-pc-shadermodel6.3-library %s -emit-llvm -disable-llvm-passes -o - | FileCheck %s --check-prefixes=CHECK,NOINLINE
+// RUN: %clang_cc1 -x hlsl -triple dxil-pc-shadermodel6.3-library %s -emit-llvm -O0 -o - | FileCheck %s --check-prefixes=CHECK,INLINE
+// RUN: %clang_cc1 -x hlsl -triple dxil-pc-shadermodel6.0-compute %s -emit-llvm -disable-llvm-passes -o - | FileCheck %s --check-prefixes=CHECK,NOINLINE
+// RUN: %clang_cc1 -x hlsl -triple dxil-pc-shadermodel6.0-compute %s -emit-llvm -O0 -o - | FileCheck %s --check-prefixes=CHECK,INLINE
+
+// Tests that user functions will always be inlined.
+// This includes exported functions and mangled entry point implementation functions.
+// The unmangled entry functions must not be alwaysinlined.
+
+#define MAX 100
+
+float nums[MAX];
+
+// Verify that all functions have the alwaysinline attribute
+// CHECK: Function Attrs: alwaysinline
+// CHECK: define void @"?swap@@YAXY0GE@III@Z"(ptr noundef byval([100 x i32]) align 4 %Buf, i32 noundef %ix1, i32 noundef %ix2) [[IntAttr:\#[0-9]+]]
+// CHECK: ret void
+// Swap the values of Buf at indices ix1 and ix2
+void swap(unsigned Buf[MAX], unsigned ix1, unsigned ix2) {
+ float tmp = Buf[ix1];
+ Buf[ix1] = Buf[ix2];
+ Buf[ix2] = tmp;
+}
+
+// CHECK: Function Attrs: alwaysinline
+// CHECK: define void @"?BubbleSort@@YAXY0GE@II@Z"(ptr noundef byval([100 x i32]) align 4 %Buf, i32 noundef %size) [[IntAttr]]
+// CHECK: ret void
+// Inefficiently sort Buf in place
+void BubbleSort(unsigned Buf[MAX], unsigned size) {
+ bool swapped = true;
+ while (swapped) {
+ swapped = false;
+ for (unsigned i = 1; i < size; i++) {
+ if (Buf[i] < Buf[i-1]) {
+ swap(Buf, i, i-1);
+ swapped = true;
+ }
+ }
+ }
+}
+
+// Note ExtAttr is the inlined export set of attribs
+// CHECK: Function Attrs: alwaysinline
+// CHECK: define noundef i32 @"?RemoveDupes@@YAIY0GE@II@Z"(ptr noundef byval([100 x i32]) align 4 %Buf, i32 noundef %size) [[ExtAttr:\#[0-9]+]]
+// CHECK: ret i32
+// Sort Buf and remove any duplicate values
+// returns the number of values left
+export
+unsigned RemoveDupes(unsigned Buf[MAX], unsigned size) {
+ BubbleSort(Buf, size);
+ unsigned insertPt = 0;
+ for (unsigned i = 1; i < size; i++) {
+ if (Buf[i] == Buf[i-1])
+ insertPt++;
+ else
+ Buf[insertPt] = Buf[i];
+ }
+ return insertPt;
+}
+
+
+RWBuffer<unsigned> Indices;
+
+// The mangled version of main only remains without inlining
+// because it has internal linkage from the start
+// Note main functions get the norecurse attrib, which IntAttr reflects
+// NOINLINE: Function Attrs: alwaysinline
+// NOINLINE: define internal void @"?main@@YAXI@Z"(i32 noundef %GI) [[IntAttr]]
+// NOINLINE: ret void
+
+// The unmangled version is not inlined, EntryAttr reflects that
+// CHECK: Function Attrs: convergent norecurse
+// CHECK: define void @main() [[EntryAttr:\#[0-9]+]]
+// Make sure function calls are inlined when AlwaysInline is run
+// This only leaves calls to llvm. intrinsics
+// INLINE-NOT: call {{[^@]*}} @{{[^l][^l][^v][^m][^\.]}}
+// CHECK: ret void
+
+[numthreads(1,1,1)]
+[shader("compute")]
+void main(unsigned int GI : SV_GroupIndex) {
+ unsigned tmpIndices[MAX];
+ if (GI > MAX) return;
+ for (unsigned i = 1; i < GI; i++)
+ tmpIndices[i] = Indices[i];
+ RemoveDupes(tmpIndices, GI);
+ for (unsigned i = 1; i < GI; i++)
+ tmpIndices[i] = Indices[i];
+}
+
+// The mangled version of main only remains without inlining
+// because it has internal linkage from the start
+// Note main functions get the norecurse attrib, which IntAttr reflects
+// NOINLINE: Function Attrs: alwaysinline
+// NOINLINE: define internal void @"?main10@@YAXXZ"() [[IntAttr]]
+// NOINLINE: ret void
+
+// The unmangled version is not inlined, EntryAttr reflects that
+// CHECK: Function Attrs: convergent norecurse
+// CHECK: define void @main10() [[EntryAttr]]
+// Make sure function calls are inlined when AlwaysInline is run
+// This only leaves calls to llvm. intrinsics
+// INLINE-NOT: call {{[^@]*}} @{{[^l][^l][^v][^m][^\.]}}
+// CHECK: ret void
+
+[numthreads(1,1,1)]
+[shader("compute")]
+void main10() {
+ main(10);
+}
+
+// CHECK: attributes [[IntAttr]] = {{.*}} alwaysinline
+// CHECK: attributes [[ExtAttr]] = {{.*}} alwaysinline
+// CHECK-NOT: attributes [[EntryAttr]] = {{.*}} alwaysinline
|
|
@llvm/pr-subscribers-clang-codegen Author: Greg Roth (pow2clk) ChangesHLSL inlines all its functions by default. This uses the alwaysinline attribute to force that in the corresponding pass for user functions by default and overrides the default noinline of some implicit functions. This makes an instance of explicit inlining for buffer subscripts unnecessary. Adds tests for function and constructor inlining and augments some existing tests to verify correct inlining of implicitly created functions as well. incidentally restore RUN line that I believe was mistakenly removed as part of #88918 fixes #89282 Full diff: https://github.com/llvm/llvm-project/pull/106588.diff 10 Files Affected:
diff --git a/clang/lib/CodeGen/CGHLSLRuntime.cpp b/clang/lib/CodeGen/CGHLSLRuntime.cpp
index 4bd7b6ba58de0d..24d126ced0d9f7 100644
--- a/clang/lib/CodeGen/CGHLSLRuntime.cpp
+++ b/clang/lib/CodeGen/CGHLSLRuntime.cpp
@@ -414,9 +414,20 @@ void CGHLSLRuntime::emitEntryFunction(const FunctionDecl *FD,
void CGHLSLRuntime::setHLSLFunctionAttributes(const FunctionDecl *FD,
llvm::Function *Fn) {
- if (FD->isInExportDeclContext()) {
- const StringRef ExportAttrKindStr = "hlsl.export";
- Fn->addFnAttr(ExportAttrKindStr);
+ if (FD) { // "explicit" functions with declarations
+ if (FD->isInExportDeclContext()) {
+ const StringRef ExportAttrKindStr = "hlsl.export";
+ Fn->addFnAttr(ExportAttrKindStr);
+ }
+ // Respect noinline if the explicit functions use it
+ // otherwise default to alwaysinline
+ if (!Fn->hasFnAttribute(Attribute::NoInline))
+ Fn->addFnAttr(llvm::Attribute::AlwaysInline);
+ } else { // "implicit" autogenerated functions with no declaration
+ // Implicit functions might get marked as noinline by default
+ // but we override that for HLSL
+ Fn->removeFnAttr(Attribute::NoInline);
+ Fn->addFnAttr(Attribute::AlwaysInline);
}
}
diff --git a/clang/lib/CodeGen/CodeGenFunction.cpp b/clang/lib/CodeGen/CodeGenFunction.cpp
index a5747283e98058..aceeed0e66d130 100644
--- a/clang/lib/CodeGen/CodeGenFunction.cpp
+++ b/clang/lib/CodeGen/CodeGenFunction.cpp
@@ -1239,9 +1239,9 @@ void CodeGenFunction::StartFunction(GlobalDecl GD, QualType RetTy,
if (getLangOpts().OpenMP && CurCodeDecl)
CGM.getOpenMPRuntime().emitFunctionProlog(*this, CurCodeDecl);
- if (FD && getLangOpts().HLSL) {
+ if (getLangOpts().HLSL) {
// Handle emitting HLSL entry functions.
- if (FD->hasAttr<HLSLShaderAttr>()) {
+ if (FD && FD->hasAttr<HLSLShaderAttr>()) {
CGM.getHLSLRuntime().emitEntryFunction(FD, Fn);
}
CGM.getHLSLRuntime().setHLSLFunctionAttributes(FD, Fn);
diff --git a/clang/lib/Sema/HLSLExternalSemaSource.cpp b/clang/lib/Sema/HLSLExternalSemaSource.cpp
index 9aacbe4ad9548e..0a534d94192560 100644
--- a/clang/lib/Sema/HLSLExternalSemaSource.cpp
+++ b/clang/lib/Sema/HLSLExternalSemaSource.cpp
@@ -290,8 +290,6 @@ struct BuiltinTypeDeclBuilder {
SourceLocation()));
MethodDecl->setLexicalDeclContext(Record);
MethodDecl->setAccess(AccessSpecifier::AS_public);
- MethodDecl->addAttr(AlwaysInlineAttr::CreateImplicit(
- AST, SourceRange(), AlwaysInlineAttr::CXX11_clang_always_inline));
Record->addDecl(MethodDecl);
return *this;
diff --git a/clang/test/CodeGenHLSL/GlobalConstructorFunction.hlsl b/clang/test/CodeGenHLSL/GlobalConstructorFunction.hlsl
index f954c9d2f029f2..b39311ad67cd62 100644
--- a/clang/test/CodeGenHLSL/GlobalConstructorFunction.hlsl
+++ b/clang/test/CodeGenHLSL/GlobalConstructorFunction.hlsl
@@ -1,4 +1,5 @@
-// RUN: %clang_cc1 -triple dxil-pc-shadermodel6.0-compute -emit-llvm -disable-llvm-passes %s -o - | FileCheck %s
+// RUN: %clang_cc1 -triple dxil-pc-shadermodel6.0-compute -emit-llvm -disable-llvm-passes %s -o - | FileCheck %s --check-prefixes=CHECK,NOINLINE
+// RUN: %clang_cc1 -triple dxil-pc-shadermodel6.0-compute -emit-llvm -O0 %s -o - | FileCheck %s --check-prefixes=CHECK,INLINE
int i;
@@ -7,7 +8,7 @@ __attribute__((constructor)) void call_me_first(void) {
}
__attribute__((constructor)) void then_call_me(void) {
- i = 12;
+ i = 13;
}
__attribute__((destructor)) void call_me_last(void) {
@@ -21,11 +22,21 @@ void main(unsigned GI : SV_GroupIndex) {}
// CHECK-NOT:@llvm.global_ctors
// CHECK-NOT:@llvm.global_dtors
-//CHECK: define void @main()
-//CHECK-NEXT: entry:
-//CHECK-NEXT: call void @"?call_me_first@@YAXXZ"()
-//CHECK-NEXT: call void @"?then_call_me@@YAXXZ"()
-//CHECK-NEXT: %0 = call i32 @llvm.dx.flattened.thread.id.in.group()
-//CHECK-NEXT: call void @"?main@@YAXI@Z"(i32 %0)
-//CHECK-NEXT: call void @"?call_me_last@@YAXXZ"(
-//CHECK-NEXT: ret void
+// CHECK: define void @main()
+// CHECK-NEXT: entry:
+// Verify function constructors are emitted
+// NOINLINE-NEXT: call void @"?call_me_first@@YAXXZ"()
+// NOINLINE-NEXT: call void @"?then_call_me@@YAXXZ"()
+// NOINLINE-NEXT: %0 = call i32 @llvm.dx.flattened.thread.id.in.group()
+// NOINLINE-NEXT: call void @"?main@@YAXI@Z"(i32 %0)
+// NOINLINE-NEXT: call void @"?call_me_last@@YAXXZ"(
+// NOINLINE-NEXT: ret void
+
+// Verify constructor calls are inlined when AlwaysInline is run
+// INLINE-NEXT: alloca
+// INLINE-NEXT: store i32 12
+// INLINE-NEXT: store i32 13
+// INLINE-NEXT: %0 = call i32 @llvm.dx.flattened.thread.id.in.group()
+// INLINE-NEXT: store i32 %
+// INLINE-NEXT: store i32 0
+// INLINE: ret void
diff --git a/clang/test/CodeGenHLSL/GlobalConstructorLib.hlsl b/clang/test/CodeGenHLSL/GlobalConstructorLib.hlsl
index 2c5c4e19c3296d..78f6475462bc47 100644
--- a/clang/test/CodeGenHLSL/GlobalConstructorLib.hlsl
+++ b/clang/test/CodeGenHLSL/GlobalConstructorLib.hlsl
@@ -1,4 +1,5 @@
-// RUN: %clang_cc1 -triple dxil-pc-shadermodel6.3-library -emit-llvm -disable-llvm-passes %s -o - | FileCheck %s
+// RUN: %clang_cc1 -triple dxil-pc-shadermodel6.3-library -emit-llvm -disable-llvm-passes %s -o - | FileCheck %s --check-prefixes=CHECK,NOINLINE
+// RUN: %clang_cc1 -triple dxil-pc-shadermodel6.3-library -emit-llvm -O0 %s -o - | FileCheck %s --check-prefixes=CHECK,INLINE
// Make sure global variable for ctors exist for lib profile.
// CHECK:@llvm.global_ctors
@@ -11,7 +12,11 @@ void FirstEntry() {}
// CHECK: define void @FirstEntry()
// CHECK-NEXT: entry:
-// CHECK-NEXT: call void @_GLOBAL__sub_I_GlobalConstructorLib.hlsl()
+// NOINLINE-NEXT: call void @_GLOBAL__sub_I_GlobalConstructorLib.hlsl()
+// NOINLINE-NEXT: call void @"?FirstEntry@@YAXXZ"()
+// Verify inlining leaves only calls to "llvm." intrinsics
+// INLINE-NOT: call {{[^@]*}} @{{[^l][^l][^v][^m][^\.]}}
+// CHECK: ret void
[shader("compute")]
[numthreads(1,1,1)]
@@ -19,5 +24,15 @@ void SecondEntry() {}
// CHECK: define void @SecondEntry()
// CHECK-NEXT: entry:
-// CHECK-NEXT: call void @_GLOBAL__sub_I_GlobalConstructorLib.hlsl()
-// CHECK-NEXT: call void @"?SecondEntry@@YAXXZ"()
+// NOINLINE-NEXT: call void @_GLOBAL__sub_I_GlobalConstructorLib.hlsl()
+// NOINLINE-NEXT: call void @"?SecondEntry@@YAXXZ"()
+// Verify inlining leaves only calls to "llvm." intrinsics
+// INLINE-NOT: call {{[^@]*}} @{{[^l][^l][^v][^m][^\.]}}
+// CHECK: ret void
+
+
+// Verify the constructor is alwaysinline
+// NOINLINE: ; Function Attrs: {{.*}}alwaysinline
+// NOINLINE-NEXT: define internal void @_GLOBAL__sub_I_GlobalConstructorLib.hlsl() [[IntAttr:\#[0-9]+]]
+
+// NOINLINE: attributes [[IntAttr]] = {{.*}} alwaysinline
diff --git a/clang/test/CodeGenHLSL/GlobalDestructors.hlsl b/clang/test/CodeGenHLSL/GlobalDestructors.hlsl
index 24c3c039fc6192..ea28354222f885 100644
--- a/clang/test/CodeGenHLSL/GlobalDestructors.hlsl
+++ b/clang/test/CodeGenHLSL/GlobalDestructors.hlsl
@@ -1,10 +1,18 @@
-// RUN: %clang_cc1 -triple dxil-pc-shadermodel6.0-compute -std=hlsl202x -emit-llvm -disable-llvm-passes %s -o - | FileCheck %s --check-prefixes=CS,CHECK
-// RUN: %clang_cc1 -triple dxil-pc-shadermodel6.3-library -std=hlsl202x -emit-llvm -disable-llvm-passes %s -o - | FileCheck %s --check-prefixes=LIB,CHECK
+// RUN: %clang_cc1 -triple dxil-pc-shadermodel6.0-compute -std=hlsl202x -emit-llvm -disable-llvm-passes %s -o - | FileCheck %s --check-prefixes=CS,NOINLINE,CHECK
+// RUN: %clang_cc1 -triple dxil-pc-shadermodel6.3-library -std=hlsl202x -emit-llvm -disable-llvm-passes %s -o - | FileCheck %s --check-prefixes=LIB,NOINLINE,CHECK
+// RUN: %clang_cc1 -triple dxil-pc-shadermodel6.0-compute -std=hlsl202x -emit-llvm -O0 %s -o - | FileCheck %s --check-prefixes=INLINE,CHECK
+// RUN: %clang_cc1 -triple dxil-pc-shadermodel6.3-library -std=hlsl202x -emit-llvm -O0 %s -o - | FileCheck %s --check-prefixes=INLINE,CHECK
-// Make sure global variable for dtors exist for lib profile.
+// Tests that constructors and destructors are appropriately generated for globals
+// and that their calls are inlined when AlwaysInline is run
+// but global variables are retained for the library profiles
+
+// Make sure global variable for ctors/dtors exist for lib profile.
+// LIB:@llvm.global_ctors
// LIB:@llvm.global_dtors
-// Make sure global variable for dtors removed for compute profile.
-// CS-NOT:llvm.global_dtors
+// Make sure global variable for ctors/dtors removed for compute profile.
+// CS-NOT:@llvm.global_ctors
+// CS-NOT:@llvm.global_dtors
struct Tail {
Tail() {
@@ -46,22 +54,25 @@ void main(unsigned GI : SV_GroupIndex) {
Wag();
}
-// Make sure global variable for ctors/dtors removed.
-// CHECK-NOT:@llvm.global_ctors
-// CHECK-NOT:@llvm.global_dtors
-//CHECK: define void @main()
-//CHECK-NEXT: entry:
-//CHECK-NEXT: call void @_GLOBAL__sub_I_GlobalDestructors.hlsl()
-//CHECK-NEXT: %0 = call i32 @llvm.dx.flattened.thread.id.in.group()
-//CHECK-NEXT: call void @"?main@@YAXI@Z"(i32 %0)
-//CHECK-NEXT: call void @_GLOBAL__D_a()
-//CHECK-NEXT: ret void
+// CHECK: define void @main()
+// CHECK-NEXT: entry:
+// Verify destructor is emitted
+// NOINLINE-NEXT: call void @_GLOBAL__sub_I_GlobalDestructors.hlsl()
+// NOINLINE-NEXT: %0 = call i32 @llvm.dx.flattened.thread.id.in.group()
+// NOINLINE-NEXT: call void @"?main@@YAXI@Z"(i32 %0)
+// NOINLINE-NEXT: call void @_GLOBAL__D_a()
+// NOINLINE-NEXT: ret void
+// Verify inlining leaves only calls to "llvm." intrinsics
+// INLINE-NOT: call {{[^@]*}} @{{[^l][^l][^v][^m][^\.]}}
+// INLINE: ret void
// This is really just a sanity check I needed for myself to verify that
// function scope static variables also get destroyed properly.
-//CHECK: define internal void @_GLOBAL__D_a()
-//CHECK-NEXT: entry:
-//CHECK-NEXT: call void @"??1Tail@@QAA@XZ"(ptr @"?T@?1??Wag@@YAXXZ@4UTail@@A")
-//CHECK-NEXT: call void @"??1Pupper@@QAA@XZ"(ptr @"?GlobalPup@@3UPupper@@A")
-//CHECK-NEXT: ret void
+// NOINLINE: define internal void @_GLOBAL__D_a() [[IntAttr:\#[0-9]+]]
+// NOINLINE-NEXT: entry:
+// NOINLINE-NEXT: call void @"??1Tail@@QAA@XZ"(ptr @"?T@?1??Wag@@YAXXZ@4UTail@@A")
+// NOINLINE-NEXT: call void @"??1Pupper@@QAA@XZ"(ptr @"?GlobalPup@@3UPupper@@A")
+// NOINLINE-NEXT: ret void
+
+// NOINLINE: attributes [[IntAttr]] = {{.*}} alwaysinline
diff --git a/clang/test/CodeGenHLSL/builtins/RWBuffer-constructor.hlsl b/clang/test/CodeGenHLSL/builtins/RWBuffer-constructor.hlsl
index baddfcf2cf1d52..174f4c3eaaad26 100644
--- a/clang/test/CodeGenHLSL/builtins/RWBuffer-constructor.hlsl
+++ b/clang/test/CodeGenHLSL/builtins/RWBuffer-constructor.hlsl
@@ -1,3 +1,4 @@
+// RUN: %clang_cc1 -triple dxil-pc-shadermodel6.3-library -x hlsl -emit-llvm -disable-llvm-passes -o - %s | FileCheck %s
// RUN: %clang_cc1 -triple spirv-vulkan-library -x hlsl -emit-llvm -disable-llvm-passes -o - %s | FileCheck %s --check-prefix=CHECK-SPIRV
RWBuffer<float> Buf;
diff --git a/clang/test/CodeGenHLSL/builtins/RWBuffer-subscript.hlsl b/clang/test/CodeGenHLSL/builtins/RWBuffer-subscript.hlsl
index da8a1e538ec5e7..2a350c1619bd6e 100644
--- a/clang/test/CodeGenHLSL/builtins/RWBuffer-subscript.hlsl
+++ b/clang/test/CodeGenHLSL/builtins/RWBuffer-subscript.hlsl
@@ -11,6 +11,7 @@ void main(unsigned GI : SV_GroupIndex) {
// Even at -O0 the subscript operators get inlined. The -O0 IR is a bit messy
// and confusing to follow so the match here is pretty weak.
-// CHECK: define internal void @"?main@@YAXI@Z"
-// CHECK-NOT: call
+// CHECK: define void @main()
+// Verify inlining leaves only calls to "llvm." intrinsics
+// CHECK-NOT: call {{[^@]*}} @{{[^l][^l][^v][^m][^\.]}}
// CHECK: ret void
diff --git a/clang/test/CodeGenHLSL/inline-constructors.hlsl b/clang/test/CodeGenHLSL/inline-constructors.hlsl
new file mode 100644
index 00000000000000..40c7a42bdc262e
--- /dev/null
+++ b/clang/test/CodeGenHLSL/inline-constructors.hlsl
@@ -0,0 +1,74 @@
+// RUN: %clang_cc1 -triple dxil-pc-shadermodel6.0-compute -std=hlsl202x -emit-llvm -o - -disable-llvm-passes %s | FileCheck %s --check-prefixes=CHECK,NOINLINE
+// RUN: %clang_cc1 -triple dxil-pc-shadermodel6.3-library -std=hlsl202x -emit-llvm -o - -disable-llvm-passes %s | FileCheck %s --check-prefixes=CHECK,NOINLINE
+// RUN: %clang_cc1 -triple dxil-pc-shadermodel6.0-compute -std=hlsl202x -emit-llvm -o - -O0 %s | FileCheck %s --check-prefixes=CHECK,INLINE
+// RUN: %clang_cc1 -triple dxil-pc-shadermodel6.3-library -std=hlsl202x -emit-llvm -o - -O0 %s | FileCheck %s --check-prefixes=CHECK,INLINE
+
+// Tests that implicit contstructor calls for user classes will always be inlined.
+
+struct Weed {
+ Weed() {Count += 1;}
+ [[maybe_unused]] void pull() {Count--;}
+ static int weedCount() { return Count; }
+private:
+ static int Count;
+
+} YardWeeds;
+
+int Weed::Count = 1; // It begins. . .
+
+struct Kitty {
+ unsigned burrsInFur;
+
+ Kitty() {
+ burrsInFur = 0;
+ }
+
+ void wanderInYard(int hours) {
+ burrsInFur = hours*Weed::weedCount()/8;
+ }
+
+ void lick() {
+ if(burrsInFur) {
+ burrsInFur--;
+ Weed w;
+ }
+ }
+
+} Nion;
+
+void NionsDay(int hours) {
+ static Kitty Nion;
+ Nion.wanderInYard(hours);
+ while(Nion.burrsInFur) Nion.lick();
+}
+
+// CHECK: define void @main()
+// CHECK-NEXT: entry:
+// Verify constructor is emitted
+// NOINLINE-NEXT: call void @_GLOBAL__sub_I_inline_constructors.hlsl()
+// NOINLINE-NEXT: %0 = call i32 @llvm.dx.flattened.thread.id.in.group()
+// NOINLINE-NEXT: call void @"?main@@YAXI@Z"(i32 %0)
+// Verify inlining leaves only calls to "llvm." intrinsics
+// INLINE-NOT: call {{[^@]*}} @{{[^l][^l][^v][^m][^\.]}}
+// CHECK: ret void
+[shader("compute")]
+[numthreads(1,1,1)]
+void main(unsigned GI : SV_GroupIndex) {
+ NionsDay(10);
+}
+
+
+// CHECK: define void @rainyMain()
+// CHECK-NEXT: entry:
+// Verify constructor is emitted
+// NOINLINE-NEXT: call void @_GLOBAL__sub_I_inline_constructors.hlsl()
+// NOINLINE-NEXT: call void @"?rainyMain@@YAXXZ"()
+// Verify inlining leaves only calls to "llvm." intrinsics
+// INLINE-NOT: call {{[^@]*}} @{{[^l][^l][^v][^m][^\.]}}
+// CHECK: ret void
+[shader("compute")]
+[numthreads(1,1,1)]
+void rainyMain() {
+ NionsDay(1);
+}
+
diff --git a/clang/test/CodeGenHLSL/inline-functions.hlsl b/clang/test/CodeGenHLSL/inline-functions.hlsl
new file mode 100644
index 00000000000000..9c00e096a9eef2
--- /dev/null
+++ b/clang/test/CodeGenHLSL/inline-functions.hlsl
@@ -0,0 +1,114 @@
+// RUN: %clang_cc1 -x hlsl -triple dxil-pc-shadermodel6.3-library %s -emit-llvm -disable-llvm-passes -o - | FileCheck %s --check-prefixes=CHECK,NOINLINE
+// RUN: %clang_cc1 -x hlsl -triple dxil-pc-shadermodel6.3-library %s -emit-llvm -O0 -o - | FileCheck %s --check-prefixes=CHECK,INLINE
+// RUN: %clang_cc1 -x hlsl -triple dxil-pc-shadermodel6.0-compute %s -emit-llvm -disable-llvm-passes -o - | FileCheck %s --check-prefixes=CHECK,NOINLINE
+// RUN: %clang_cc1 -x hlsl -triple dxil-pc-shadermodel6.0-compute %s -emit-llvm -O0 -o - | FileCheck %s --check-prefixes=CHECK,INLINE
+
+// Tests that user functions will always be inlined.
+// This includes exported functions and mangled entry point implementation functions.
+// The unmangled entry functions must not be alwaysinlined.
+
+#define MAX 100
+
+float nums[MAX];
+
+// Verify that all functions have the alwaysinline attribute
+// CHECK: Function Attrs: alwaysinline
+// CHECK: define void @"?swap@@YAXY0GE@III@Z"(ptr noundef byval([100 x i32]) align 4 %Buf, i32 noundef %ix1, i32 noundef %ix2) [[IntAttr:\#[0-9]+]]
+// CHECK: ret void
+// Swap the values of Buf at indices ix1 and ix2
+void swap(unsigned Buf[MAX], unsigned ix1, unsigned ix2) {
+ float tmp = Buf[ix1];
+ Buf[ix1] = Buf[ix2];
+ Buf[ix2] = tmp;
+}
+
+// CHECK: Function Attrs: alwaysinline
+// CHECK: define void @"?BubbleSort@@YAXY0GE@II@Z"(ptr noundef byval([100 x i32]) align 4 %Buf, i32 noundef %size) [[IntAttr]]
+// CHECK: ret void
+// Inefficiently sort Buf in place
+void BubbleSort(unsigned Buf[MAX], unsigned size) {
+ bool swapped = true;
+ while (swapped) {
+ swapped = false;
+ for (unsigned i = 1; i < size; i++) {
+ if (Buf[i] < Buf[i-1]) {
+ swap(Buf, i, i-1);
+ swapped = true;
+ }
+ }
+ }
+}
+
+// Note ExtAttr is the inlined export set of attribs
+// CHECK: Function Attrs: alwaysinline
+// CHECK: define noundef i32 @"?RemoveDupes@@YAIY0GE@II@Z"(ptr noundef byval([100 x i32]) align 4 %Buf, i32 noundef %size) [[ExtAttr:\#[0-9]+]]
+// CHECK: ret i32
+// Sort Buf and remove any duplicate values
+// returns the number of values left
+export
+unsigned RemoveDupes(unsigned Buf[MAX], unsigned size) {
+ BubbleSort(Buf, size);
+ unsigned insertPt = 0;
+ for (unsigned i = 1; i < size; i++) {
+ if (Buf[i] == Buf[i-1])
+ insertPt++;
+ else
+ Buf[insertPt] = Buf[i];
+ }
+ return insertPt;
+}
+
+
+RWBuffer<unsigned> Indices;
+
+// The mangled version of main only remains without inlining
+// because it has internal linkage from the start
+// Note main functions get the norecurse attrib, which IntAttr reflects
+// NOINLINE: Function Attrs: alwaysinline
+// NOINLINE: define internal void @"?main@@YAXI@Z"(i32 noundef %GI) [[IntAttr]]
+// NOINLINE: ret void
+
+// The unmangled version is not inlined, EntryAttr reflects that
+// CHECK: Function Attrs: convergent norecurse
+// CHECK: define void @main() [[EntryAttr:\#[0-9]+]]
+// Make sure function calls are inlined when AlwaysInline is run
+// This only leaves calls to llvm. intrinsics
+// INLINE-NOT: call {{[^@]*}} @{{[^l][^l][^v][^m][^\.]}}
+// CHECK: ret void
+
+[numthreads(1,1,1)]
+[shader("compute")]
+void main(unsigned int GI : SV_GroupIndex) {
+ unsigned tmpIndices[MAX];
+ if (GI > MAX) return;
+ for (unsigned i = 1; i < GI; i++)
+ tmpIndices[i] = Indices[i];
+ RemoveDupes(tmpIndices, GI);
+ for (unsigned i = 1; i < GI; i++)
+ tmpIndices[i] = Indices[i];
+}
+
+// The mangled version of main only remains without inlining
+// because it has internal linkage from the start
+// Note main functions get the norecurse attrib, which IntAttr reflects
+// NOINLINE: Function Attrs: alwaysinline
+// NOINLINE: define internal void @"?main10@@YAXXZ"() [[IntAttr]]
+// NOINLINE: ret void
+
+// The unmangled version is not inlined, EntryAttr reflects that
+// CHECK: Function Attrs: convergent norecurse
+// CHECK: define void @main10() [[EntryAttr]]
+// Make sure function calls are inlined when AlwaysInline is run
+// This only leaves calls to llvm. intrinsics
+// INLINE-NOT: call {{[^@]*}} @{{[^l][^l][^v][^m][^\.]}}
+// CHECK: ret void
+
+[numthreads(1,1,1)]
+[shader("compute")]
+void main10() {
+ main(10);
+}
+
+// CHECK: attributes [[IntAttr]] = {{.*}} alwaysinline
+// CHECK: attributes [[ExtAttr]] = {{.*}} alwaysinline
+// CHECK-NOT: attributes [[EntryAttr]] = {{.*}} alwaysinline
|
Previously, the alwaysinline attribute was set for the RWBuffer subscript operator as soon as it was created. Since they are all in a common place now, that setting was redundant, but it does mean that it won't show up in the AST
llvm-beanz
reviewed
Aug 30, 2024
clang/lib/CodeGen/CGHLSLRuntime.cpp
Outdated
| } else { // "implicit" autogenerated functions with no declaration | ||
| // Implicit functions might get marked as noinline by default | ||
| // but we override that for HLSL | ||
| Fn->removeFnAttr(Attribute::NoInline); |
There was a problem hiding this comment.
Which auto-generated functions are getting noinline attached to them? Do you know where that is happening?
There was a problem hiding this comment.
It is applied here:
llvm-project/clang/lib/CodeGen/CodeGenModule.cpp
Lines 2477 to 2479 in df159d3
on functions not explicitly created by the user, but generated to construct or destruct global/static local variables as well as the global functions that call each of these at the top and bottom of the entry function.
There was a problem hiding this comment.
It seems to me like it might make more sense to change that condition to not do that for HLSL rather than to modify the IR after we generate it incorrectly.
llvm-beanz
reviewed
Aug 30, 2024
| @@ -290,8 +290,6 @@ struct BuiltinTypeDeclBuilder { | |||
| SourceLocation())); | |||
| MethodDecl->setLexicalDeclContext(Record); | |||
| MethodDecl->setAccess(AccessSpecifier::AS_public); | |||
| MethodDecl->addAttr(AlwaysInlineAttr::CreateImplicit( | |||
| AST, SourceRange(), AlwaysInlineAttr::CXX11_clang_always_inline)); | |||
There was a problem hiding this comment.
Why was it necessary to remove this? I think we want this because we'll want these calls inlined even in separate compilation cases.
There was a problem hiding this comment.
It wasn't necessary so much as redundant for the compilations I tested and actually gets applied to the LLVM function representation after that set by setHLSLFunctionAttributes.
How would I test a relevant separate compilation case?
There was a problem hiding this comment.
We don't really have a workflow in Clang implemented yet for separate compilation. The analog in DXC would be something like -T lib_6_x -default-linkage=external. In that case we would still want these functions to be inlined, and we can accomplish that with this attribute.
There was a problem hiding this comment.
Would that involve generating just AST then? The DXC analog would still be generating LLVM IR, which this setting would catch and does for the inline-constructors.hlsl test among others.
Restore additional alwaysinline setting for RWBuffer subscripts Add HLSL check to prevent marking autogenerated functions as noinline
pow3clk
reviewed
Sep 4, 2024
clang/lib/CodeGen/CGHLSLRuntime.cpp
Outdated
| } else { // "implicit" autogenerated functions with no declaration | ||
| // Implicit functions might get marked as noinline by default | ||
| // but we override that for HLSL | ||
| Fn->removeFnAttr(Attribute::NoInline); |
| @@ -290,8 +290,6 @@ struct BuiltinTypeDeclBuilder { | |||
| SourceLocation())); | |||
| MethodDecl->setLexicalDeclContext(Record); | |||
| MethodDecl->setAccess(AccessSpecifier::AS_public); | |||
| MethodDecl->addAttr(AlwaysInlineAttr::CreateImplicit( | |||
| AST, SourceRange(), AlwaysInlineAttr::CXX11_clang_always_inline)); | |||
There was a problem hiding this comment.
Would that involve generating just AST then? The DXC analog would still be generating LLVM IR, which this setting would catch and does for the inline-constructors.hlsl test among others.
For whatever reason a function might have noinline, if we try to set alwaysinline, we get an incompatible attribute error. This avoids that in any case.
|
✅ With the latest revision this PR passed the C/C++ code formatter. |
python3kgae
approved these changes
Sep 5, 2024
damyanp
reviewed
Sep 5, 2024
Move all logic into SetLLVMFunctionAttributesForDefinition where most inlining attribute calculations are done. The way autogenerated init functions and user functions were created meant that this was hit at different times for each relative to setHLSLFunctionAttributes. Instead of trying to resolve that fundamental issue or work around it, better to keep all the setting in the same place so it doesn't matter where it is called relative to function body creation. This required making exceptions for entry functions as they will get attributes from this code as well and we want it to not be always inlined and receive the relevant optimization attributes that allows.
llvm-beanz
reviewed
Sep 9, 2024
clang/lib/CodeGen/CodeGenModule.cpp
Outdated
| @@ -2471,11 +2471,14 @@ void CodeGenModule::SetLLVMFunctionAttributesForDefinition(const Decl *D, | |||
| B.addAttribute(llvm::Attribute::StackProtectReq); | |||
|
|
|||
| if (!D) { | |||
| // HLSL functions must always be inlined | |||
| if (getLangOpts().HLSL && !F->hasFnAttribute("hlsl.shader")) | |||
There was a problem hiding this comment.
What if we put noinline on the entry function? Does that simplify this logic? Since we'll also at some point need to support the noinline keyword we'll need to be checking that attribute somewhere.
There was a problem hiding this comment.
In earlier discussion, we resolved to make this the default and not worry about noinline as of yet seeing as there are a lot of issues to resolve before that can work properly. In a previous version, I was forced to check for noinline as it was a sign that a function had passed through this function, but my preference is to not respect it until we've worked out the issues that prevent it from working right.
It wouldn't simplify the logic since the entry function doesn't get marked inline until it is passed into this function.
There was a problem hiding this comment.
Just an update. As a result of the waning minutes of a design discussion, we resolved to produce a warning when users apply noinline and explicitly set noinline for the outermost entry function which would allow checking for that in place of hlsl.shader here, which might allow graceful resolution of any other functions that happen to get it applied here which we could have an assert for.
llvm-beanz
reviewed
Sep 9, 2024
llvm-beanz
reviewed
Sep 9, 2024
damyanp
approved these changes
Sep 10, 2024
Set noinline explicitly on the outermost entry function and use that attribute to decide to apply alwaysinline. Generate a warning and ignore user-specified noinline Incidentally tidy up some RUN lines
llvm-beanz
reviewed
Sep 12, 2024
clang/lib/CodeGen/CodeGenModule.cpp
Outdated
| if (D->hasAttr<NoInlineAttr>()) | ||
| getDiags().Report(D->getLocation(), | ||
| diag::warn_unsupported_attribute_ignored) | ||
| << "noinline" << "HLSL"; |
There was a problem hiding this comment.
Why are we warning on this? Does the Clang implementation not work?
If we're going to warn on this, it needs to be in Sema not CodeGen!
llvm-beanz
reviewed
Sep 12, 2024
clang/test/SemaHLSL/noinline.hlsl
Outdated
| @@ -0,0 +1,13 @@ | |||
| // RUN: %clang_cc1 -triple dxil-pc-shadermodel6.0-compute -emit-llvm-only %s -verify | |||
There was a problem hiding this comment.
We should not need to emit IR to test a warning. This should work with -fsyntax-only or without the -emit-llvm-only flag.
llvm-beanz
approved these changes
Sep 17, 2024
|
LLVM Buildbot has detected a new failure on builder Full details are available at: https://lab.llvm.org/buildbot/#/builders/144/builds/7235 Here is the relevant piece of the build log for the reference |
|
LLVM Buildbot has detected a new failure on builder Full details are available at: https://lab.llvm.org/buildbot/#/builders/190/builds/5917 Here is the relevant piece of the build log for the reference |
|
LLVM Buildbot has detected a new failure on builder Full details are available at: https://lab.llvm.org/buildbot/#/builders/168/builds/3425 Here is the relevant piece of the build log for the reference |
|
LLVM Buildbot has detected a new failure on builder Full details are available at: https://lab.llvm.org/buildbot/#/builders/153/builds/9181 Here is the relevant piece of the build log for the reference |
thurstond
added a commit
that referenced
this pull request
Sep 17, 2024
This reverts commit a729e70. Reason:bBuildbot failure (https://lab.llvm.org/buildbot/#/builders/25/builds/2541): 'Clang :: CodeGenHLSL/builtins/StructuredBuffer-subscript.hlsl' failed
|
Looks like the revert broke check-clang: http://45.33.8.238/linux/148045/step_6.txt Please take a look, and uh, mark that test as UNSUPPORTED if it takes a while to fix, I suppose. |
|
Oh, actually, I just realized the test update was the fix-forward. So the correct fix would be to reland both the change and the test update. Sorry! |
thurstond
added a commit
that referenced
this pull request
Sep 17, 2024
|
I've reverted both of my reverts |
|
Sorry I didn't mention here that the fix was in. It looks like everything is clean now? |
Yes, both your changes are back in and I anticipate the buildbots will be green again. Sorry for the noise! |
|
LLVM Buildbot has detected a new failure on builder Full details are available at: https://lab.llvm.org/buildbot/#/builders/56/builds/7605 Here is the relevant piece of the build log for the reference |
|
LLVM Buildbot has detected a new failure on builder Full details are available at: https://lab.llvm.org/buildbot/#/builders/60/builds/7827 Here is the relevant piece of the build log for the reference |
hamphet
pushed a commit
to hamphet/llvm-project
that referenced
this pull request
Sep 18, 2024
HLSL inlines all its functions by default. This uses the alwaysinline attribute to make the alwaysinliner pass inline any function not explicitly marked noinline by the user or autogeneration. The alwayslinline marking takes place in `SetLLVMFunctionAttributesForDefinitions` where all other inlining interactions are determined. The outermost entry function is marked noinline because there's no reason to inline it. Any user calls to an entry function will instead call the internal mangled version of the entry function. Adds tests for function and constructor inlining and augments some existing tests to verify correct inlining of implicitly created functions as well. Incidentally restore RUN line that I believe was mistakenly removed as part of llvm#88918 Fixes llvm#89282
hamphet
pushed a commit
to hamphet/llvm-project
that referenced
this pull request
Sep 18, 2024
This reverts commit a729e70. Reason:bBuildbot failure (https://lab.llvm.org/buildbot/#/builders/25/builds/2541): 'Clang :: CodeGenHLSL/builtins/StructuredBuffer-subscript.hlsl' failed
hamphet
pushed a commit
to hamphet/llvm-project
that referenced
this pull request
Sep 18, 2024
llvm#109023)" This reverts commit f4fe26d. Reason: 4a63f4d reverted "[HLSL] set alwaysinline on HLSL functions (llvm#106588)" due to a buildbot failure; this test (which builds upon the reverted patch) also needs to be reverted.
hamphet
pushed a commit
to hamphet/llvm-project
that referenced
this pull request
Sep 18, 2024
This reverts commit 4a63f4d. It was reverted because of a buildbot breakage, but the fix-forward has landed (llvm#109023).
tmsri
pushed a commit
to tmsri/llvm-project
that referenced
this pull request
Sep 19, 2024
HLSL inlines all its functions by default. This uses the alwaysinline attribute to make the alwaysinliner pass inline any function not explicitly marked noinline by the user or autogeneration. The alwayslinline marking takes place in `SetLLVMFunctionAttributesForDefinitions` where all other inlining interactions are determined. The outermost entry function is marked noinline because there's no reason to inline it. Any user calls to an entry function will instead call the internal mangled version of the entry function. Adds tests for function and constructor inlining and augments some existing tests to verify correct inlining of implicitly created functions as well. Incidentally restore RUN line that I believe was mistakenly removed as part of llvm#88918 Fixes llvm#89282
tmsri
pushed a commit
to tmsri/llvm-project
that referenced
this pull request
Sep 19, 2024
This reverts commit a729e70. Reason:bBuildbot failure (https://lab.llvm.org/buildbot/#/builders/25/builds/2541): 'Clang :: CodeGenHLSL/builtins/StructuredBuffer-subscript.hlsl' failed
tmsri
pushed a commit
to tmsri/llvm-project
that referenced
this pull request
Sep 19, 2024
llvm#109023)" This reverts commit f4fe26d. Reason: 4a63f4d reverted "[HLSL] set alwaysinline on HLSL functions (llvm#106588)" due to a buildbot failure; this test (which builds upon the reverted patch) also needs to be reverted.
tmsri
pushed a commit
to tmsri/llvm-project
that referenced
this pull request
Sep 19, 2024
This reverts commit 4a63f4d. It was reverted because of a buildbot breakage, but the fix-forward has landed (llvm#109023).
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
HLSL inlines all its functions by default. This uses the alwaysinline attribute to force that in the corresponding pass for user functions by default and overrides the default noinline of some implicit functions. This makes an instance of explicit inlining for buffer subscripts unnecessary.
Adds tests for function and constructor inlining and augments some existing tests to verify correct inlining of implicitly created functions as well.
incidentally restore RUN line that I believe was mistakenly removed as part of #88918
fixes #89282