-
Notifications
You must be signed in to change notification settings - Fork 5.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
8312425: [vectorapi] AArch64: Optimize vector math operations with SLEEF #18605
Closed
Hamlin-Li
wants to merge
35
commits into
openjdk:master
from
Hamlin-Li:sleef-aarch64-integrate-source
Closed
Changes from 23 commits
Commits
Show all changes
35 commits
Select commit
Hold shift + click to select a range
5ecbc2d
8312425: [vectorapi] AArch64: Optimize vector math operations with SLEEF
46374af
Disable sleef by default
1ea5f7d
Add a bundled native lib in jdk as a bridge to libsleef
85398fa
Address review comments in build system
6a3f78f
Rename vmath to sleef in configure
4ec3420
Separate neon and sve functions into two source files
f210779
Add "--with-libsleef-lib" and "--with-libsleef-include" options
6d7afe8
Remove -fvisibility in makefile and add the attribute in source code
b883ac4
Fix potential attribute issue
7aadb60
Merge branch 'master' into sleef-aarch64
e9ce59c
add libsleef as extra lib in CI cross-build on aarch64
b64f7e7
distinguish call names between non-scalable and scalable(e.g. sve)
55b118d
copyright date
668d08c
fix variable name in github workflow
7165338
resolve magicus's comments
7b7ec31
rename
d0ed032
fix jni includes
626d9fa
add [generated] src from sleef
57e23e7
resolve build erorrs
55277be
remove unnecessary changes
afe2cba
merge master
06ff2d8
add maintenance nodes
3ab4795
minor
cd68dc1
minor
34529ff
disable unused-function warnings; add log msg
cd70f5a
fix performance issue
cbcd463
remove notes about sleef changes
bd9c093
add inline header file for riscv64
36415c3
update header files for arm
7891617
sleef 3.6.1
c279a3c
sleef 3.6.1 for riscv
fe4be2c
merge master
b54fc86
Merge branch 'master' into sleef-aarch64-integrate-source
da65cfa
minor
6061c25
skip TANH
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -42,6 +42,7 @@ | |
#include "oops/oop.inline.hpp" | ||
#include "prims/methodHandles.hpp" | ||
#include "prims/upcallLinker.hpp" | ||
#include "runtime/arguments.hpp" | ||
#include "runtime/atomic.hpp" | ||
#include "runtime/continuation.hpp" | ||
#include "runtime/continuationEntry.inline.hpp" | ||
|
@@ -8535,6 +8536,72 @@ class StubGenerator: public StubCodeGenerator { | |
if (UseAdler32Intrinsics) { | ||
StubRoutines::_updateBytesAdler32 = generate_updateBytesAdler32(); | ||
} | ||
|
||
#ifdef COMPILER2 | ||
// Get native vector math stub routine addresses | ||
void* libvectormath = nullptr; | ||
char ebuf[1024]; | ||
char dll_name[JVM_MAXPATHLEN]; | ||
if (os::dll_locate_lib(dll_name, sizeof(dll_name), Arguments::get_dll_dir(), "vectormath")) { | ||
libvectormath = os::dll_load(dll_name, ebuf, sizeof ebuf); | ||
} | ||
if (libvectormath != nullptr) { | ||
// Method naming convention | ||
// All the methods are named as <OP><T><N>_<U><suffix> | ||
// Where: | ||
// <OP> is the operation name, e.g. sin | ||
// <T> is optional to indicate float/double | ||
// "f/d" for vector float/double operation | ||
// <N> is the number of elements in the vector | ||
// "2/4" for neon, and "x" for sve | ||
// <U> is the precision level | ||
// "u10/u05" represents 1.0/0.5 ULP error bounds | ||
// We use "u10" for all operations by default | ||
// But for those functions do not have u10 support, we use "u05" instead | ||
// <suffix> indicates neon/sve | ||
// "sve/advsimd" for sve/neon implementations | ||
// e.g. sinfx_u10sve is the method for computing vector float sin using SVE instructions | ||
// cosd2_u10advsimd is the method for computing 2 elements vector double cos using NEON instructions | ||
// | ||
log_info(library)("Loaded library %s, handle " INTPTR_FORMAT, JNI_LIB_PREFIX "vectormath" JNI_LIB_SUFFIX, p2i(libvectormath)); | ||
|
||
// Math vector stubs implemented with SVE for scalable vector size. | ||
if (UseSVE > 0) { | ||
for (int op = 0; op < VectorSupport::NUM_VECTOR_OP_MATH; op++) { | ||
int vop = VectorSupport::VECTOR_OP_MATH_START + op; | ||
|
||
// The native library does not support u10 level of "hypot". | ||
const char* ulf = (vop == VectorSupport::VECTOR_OP_HYPOT) ? "u05" : "u10"; | ||
|
||
snprintf(ebuf, sizeof(ebuf), "%sfx_%ssve", VectorSupport::mathname[op], ulf); | ||
StubRoutines::_vector_f_math[VectorSupport::VEC_SIZE_SCALABLE][op] = (address)os::dll_lookup(libvectormath, ebuf); | ||
|
||
snprintf(ebuf, sizeof(ebuf), "%sdx_%ssve", VectorSupport::mathname[op], ulf); | ||
StubRoutines::_vector_d_math[VectorSupport::VEC_SIZE_SCALABLE][op] = (address)os::dll_lookup(libvectormath, ebuf); | ||
} | ||
} | ||
|
||
// Math vector stubs implemented with NEON for 64/128 bits vector size. | ||
for (int op = 0; op < VectorSupport::NUM_VECTOR_OP_MATH; op++) { | ||
int vop = VectorSupport::VECTOR_OP_MATH_START + op; | ||
|
||
// The native library does not support u10 level of "hypot". | ||
const char* ulf = (vop == VectorSupport::VECTOR_OP_HYPOT) ? "u05" : "u10"; | ||
|
||
snprintf(ebuf, sizeof(ebuf), "%sf4_%sadvsimd", VectorSupport::mathname[op], ulf); | ||
StubRoutines::_vector_f_math[VectorSupport::VEC_SIZE_64][op] = (address)os::dll_lookup(libvectormath, ebuf); | ||
|
||
snprintf(ebuf, sizeof(ebuf), "%sf4_%sadvsimd", VectorSupport::mathname[op], ulf); | ||
StubRoutines::_vector_f_math[VectorSupport::VEC_SIZE_128][op] = (address)os::dll_lookup(libvectormath, ebuf); | ||
|
||
snprintf(ebuf, sizeof(ebuf), "%sd2_%sadvsimd", VectorSupport::mathname[op], ulf); | ||
StubRoutines::_vector_d_math[VectorSupport::VEC_SIZE_128][op] = (address)os::dll_lookup(libvectormath, ebuf); | ||
} | ||
} else { | ||
log_info(library)("Failed to load native vector math library!"); | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Include the |
||
} | ||
#endif // COMPILER2 | ||
|
||
#endif // COMPILER2_OR_JVMCI | ||
} | ||
|
||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should the unused-function be passed in using
DISABLE_WARNINGS_*
instead?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks!
Good suggestion, it makes the output clean.