Skip to content

Commit

Permalink
Merge branch 'master' into feature/wgsl_intrinsic_texture
Browse files Browse the repository at this point in the history
  • Loading branch information
jkwak-work authored Sep 20, 2024
2 parents 2d7dd10 + b4c851f commit eb7fad3
Show file tree
Hide file tree
Showing 108 changed files with 4,069 additions and 1,632 deletions.
4 changes: 3 additions & 1 deletion CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -101,6 +101,8 @@ option(SLANG_ENABLE_TESTS "Enable test targets, some tests may require SLANG_ENA
option(SLANG_ENABLE_EXAMPLES "Enable example targets, requires SLANG_ENABLE_GFX" ON)
option(SLANG_ENABLE_REPLAYER "Enable slang-replay tool" ON)

option(SLANG_GITHUB_TOKEN "Use a given token value for accessing Github REST API" "")

enum_option(
SLANG_LIB_TYPE
# Default
Expand Down Expand Up @@ -136,7 +138,7 @@ enum_option(
if(SLANG_SLANG_LLVM_FLAVOR MATCHES FETCH_BINARY)
# If the user didn't specify a URL, find the best one now
if(NOT SLANG_SLANG_LLVM_BINARY_URL)
get_best_slang_binary_release_url(url)
get_best_slang_binary_release_url("${SLANG_GITHUB_TOKEN}" url)
if(NOT DEFINED url)
message(FATAL_ERROR "Unable to find binary release for slang-llvm, please set a different SLANG_SLANG_LLVM_FLAVOR or set SLANG_SLANG_LLVM_BINARY_URL manually")
endif()
Expand Down
28 changes: 23 additions & 5 deletions cmake/GitHubRelease.cmake
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
function(check_release_and_get_latest owner repo version os arch out_var)
function(check_release_and_get_latest owner repo version os arch github_token out_var)
# Construct the URL for the specified version's release API endpoint
set(version_url "https://api.github.com/repos/${owner}/${repo}/releases/tags/v${version}")

Expand All @@ -17,8 +17,22 @@ function(check_release_and_get_latest owner repo version os arch out_var)
set(${found_var} "${found}" PARENT_SCOPE)
endfunction()

# Download the specified release info from GitHub
file(DOWNLOAD "${version_url}" "${json_output_file}" STATUS download_statuses)
# Prepare download arguments
set(download_args
"${version_url}"
"${json_output_file}"
STATUS download_statuses
)

if(github_token)
# Add authorization header if token is provided
list(APPEND download_args HTTPHEADER "Authorization: token ${github_token}")
endif()

# Perform the download
file(DOWNLOAD ${download_args})

# Check if the downloading was successful
list(GET download_statuses 0 status_code)
if(status_code EQUAL 0)
file(READ "${json_output_file}" json_content)
Expand All @@ -34,6 +48,10 @@ function(check_release_and_get_latest owner repo version os arch out_var)
message(WARNING "Failed to find ${desired_zip} in release assets for ${version} from ${version_url}\nFalling back to latest version if it differs")
else()
message(WARNING "Failed to download release info for version ${version} from ${version_url}\nFalling back to latest version if it differs")

if(status_code EQUAL 22)
message(WARNING "If API rate limit is exceeded, Github allows a higher limit when you use token. Try a cmake option -DSLANG_GITHUB_TOKEN=your_token_here")
endif()
endif()


Expand Down Expand Up @@ -69,7 +87,7 @@ function(check_release_and_get_latest owner repo version os arch out_var)
endif()
endfunction()

function(get_best_slang_binary_release_url out_var)
function(get_best_slang_binary_release_url github_token out_var)
if(CMAKE_SYSTEM_PROCESSOR MATCHES "x86_64|amd64|AMD64")
set(arch "x86_64")
elseif(CMAKE_SYSTEM_PROCESSOR MATCHES "aarch64|ARM64|arm64")
Expand All @@ -93,7 +111,7 @@ function(get_best_slang_binary_release_url out_var)
set(owner "shader-slang")
set(repo "slang")

check_release_and_get_latest(${owner} ${repo} ${SLANG_VERSION_NUMERIC} ${os} ${arch} release_version)
check_release_and_get_latest(${owner} ${repo} ${SLANG_VERSION_NUMERIC} ${os} ${arch} "${github_token}" release_version)
if(DEFINED release_version)
set(${out_var} "https://github.com/${owner}/${repo}/releases/download/v${release_version}/slang-${release_version}-${os}-${arch}.zip" PARENT_SCOPE)
endif()
Expand Down
116 changes: 116 additions & 0 deletions docs/proposals/003-atomic-t.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,116 @@
SP #003 - `Atomic<T>` type
==============


Status
------

Author: Yong He

Status: Design Discussion.

Implementation: N/A

Reviewed by: N/A

Background
----------

HLSL defines atomic intrinsics to work on free references to ordinary values such as `int` and `float`. However, this doesn't translate well to Metal and WebGPU,
which defines `atomic<T>` type and only allow atomic operations to be applied on values of `atomic<T>` types.

Slang's Metal backend follows the same technique in SPIRV-Cross and DXIL->Metal converter that relies on a C++ undefined behavior that casts an ordinary `int*` pointer to a `atomic<int>*` pointer
and then call atomic intrinsic on the reinterpreted pointer. This is fragile and not guaranteed to work in the future.

To make the situation worse, WebGPU bans all possible ways to cast a normal pointer into an `atomic` pointer. In order to provide a truly portable way to define
atomic operations and allow them to be translatable to all targets, we will also need an `atomic<T>` type in Slang that maps to `atomic<T>` in WGSL and Metal, and maps to
`T` for HLSL/SPIRV.


Proposed Approach
-----------------

We define an `Atomic<T>` type that functions as a wrapper of `T` and provides atomic operations:
```
[sealed] interface IAtomicable {}
[sealed] interface IArithmeticAtomicable : IAtomicable {}
[sealed] interface IBitAtomicable : IArithmeticAtomicable {}
extension int : IArithmeticAtomicable {}
extension uint : IArithmeticAtomicable {}
extension int64_t : IBitAtomicable {}
extension uint64_t : IBitAtomicable {}
extension float : IArithmeticAtomicable {}
extension half : IArithmeticAtomicable {}
struct Atomic<T : IAtomicable>
{
T load();
[ref] void store(T newValue); // Question: do we really need this?
[ref] T exchange(T newValue); // returns old value
[ref] T compareExchange(T compareValue, T newValue); // returns old value.
}
extension<T:IArithmeticAtomicable> Atomic<T>
{
[ref] T atomicAdd(T value); // returns original value
[ref] T atomicSub(T value); // returns original value
[ref] T atomicMax(T value); // returns original value
[ref] T atomicMin(T value); // returns original value
[ref] T atomicIncrement();
[ref] T atomicDecrement();
}
extension<T:IBitAtomicable> Atomic<T>
{
[ref] T atomicAnd(T value); // returns original value
[ref] T atomicOr(T value); // returns original value
[ref] T atomicXor(T value); // returns original value
}
```

We allow `Atomic<T>` to be defined anywhere: as struct fields, as array elements, as elements of `RWStructuredBuffer` types,
or as local, global and groupshared variable types or function parameter types. For example, in global memory:

```hlsl
struct MyType
{
int ordinaryValue;
Atomic<int> atomicValue;
}
RWStructuredBuffer<MyType> atomicBuffer;
void main()
{
atomicBuffer[0].atomicValue.atomicAdd(1);
printf("%d", atomicBuffer[0].atomicValue.load());
}
```

In groupshared memory:

```hlsl
void main()
{
groupshared atomic<int> c;
c.atomicAdd(1);
}
```

When generating WGSL code where `atomic<T>` isn't allowed on local variables or other illegal address spaces, we will lower the type
into its underlying type. The use of atomic type in these positions will simply have no meaning. A caveat is what the semantics should be
when there is a function that takes `inout Atomic<T>` as parameter. This likely need to be a warning or error.

This should be handled by a legalization pass similar to `lowerBufferElementTypeToStorageType` but operates
in the opposite direction: the "loaded" value from a buffer is converted into an atomic-free type, and storing a value leads to an
atomic store at the corresponding locations.

For non-WGSL/Metal targets, we can simply lower the type out of existence into its underlying type.

# Related Work

`Atomic<T>` type exists in almost all CPU programming languages and is the proven way to express atomic operations over different
architectures that have different memory models. WGSL and Metal follows this trend to require atomic operations being expressed
this way. This proposal is to make Slang follow this trend and make `Atomic<T>` the recommened way to express atomic operation
going forward.
3 changes: 2 additions & 1 deletion docs/user-guide/06-interfaces-generics.md
Original file line number Diff line number Diff line change
Expand Up @@ -1036,7 +1036,8 @@ Slang supports the following builtin interfaces:
- `IInteger`, represents a logical integer that supports both `IArithmetic` and `ILogical` operations. Implemented by all builtin integer scalar types.
- `IDifferentiable`, represents a value that is differentiable.
- `IFloat`, represents a logical float that supports both `IArithmetic`, `ILogical` and `IDifferentiable` operations. Also provides methods to convert to and from `float`. Implemented by all builtin floating-point scalar, vector and matrix types.
- `IArray<T>`, represents a logical array that supports retrieving an element of type `T` from an index. Implemented by array types, vectors and matrices.
- `IArray<T>`, represents a logical array that supports retrieving an element of type `T` from an index. Implemented by array types, vectors, matrices and `StructuredBuffer`.
- `IRWArray<T>`, represents a logical array whose elements are mutable. Implemented by array types, vectors, matrices, `RWStructuredBuffer` and `RasterizerOrderedStructuredBuffer`.
- `IFunc<TResult, TParams...>` represent a callable object (with `operator()`) that returns `TResult` and takes `TParams...` as argument.
- `IMutatingFunc<TResult, TParams...>`, similar to `IFunc`, but the `operator()` method is `[mutating]`.
- `IDifferentiableFunc<TResult, TParams...>`, similar to `IFunc`, but the `operator()` method is `[Differentiable]`.
Expand Down
Loading

0 comments on commit eb7fad3

Please sign in to comment.