Skip to content

Commit

Permalink
Merge pull request #25 from rjodinchr/dev
Browse files Browse the repository at this point in the history
Add input/output buffers in runner
  • Loading branch information
rjodinchr authored Apr 19, 2024
2 parents 7d8373f + e9778aa commit 4f1f3e7
Show file tree
Hide file tree
Showing 21 changed files with 1,255 additions and 141 deletions.
13 changes: 11 additions & 2 deletions .github/workflows/presubmit.yml
Original file line number Diff line number Diff line change
Expand Up @@ -36,8 +36,8 @@ jobs:
sudo apt-get install ninja-build libvulkan-dev clang-15 mesa-vulkan-drivers
- name: Build SPIRV-Tools
run: |
git clone --depth 1 --branch main https://github.com/KhronosGroup/SPIRV-Tools.git third_party/spirv-tools
git clone --depth 1 --branch main https://github.com/KhronosGroup/SPIRV-Headers.git third_party/spirv-tools/external/spirv-headers
git clone --depth 1 --branch vksp https://github.com/rjodinchr/SPIRV-Tools.git third_party/spirv-tools
git clone --depth 1 --branch vksp https://github.com/rjodinchr/SPIRV-Headers.git third_party/spirv-tools/external/spirv-headers
cmake -B third_party/spirv-tools/build -S third_party/spirv-tools/ -G Ninja \
-DCMAKE_CXX_COMPILER="$(which clang++)" \
-DCMAKE_BUILD_TYPE=Release
Expand Down Expand Up @@ -112,3 +112,12 @@ jobs:
PERFETTO_BINARY: ${{ github.workspace }}/third_party/perfetto/out/linux_clang_release/perfetto
run: |
$(pwd)/tests/test-extractor.sh
- name: Test buffers
env:
LD_LIBRARY_PATH: ${{ github.workspace }}/build_release/layer/:${{ github.workspace }}/third_party/vulkan-loader/build/loader/:$LD_LIBRARY_PATH
VKSP_RUNNER: ${{ github.workspace }}/build_release/runner/vulkan-shader-profiler-runner
PERFETTO_TRACED: ${{ github.workspace }}/third_party/perfetto/out/linux_clang_release/traced
PERFETTO_BINARY: ${{ github.workspace }}/third_party/perfetto/out/linux_clang_release/perfetto
run: |
$(pwd)/tests/test-buffers.sh
18 changes: 17 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@

It allows to visualize a vulkan application using perfetto with information about the compute shader to easily identify which shader is taking most of the application time, and what is its Vulkan SPIR-V source code.

Using the `vulkan-shader-profiler-extractor` and `vulkan-shader-profiler-runner`, it is also possible to extract a specific dispatch from the trace (using the `dispatchId` debug information from the trace), and replay it with profiled section with the runner.
Using the `vulkan-shader-profiler-extractor` and `vulkan-shader-profiler-runner`, it is also possible to extract a specific dispatch from the trace (using the `dispatchId` debug information from the trace), and replay it with the runner.

# Legal

Expand Down Expand Up @@ -79,6 +79,8 @@ To run an application with the `vulkan-kernel-profiler`, one need to ensure the
* The `Vulkan-Loader` needs to be able to find the manifest in `<vulkan-shader-profiler>/manifest/vulkan-shader-profiler.json`. This can be achieve by using the follow environment variable: `VK_ADD_LAYER_PATH=<path-to-vulkan-shader-profiler-manifest>`.
* The Layer needs to be enabled. Either directly from the application, or using the following environment variable: `VK_LOADER_LAYERS_ENABLE="VK_LAYER_SHADER_PROFILER"`.

It is also possible to extract the content of the memories of buffers and images used by a specific dispatch. It requires to first do a first run to then extract the targeted dispatch. After that a second run can be done with `VKSP_EXTRACT_BUFFERS_FROM=<trace.spvasm>` set. It will generates a `<trace.spvasm.buffers>` file that can be used later on with the `vulkan-shader-profiler-runner` to initialize the memories of the images and buffers used.

## On ChromeOS

Make sure to have emerged and deployed the `vulkan-shader-profiler`.
Expand Down Expand Up @@ -125,6 +127,18 @@ Functions used by `vulkan-shader-profiler` internally:
* `CmdWriteTimestamp`: To store the timestamp during the command buffer execution.
* `GetCalibratedTimestampsEXT`: To convert the device timestamp to the host timeline
* `GetPhysicalDeviceProperties`: To convert the number of ticks returned by `CmdWriteTimestamp` to actual time information in nano-seconds.
* The following functions are used for the extracting buffers feature:
* `CmdPipelineBarrier`
* `CmdCopyBuffer`
* `CmdCopyImage`
* `MapMemory`
* `UnmapMemory`
* `GetImageMemoryRequirements`
* `GetBufferMemoryRequirements`
* `DestroyImage`
* `DestroyBuffer`
* `FreeMemory`
* `GetPhysicalDeviceMemoryProperties`

# Extracting a dispatch from a trace

Expand Down Expand Up @@ -158,10 +172,12 @@ Required options:

Optional options:

* `-b`: path to a buffers file associated to the input (generated when tracing with `VKSP_EXTRACT_BUFFERS_FROM`).
* `-c`: disable the counters. Allow to run with no overhead introduced by the counters.
* `-e`: allow to choose the `spv_target_env` to use when using a non-binary input to convert it to binary (default: `vulkan1.3`)
* `-n`: allow to run the program multiple times
* `-m`: allow to run the program multiple times before starting to benchmark it
* `-o`: descriptor set index and binding of a buffer to dump after the execution (example: `1.2`, meaning descriptor set `1`, binding `2`).
* `-p`: allow to force the usage of the vulkan queue global priority:
* `0`:`low`
* `1`:`medium`
Expand Down
17 changes: 15 additions & 2 deletions chromeos-utils/vulkan-shader-profiler.sh
Original file line number Diff line number Diff line change
Expand Up @@ -16,5 +16,18 @@

set -x

VK_LOADER_LAYERS_ENABLE="VK_LAYER_SHADER_PROFILER" \
"$@"
LAYERS="VK_LAYER_SHADER_PROFILER"
if [[ -n "${VK_LOADER_LAYERS_ENABLE}" ]]
then
LAYERS+=",${VK_LOADER_LAYERS_ENABLE}"
fi

if [[ "$1" == "-e" ]]
then
shift
SPV_FILE="$1"
shift
VKSP_EXTRACT_BUFFERS_FROM="${SPV_FILE}" VK_LOADER_LAYERS_ENABLE="${LAYERS}" "$@"
else
VK_LOADER_LAYERS_ENABLE="${LAYERS}" "$@"
fi
132 changes: 132 additions & 0 deletions common/buffers_file.hpp
Original file line number Diff line number Diff line change
@@ -0,0 +1,132 @@
// Copyright 2024 The Vulkan Shader Profiler authors.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

#pragma once

#include <map>
#include <stdint.h>
#include <stdio.h>
#include <stdlib.h>

namespace vksp {

using buffer_map_key = std::pair<uint32_t, uint32_t>;
using buffer_map_val = std::pair<uint32_t, void *>;
using buffers_map = std::map<buffer_map_key, buffer_map_val>;

class BuffersFile {
public:
BuffersFile(uint32_t dispatchId)
: m_version(1)
, m_magic(0x766B7370) // VKSP in ASCII
, m_dispatchId(dispatchId)
{
}

bool ReadFromFile(const char *filename)
{
FILE *fd = fopen(filename, "r");
uint32_t file_header[3];
if (fread(file_header, sizeof(file_header), 1, fd) != 1) {
return false;
}
if (file_header[0] != m_magic || file_header[1] != m_version || file_header[2] != m_dispatchId) {
return false;
}
while (true) {
uint32_t buffer_header[3];
if (fread(buffer_header, sizeof(buffer_header), 1, fd) != 1) {
return false;
}
if (buffer_header[0] == UINT32_MAX || buffer_header[1] == UINT32_MAX || buffer_header[2] == UINT32_MAX) {
break;
}
buffer_map_key key = std::make_pair(buffer_header[0], buffer_header[1]);

uint32_t size = buffer_header[2];
void *data = malloc(size);
if (data == nullptr) {
return false;
}
size_t byte_read = 0;
while (byte_read != size) {
byte_read += fread(&(((char *)data)[byte_read]), sizeof(char), size - byte_read, fd);
}
buffer_map_val val = std::make_pair(size, data);
m_buffers[key] = val;
}

fclose(fd);
return true;
}

bool WriteToFile(const char *filename)
{
FILE *fd = fopen(filename, "w");
if (fd == nullptr) {
return false;
}
if (fwrite(&m_magic, sizeof(m_magic), 1, fd) != 1) {
return false;
}
if (fwrite(&m_version, sizeof(m_version), 1, fd) != 1) {
return false;
}
if (fwrite(&m_dispatchId, sizeof(m_dispatchId), 1, fd) != 1) {
return false;
}
for (auto &buffer : m_buffers) {
uint32_t set = buffer.first.first;
uint32_t binding = buffer.first.second;
uint32_t size = buffer.second.first;
void *data = buffer.second.second;
if (fwrite(&set, sizeof(set), 1, fd) != 1) {
return false;
}
if (fwrite(&binding, sizeof(binding), 1, fd) != 1) {
return false;
}
if (fwrite(&size, sizeof(size), 1, fd) != 1) {
return false;
}
uint32_t byte_written = 0;
while (byte_written != size) {
byte_written += fwrite(&(((char *)data)[byte_written]), sizeof(char), size - byte_written, fd);
}
}
uint32_t eof[3] = { UINT32_MAX, UINT32_MAX, UINT32_MAX };
if (fwrite(eof, sizeof(eof), 1, fd) != 1) {
return false;
}
fclose(fd);

return true;
}

void AddBuffer(uint32_t set, uint32_t binding, uint32_t size, void *data)
{
buffer_map_key key = std::make_pair(set, binding);
buffer_map_val val = std::make_pair(size, data);
m_buffers[key] = val;
}

buffers_map *GetBuffers() { return &m_buffers; }

private:
const uint32_t m_version;
const uint32_t m_magic;
const uint32_t m_dispatchId;
buffers_map m_buffers;
};
}
15 changes: 15 additions & 0 deletions common/common.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,20 @@
#include <stdint.h>
#include <vulkan/vulkan.h>

#define PRINT_IMPL(file, message, ...) \
do { \
fprintf(file, "[VKSP] %s: " message "\n", __func__, ##__VA_ARGS__); \
} while (0)

#define ERROR(message, ...) PRINT_IMPL(stderr, message, ##__VA_ARGS__)

#define PRINT(message, ...) \
do { \
if (gVerbose) { \
PRINT_IMPL(stdout, message, ##__VA_ARGS__); \
} \
} while (0)

namespace vksp {

struct vksp_push_constant {
Expand Down Expand Up @@ -121,6 +135,7 @@ struct vksp_configuration {
uint32_t groupCountX;
uint32_t groupCountY;
uint32_t groupCountZ;
uint32_t dispatchId;
};

struct vksp_specialization_map_entry {
Expand Down
70 changes: 69 additions & 1 deletion runner/spirv.hpp → common/spirv-extract.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,8 @@
#include "source/opt/pass.h"
#include "spirv/unified1/NonSemanticVkspReflection.h"

#include <spirv-tools/optimizer.hpp>

#define UNDEFINED_ID (UINT32_MAX)

namespace vksp {
Expand All @@ -39,7 +41,7 @@ class ExtractVkspReflectInfoPass : public spvtools::opt::Pass {
Status Process() override
{
auto module = context()->module();
uint32_t ext_inst_id = module->GetExtInstImportId("NonSemantic.VkspReflection.1");
uint32_t ext_inst_id = module->GetExtInstImportId("NonSemantic.VkspReflection.2");
int32_t descriptor_set_0_max_binding = -1;
std::map<uint32_t, uint32_t> id_to_descriptor_set;
std::map<uint32_t, uint32_t> id_to_binding;
Expand Down Expand Up @@ -205,6 +207,7 @@ class ExtractVkspReflectInfoPass : public spvtools::opt::Pass {
config_->groupCountX = inst->GetOperand(op_id++).words[0];
config_->groupCountY = inst->GetOperand(op_id++).words[0];
config_->groupCountZ = inst->GetOperand(op_id++).words[0];
config_->dispatchId = inst->GetOperand(op_id++).words[0];
break;
case NonSemanticVkspReflectionDescriptorSetBuffer: {
vksp_descriptor_set ds;
Expand Down Expand Up @@ -550,4 +553,69 @@ class ExtractVkspReflectInfoPass : public spvtools::opt::Pass {
bool disableCounters_;
};

bool extract_from_input(const char *filename, spv_target_env &spv_target_env, bool disable_counters, bool verbose,
std::vector<uint32_t> &shader, std::vector<vksp::vksp_descriptor_set> &ds,
std::vector<vksp::vksp_push_constant> &pc, std::vector<vksp::vksp_specialization_map_entry> &me,
std::vector<vksp::vksp_counter> &counters, vksp::vksp_configuration &config)
{
FILE *input = fopen(filename, "r");
fseek(input, 0, SEEK_END);
size_t input_size = ftell(input);
fseek(input, 0, SEEK_SET);
std::vector<char> input_buffer(input_size);
size_t size_read = 0;
do {
size_read += fread(&input_buffer.data()[size_read], 1, input_size - size_read, input);
} while (size_read != input_size);
fclose(input);

const uint32_t spirv_magic = 0x07230203;
spv_context context = spvContextCreate(spv_target_env);
uint32_t *binary = (uint32_t *)input_buffer.data();
size_t size = input_size / sizeof(uint32_t);
spv_binary tmp_binary;
if (*(uint32_t *)input_buffer.data() != spirv_magic) {
spv_diagnostic diagnostic;
auto status = spvTextToBinary(context, input_buffer.data(), input_size, &tmp_binary, &diagnostic);
if (status != SPV_SUCCESS) {
ERROR("Error while converting shader from text to binary: %s", diagnostic->error);
spvDiagnosticDestroy(diagnostic);
return false;
}

binary = tmp_binary->code;
size = tmp_binary->wordCount;
}

spvtools::Optimizer opt(SPV_ENV_VULKAN_1_3);
opt.RegisterPass(spvtools::Optimizer::PassToken(
std::make_unique<vksp::ExtractVkspReflectInfoPass>(&pc, &ds, &me, &counters, &config, disable_counters)));
opt.RegisterPass(spvtools::CreateStripReflectInfoPass());
spvtools::OptimizerOptions options;
options.set_run_validator(false);
if (!opt.Run(binary, size, &shader, options)) {
ERROR("Error while running 'CreateVkspReflectInfoPass' and 'CreateStripReflectInfoPass'");
return false;
}

if (verbose) {
spv_text text;
spv_diagnostic diag;
spv_result_t spv_result = spvBinaryToText(context, shader.data(), shader.size(),
SPV_BINARY_TO_TEXT_OPTION_INDENT | SPV_BINARY_TO_TEXT_OPTION_FRIENDLY_NAMES
| SPV_BINARY_TO_TEXT_OPTION_COMMENT,
&text, &diag);
if (spv_result == SPV_SUCCESS) {
PRINT("Shader:\n%s", text->str);
spvTextDestroy(text);
} else {
ERROR("Could not convert shader from binary to text: %s", diag->error);
spvDiagnosticDestroy(diag);
}
}

spvContextDestroy(context);

return true;
}
}
11 changes: 3 additions & 8 deletions extractor/extractor.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -26,19 +26,12 @@
#include <set>
#include <string>

#include "common/common.hpp"
#include "spirv.hpp"
#include "utils.hpp"

using namespace perfetto::trace_processor;
using namespace spvtools;

#define PRINT(message, ...) \
do { \
if (gVerbose) { \
PRINT_IMPL(stdout, message, ##__VA_ARGS__); \
} \
} while (0)

#define CHECK(statement, message, ...) \
do { \
if (!(statement)) { \
Expand Down Expand Up @@ -141,6 +134,8 @@ bool get_dispatch_compute_and_commandBuffer_from_dispatchId(TraceProcessor *tp,

GET_STR_VALUE(tp, compute, "debug.shader_name", config.entryPoint);

config.dispatchId = dispatchId;

assert(!it.Next());
return true;
}
Expand Down
Loading

0 comments on commit 4f1f3e7

Please sign in to comment.