Skip to content

Commit

Permalink
Initial commit
Browse files Browse the repository at this point in the history
  • Loading branch information
RatinCN committed Jul 25, 2024
1 parent 4b8c659 commit e79f8d3
Show file tree
Hide file tree
Showing 235 changed files with 8,413 additions and 121 deletions.
76 changes: 71 additions & 5 deletions .editorconfig
Original file line number Diff line number Diff line change
@@ -1,10 +1,76 @@
# Top-most EditorConfig file
# Visual Studio generated .editorconfig file with C++ settings.
root = true

# Use same style for all files
[*]
[*.{c,c++,cc,cpp,cppm,cxx,h,h++,hh,hpp,hxx,inl,ipp,ixx,tlh,tli}]

charset = utf-8-bom
indent_style = space
indent_size = 4
end_of_line = crlf
insert_final_newline = false
trim_trailing_whitespace = true
insert_final_newline = true
max_line_length = 120

# Visual C++ Code Style settings

cpp_generate_documentation_comments = xml

# Visual C++ Formatting settings

cpp_indent_braces = false
cpp_indent_multi_line_relative_to = innermost_parenthesis
cpp_indent_within_parentheses = align_to_parenthesis
cpp_indent_preserve_within_parentheses = true
cpp_indent_case_contents = true
cpp_indent_case_labels = true
cpp_indent_case_contents_when_block = false
cpp_indent_lambda_braces_when_parameter = true
cpp_indent_goto_labels = leftmost_column
cpp_indent_preprocessor = leftmost_column
cpp_indent_access_specifiers = false
cpp_indent_namespace_contents = true
cpp_indent_preserve_comments = true
cpp_new_line_before_open_brace_namespace = new_line
cpp_new_line_before_open_brace_type = new_line
cpp_new_line_before_open_brace_function = new_line
cpp_new_line_before_open_brace_block = new_line
cpp_new_line_before_open_brace_lambda = new_line
cpp_new_line_scope_braces_on_separate_lines = true
cpp_new_line_close_brace_same_line_empty_type = true
cpp_new_line_close_brace_same_line_empty_function = true
cpp_new_line_before_catch = false
cpp_new_line_before_else = false
cpp_new_line_before_while_in_do_while = false
cpp_space_before_function_open_parenthesis = remove
cpp_space_within_parameter_list_parentheses = false
cpp_space_between_empty_parameter_list_parentheses = false
cpp_space_after_keywords_in_control_flow_statements = true
cpp_space_within_control_flow_statement_parentheses = false
cpp_space_before_lambda_open_parenthesis = false
cpp_space_within_cast_parentheses = false
cpp_space_after_cast_close_parenthesis = false
cpp_space_within_expression_parentheses = false
cpp_space_before_block_open_brace = true
cpp_space_between_empty_braces = false
cpp_space_before_initializer_list_open_brace = false
cpp_space_within_initializer_list_braces = true
cpp_space_preserve_in_initializer_list = true
cpp_space_before_open_square_bracket = false
cpp_space_within_square_brackets = false
cpp_space_before_empty_square_brackets = false
cpp_space_between_empty_square_brackets = false
cpp_space_group_square_brackets = true
cpp_space_within_lambda_brackets = false
cpp_space_between_empty_lambda_brackets = false
cpp_space_before_comma = false
cpp_space_after_comma = true
cpp_space_remove_around_member_operators = true
cpp_space_before_inheritance_colon = true
cpp_space_before_constructor_colon = true
cpp_space_remove_before_semicolon = true
cpp_space_after_semicolon = true
cpp_space_remove_around_unary_operator = true
cpp_space_around_binary_operator = insert
cpp_space_around_assignment_operator = insert
cpp_space_pointer_reference_alignment = ignore
cpp_space_around_ternary_operator = insert
cpp_wrap_preserve_blocks = never
38 changes: 38 additions & 0 deletions .github/workflows/msbuild.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
name: MSBuild

on:
push:
branches: [ "main" ]
pull_request:
branches: [ "main" ]

env:
SOLUTION_FILE_PATH: .\Source\KNSoft.SlimDetours.sln

permissions:
contents: read

jobs:
build:
strategy:
matrix:
platform: [x64, x86, ARM64]
config: [Debug, Release]
fail-fast: false
runs-on: windows-latest
steps:
- name: Source checkout
uses: actions/checkout@main
with:
submodules: recursive
- name: Prepare MSBuild
uses: microsoft/setup-msbuild@main
- name: Build
working-directory: ${{env.GITHUB_WORKSPACE}}
run: msbuild ${{env.SOLUTION_FILE_PATH}} /restore /m /p:Configuration=${{matrix.config}} /p:Platform=${{matrix.platform}} /p:RestorePackagesConfig=true
- name: Run Demo (x64, x86)
if: ${{ matrix.platform == 'x64' || matrix.platform == 'x86' }}
working-directory: ${{env.GITHUB_WORKSPACE}}
run: |
.\Source\OutDir\${{matrix.platform}}\${{matrix.config}}\Demo.exe -Run DeadLock -Engine=SlimDetours
.\Source\OutDir\${{matrix.platform}}\${{matrix.config}}\Demo.exe -Run DelayHook
45 changes: 6 additions & 39 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,41 +1,8 @@
# C extensions
*.so

# Unit test / coverage reports
.coverage
.tox
nosetests.xml

# Translations
*.mo

# Mr Developer
.mr.developer.cfg
.project
.pydevproject
.vs
*.user

# vim
*~
*.swp
/packages
/OutDir
IntDir

# Visual Studio build
*.ipch
.vs/
output/
include/
*.exp
*.pdb
*.lib
*.dll
*.exe
obj.*
*.ipdb
*.iobj
*.tlog
*.log
*.obj
*.user
*.recipe
/bin.*
*.vcxproj.FileListAbsolute.txt
*.vcxprojAssemblyReference.cache
/*.nupkg
3 changes: 3 additions & 0 deletions .gitmodules
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
[submodule "Source/Detours"]
path = Source/Detours
url = https://github.com/microsoft/Detours.git
Original file line number Diff line number Diff line change
@@ -0,0 +1,111 @@
| **English (en-US)** | [简体中文 (zh-CN)](./README.zh-CN.md) |
| --- | --- |


<br>

# Avoid Deadlocking on The Heap When Updating Threads

## Why does Detours may deadlock when updating threads?

The original [Detours](https://github.com/microsoft/Detours) uses the CRT heap (via `new/delete`), and if another thread that also uses this heap and is holding the heap lock is suspended while updating threads, then [Detours](https://github.com/microsoft/Detours) will deadlock when it accesses the heap.

[Raymond Chen](https://devblogs.microsoft.com/oldnewthing/author/oldnewthing) discussed the problem that CRT heap deadlocks when suspending threads in [blog "The Old New Thing"](https://devblogs.microsoft.com/oldnewthing/) article ["Are there alternatives to _lock and _unlock in Visual Studio 2015?"](https://devblogs.microsoft.com/oldnewthing/20170125-00/?p=95255) is the same scenario, and also mentions [Detours](https://github.com/microsoft/Detours), here quotes the original text and will not go into details:
> Furthermore, you would be best served to take the heap lock (Heap­Lock) before suspending the thread, because the Detours library will allocate memory during thread suspension.
## Demo of Detours Deadlock

[SlimDetours](https://github.com/KNSoft/KNSoft.SlimDetours) provides [Demo: DeadLock](../../../Source/Demo/DeadLock.c) to demonstrate the occurrence of a deadlock in [Detours](https://github.com/microsoft/Detours) and the resolution in [SlimDetours](https://github.com/KNSoft/KNSoft.SlimDetours).

One of the threads (`HeapUserThread`) keeps calling `malloc/free` (equivalent to `new/delete`):
```C
while (!g_bStop)
{
p = malloc(4);
if (p != NULL)
{
free(p);
}
}
```
Another thread (`SetHookThread`) uses [Detours](https://github.com/microsoft/Detours) or [SlimDetours](https://github.com/KNSoft/KNSoft.SlimDetours) constantly hook and unhook:
```C
while (!g_bStop)
{
hr = HookTransactionBegin(g_eEngineType);
if (FAILED(hr))
{
break;
}
if (g_eEngineType == EngineMicrosoftDetours)
{
hr = HRESULT_FROM_WIN32(DetourUpdateThread((HANDLE)lpThreadParameter));
if (FAILED(hr))
{
break;
}
}
hr = HookAttach(g_eEngineType, EnableHook, (PVOID*)&g_pfnEqualRect, Hooked_EqualRect);
if (FAILED(hr))
{
HookTransactionAbort(g_eEngineType);
break;
}
hr = HookTransactionCommit(g_eEngineType);
if (FAILED(hr))
{
break;
}
EnableHook = !EnableHook;
}
```
> [!NOTE]
> [SlimDetours](https://github.com/KNSoft/KNSoft.SlimDetours) updates threads automatically (see [🔗 TechWiki: Update Threads Automatically When Applying Inline Hooks](https://github.com/KNSoft/KNSoft.SlimDetours/blob/main/Docs/TechWiki/Update%20Threads%20Automatically%20When%20Applying%20Inline%20Hooks/README.md)), so there is no such function as [`DetourUpdateThread`](https://github.com/microsoft/Detours/wiki/DetourUpdateThread).
Execute these 2 threads at the same time for 10 seconds, then send a stop signal (`g_bStop = TRUE;`) and wait 10 seconds again, if it times out, there is a high probability that a deadlock is occurred, a breakpoint will be triggered, and you can observe the call stack of these 2 threads in the debugger for confirmation. For example, if you specify to run this demo with [Detours](https://github.com/microsoft/Detours) `"Demo.exe -Run DeadLock -Engine=MSDetours"`, the following call stack will see a heap deadlock:
```C
Worker Thread Demo.exe!HeapUserThread Demo.exe!heap_alloc_dbg_internal
[External Code]
Demo.exe!heap_alloc_dbg_internal(const unsigned __int64 size, const int block_use, const char * const file_name, const int line_number) Line 359
Demo.exe!heap_alloc_dbg(const unsigned __int64 size, const int block_use, const char * const file_name, const int line_number) Line 450
Demo.exe!_malloc_dbg(unsigned __int64 size, int block_use, const char * file_name, int line_number) Line 496
Demo.exe!malloc(unsigned __int64 size) Line 27
Demo.exe!HeapUserThread(void * lpThreadParameter) Line 29
[External Code]

Worker Thread Demo.exe!SetHookThread Demo.exe!__acrt_lock
[External Code]
Demo.exe!__acrt_lock(__acrt_lock_id _Lock) Line 55
Demo.exe!heap_alloc_dbg_internal(const unsigned __int64 size, const int block_use, const char * const file_name, const int line_number) Line 309
Demo.exe!heap_alloc_dbg(const unsigned __int64 size, const int block_use, const char * const file_name, const int line_number) Line 450
Demo.exe!_malloc_dbg(unsigned __int64 size, int block_use, const char * file_name, int line_number) Line 496
Demo.exe!malloc(unsigned __int64 size) Line 27
[External Code]
Demo.exe!DetourDetach(void * * ppPointer, void * pDetour) Line 2392
Demo.exe!HookAttach(_DEMO_ENGINE_TYPE EngineType, int Enable, void * * ppPointer, void * pDetour) Line 140
Demo.exe!SetHookThread(void * lpThreadParameter) Line 65
[External Code]
```
Use [SlimDetours](https://github.com/KNSoft/KNSoft.SlimDetours) run this demo `"Demo.exe -Run DeadLock -Engine=SlimDetours"` will pass successfully.

## How did mhook and SlimDetours avoid this problem?

[mhook](https://github.com/martona/mhook) is a well-known Windows API hooking library like [Detours](https://github.com/microsoft/Detours), it uses [`Virtual­Alloc`](https://learn.microsoft.com/en-us/windows/win32/api/memoryapi/nf-memoryapi-virtualalloc) to allocate memory pages instead of [`Heap­Alloc`](https://learn.microsoft.com/en-us/windows/win32/api/heapapi/nf-heapapi-heapalloc) to allocate heap memory, which is a solution mentioned at the end of the above article.

[SlimDetours](https://github.com/KNSoft/KNSoft.SlimDetours) created a new private heap for internal use, which avoids this problem and saves memory usage:
```C
_detour_memory_heap = RtlCreateHeap(HEAP_NO_SERIALIZE | HEAP_GROWABLE, NULL, 0, 0, NULL, NULL);
```
> [!NOTE]
> [Detours](https://github.com/microsoft/Detours) already has a transaction mechanism, and [SlimDetours](https://github.com/KNSoft/KNSoft.SlimDetours)' new feature "[Delay Hook](../Implement%20Delay%20Hook/README.md)" also uses SRW locks, so this heap does not need serialized access.
<br>
<hr>

This work is licensed under [Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International (CC BY-NC-SA 4.0)](http://creativecommons.org/licenses/by-nc-sa/4.0/).
<br>
**[Ratin](https://github.com/RatinCN) &lt;[<[email protected]>](mailto:[email protected])&gt;**
*China national certified senior system architect*
*[ReactOS](https://github.com/reactos/reactos) contributor*
Original file line number Diff line number Diff line change
@@ -0,0 +1,111 @@
| [English (en-US)](./README.md) | **简体中文 (zh-CN)** |
| --- | --- |

<br>

# 更新线程时避免堆死锁

## 为什么Detours更新线程时可能死锁?

原版[Detours](https://github.com/microsoft/Detours)使用了CRT堆(通过`new/delete`),更新线程时如果挂起了另一个也使用此堆且正持有堆锁的线程,[Detours](https://github.com/microsoft/Detours)再访问此堆就会发生死锁。

[Raymond Chen](https://devblogs.microsoft.com/oldnewthing/author/oldnewthing)[博客“The Old New Thing”](https://devblogs.microsoft.com/oldnewthing/)的文章[《Are there alternatives to _lock and _unlock in Visual Studio 2015?》](https://devblogs.microsoft.com/oldnewthing/20170125-00/?p=95255)中详细讨论的挂起线程时出现CRT堆死锁问题正是同一个场景,也提到了[Detours](https://github.com/microsoft/Detours),这里引用其原文不再赘述:
> Furthermore, you would be best served to take the heap lock (Heap­Lock) before suspending the thread, because the Detours library will allocate memory during thread suspension.
> 此外,最好在挂起线程前占有堆锁(Heap­Lock),因为Detours库将在线程挂起期间分配内存。
## Detours死锁的演示

[SlimDetours](https://github.com/KNSoft/KNSoft.SlimDetours)提供了[示例:DeadLock](../../../Source/Demo/DeadLock.c)演示[Detours](https://github.com/microsoft/Detours)死锁的发生与在[SlimDetours](https://github.com/KNSoft/KNSoft.SlimDetours)中得到解决。

其中一个线程(`HeapUserThread`)不断调用`malloc/free`(等效于`new/delete`):
```C
while (!g_bStop)
{
p = malloc(4);
if (p != NULL)
{
free(p);
}
}
```
另一个线程(`SetHookThread`)不断使用[Detours](https://github.com/microsoft/Detours)或[SlimDetours](https://github.com/KNSoft/KNSoft.SlimDetours)挂钩和脱钩:
```C
while (!g_bStop)
{
hr = HookTransactionBegin(g_eEngineType);
if (FAILED(hr))
{
break;
}
if (g_eEngineType == EngineMicrosoftDetours)
{
hr = HRESULT_FROM_WIN32(DetourUpdateThread((HANDLE)lpThreadParameter));
if (FAILED(hr))
{
break;
}
}
hr = HookAttach(g_eEngineType, EnableHook, (PVOID*)&g_pfnEqualRect, Hooked_EqualRect);
if (FAILED(hr))
{
HookTransactionAbort(g_eEngineType);
break;
}
hr = HookTransactionCommit(g_eEngineType);
if (FAILED(hr))
{
break;
}
EnableHook = !EnableHook;
}
```
> [!NOTE]
> [SlimDetours](https://github.com/KNSoft/KNSoft.SlimDetours)会自动更新线程(参考[🔗 技术Wiki:应用内联钩子时自动更新线程](https://github.com/KNSoft/KNSoft.SlimDetours/blob/main/Docs/TechWiki/Update%20Threads%20Automatically%20When%20Applying%20Inline%20Hooks/README.zh-CN.md)),所以不存在[`DetourUpdateThread`](https://github.com/microsoft/Detours/wiki/DetourUpdateThread)这样的函数。
同时执行这2个线程10秒,然后发送停止信号(`g_bStop = TRUE;`)后再次等待10秒,如果超时则大概率发生死锁,将触发断点,可以在调试器中观察这2个线程的调用栈进行确认。例如指定使用[Detours](https://github.com/microsoft/Detours)运行此示例`"Demo.exe -Run DeadLock -Engine=MSDetours"`,以下调用栈可见堆死锁:
```C
Worker Thread Demo.exe!HeapUserThread Demo.exe!heap_alloc_dbg_internal
[External Code]
Demo.exe!heap_alloc_dbg_internal(const unsigned __int64 size, const int block_use, const char * const file_name, const int line_number) Line 359
Demo.exe!heap_alloc_dbg(const unsigned __int64 size, const int block_use, const char * const file_name, const int line_number) Line 450
Demo.exe!_malloc_dbg(unsigned __int64 size, int block_use, const char * file_name, int line_number) Line 496
Demo.exe!malloc(unsigned __int64 size) Line 27
Demo.exe!HeapUserThread(void * lpThreadParameter) Line 29
[External Code]

Worker Thread Demo.exe!SetHookThread Demo.exe!__acrt_lock
[External Code]
Demo.exe!__acrt_lock(__acrt_lock_id _Lock) Line 55
Demo.exe!heap_alloc_dbg_internal(const unsigned __int64 size, const int block_use, const char * const file_name, const int line_number) Line 309
Demo.exe!heap_alloc_dbg(const unsigned __int64 size, const int block_use, const char * const file_name, const int line_number) Line 450
Demo.exe!_malloc_dbg(unsigned __int64 size, int block_use, const char * file_name, int line_number) Line 496
Demo.exe!malloc(unsigned __int64 size) Line 27
[External Code]
Demo.exe!DetourDetach(void * * ppPointer, void * pDetour) Line 2392
Demo.exe!HookAttach(_DEMO_ENGINE_TYPE EngineType, int Enable, void * * ppPointer, void * pDetour) Line 140
Demo.exe!SetHookThread(void * lpThreadParameter) Line 65
[External Code]
```
使用[SlimDetours](https://github.com/KNSoft/KNSoft.SlimDetours)运行此示例`"Demo.exe -Run DeadLock -Engine=SlimDetours"`则能顺利通过。

## mhook与SlimDetours如何避免这个问题?

[mhook](https://github.com/martona/mhook)[Detours](https://github.com/microsoft/Detours)一样也是一个熟知的Windows API挂钩库,它使用[`Virtual­Alloc`](https://learn.microsoft.com/en-us/windows/win32/api/memoryapi/nf-memoryapi-virtualalloc)分配内存页代替[`Heap­Alloc`](https://learn.microsoft.com/en-us/windows/win32/api/heapapi/nf-heapapi-heapalloc)分配堆内存,是上文末尾提到的一个解决方案。

[SlimDetours](https://github.com/KNSoft/KNSoft.SlimDetours)新创建了一个私有堆供内部使用,避免此问题的同时也节约了内存使用:
```C
_detour_memory_heap = RtlCreateHeap(HEAP_NO_SERIALIZE | HEAP_GROWABLE, NULL, 0, 0, NULL, NULL);
```
> [!NOTE]
> [Detours](https://github.com/microsoft/Detours)已有事务机制,[SlimDetours](https://github.com/KNSoft/KNSoft.SlimDetours)新添功能“[延迟挂钩](../Implement%20Delay%20Hook/README.zh-CN.md)”也用了[SRW锁](https://learn.microsoft.com/en-us/windows/win32/sync/slim-reader-writer--srw--locks),所以此堆无需序列化访问。
<br>
<hr>

本作品采用 [知识共享署名-非商业性使用-相同方式共享 4.0 国际许可协议 (CC BY-NC-SA 4.0)](http://creativecommons.org/licenses/by-nc-sa/4.0/) 进行许可。
<br>
**[Ratin](https://github.com/RatinCN) &lt;[<[email protected]>](mailto:[email protected])&gt;**
*中国国家认证系统架构设计师*
*[ReactOS](https://github.com/reactos/reactos)贡献者*
Loading

0 comments on commit e79f8d3

Please sign in to comment.