Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add test with ASAN enabled. #2313

Merged
merged 29 commits into from
Jun 10, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
d257449
Add test with ASAN enabled.
Rot127 Apr 7, 2024
f780776
Fix leaks in cstool and cs.c
Rot127 Apr 17, 2024
5a46528
Add work around so ASAN binaries don't DEADSIGNAL due to too many ran…
Rot127 Apr 17, 2024
9d58921
Add ASAN build arguments to cstest
Rot127 Apr 17, 2024
ec8fdcf
Fix leaks in cstest
Rot127 Apr 17, 2024
5331346
Use cstest binary build by the main build.
Rot127 Apr 17, 2024
22402ac
Add clonging step for cmocka when cstest is build
Rot127 Apr 17, 2024
05e8cec
Skip Python tests for ASAN
Rot127 Apr 17, 2024
6a67aa1
Remove make build from CI
Rot127 Apr 17, 2024
07879f2
Fix leaks in cstest.
Rot127 Apr 20, 2024
d372d68
Fix cmocka external project to stable branch.
Rot127 Apr 20, 2024
2d0f23b
Revert "Fix leaks in cstest."
Rot127 Apr 20, 2024
a77c28a
Fix memleaks in cstest
Rot127 Apr 22, 2024
b1115bc
Document adding of ASAN job to release guide
Rot127 Apr 22, 2024
f61c02c
Add CAPSTONE_BUILD_CSTEST to build docs
Rot127 Apr 22, 2024
84e4d79
Fix double free
Rot127 Apr 22, 2024
b20d1f0
Add more detail tests to CI and fix them
Rot127 Apr 22, 2024
b1c6e91
Initialize variables
Rot127 May 10, 2024
c677cd2
Fix typo
Rot127 May 10, 2024
6929e37
Update cstest build docs
Rot127 May 10, 2024
3738810
Revert "Remove make build from CI"
Rot127 May 26, 2024
b61a8ed
Make cstest only run for cmake builds.
Rot127 May 26, 2024
1bacb5b
Add cstest job for make build.
Rot127 May 26, 2024
71e1469
Add CAPSTONE_DIET build test.
Rot127 May 26, 2024
f253e07
Compile the compatibility header test with ASAN if enabled.
Rot127 Jun 1, 2024
940082b
Fix DIET build by excluding not used code.
Rot127 Jun 1, 2024
1a86e8e
Missing "
Rot127 Jun 1, 2024
b50c8df
Build static library with ASAN and DIET if enabled.
Rot127 Jun 1, 2024
d286aed
Revert "Add CAPSTONE_DIET build test."
Rot127 Jun 1, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
53 changes: 47 additions & 6 deletions .github/workflows/CITest.yml
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,7 @@ jobs:
python-arch: x64,
python-version: '3.6',
build-system: 'cmake',
enable-asan: 'OFF'
}
- {
name: 'ubuntu-22.04 x64 python3.9 make',
Expand All @@ -44,14 +45,16 @@ jobs:
python-arch: x64,
python-version: '3.9',
build-system: 'make',
}
enable-asan: 'OFF'
}
- {
name: 'ubuntu-22.04 x64 python3.9 cmake',
os: ubuntu-22.04,
arch: x64,
python-arch: x64,
python-version: '3.9',
build-system: 'cmake',
enable-asan: 'OFF'
}
- {
name: 'ubuntu-22.04 x64 python3.11 cmake',
Expand All @@ -60,6 +63,16 @@ jobs:
python-arch: x64,
python-version: '3.11',
build-system: 'cmake',
enable-asan: 'OFF'
}
- {
name: 'ubuntu-22.04 x64 python3.11 ASAN',
os: ubuntu-latest,
arch: x64,
python-arch: x64,
python-version: '3.11',
build-system: 'cmake',
enable-asan: 'ON'
}

steps:
Expand Down Expand Up @@ -88,35 +101,61 @@ jobs:

- name: cmake
if: startsWith(matrix.config.build-system, 'cmake')
env:
asan: ${{ matrix.config.enable-asan }}
run: |
mkdir build && cd build
# build static library
cmake -DCAPSTONE_INSTALL=1 -DCMAKE_INSTALL_PREFIX=/usr ..
cmake -DCAPSTONE_INSTALL=1 -DCMAKE_INSTALL_PREFIX=/usr -DENABLE_ASAN=${asan} -DCAPSTONE_BUILD_DIET=${diet_build} ..
cmake --build . --config Release
# build shared library
cmake -DCAPSTONE_INSTALL=1 -DBUILD_SHARED_LIBS=1 -DCMAKE_INSTALL_PREFIX=/usr ..
cmake -DCAPSTONE_INSTALL=1 -DBUILD_SHARED_LIBS=1 -DCMAKE_INSTALL_PREFIX=/usr -DCAPSTONE_BUILD_CSTEST=ON -DENABLE_ASAN=${asan} ..
sudo cmake --build . --config Release --target install
cp libcapstone.* ../
cp libcapstone.* ../tests/
cp test_* ../tests/

- name: Lower number of KASL randomized address bits
run: |
# Work-around ASAN bug https://github.com/google/sanitizers/issues/1716
sudo sysctl vm.mmap_rnd_bits=28

- name: "Compatibility header test build"
if: matrix.config.diet-build == 'OFF'
env:
asan: ${{ matrix.config.enable-asan }}
run: |
cd "$(git rev-parse --show-toplevel)/suite/auto-sync/c_tests/"
clang -lcapstone src/test_arm64_compatibility_header.c -o test_arm64_compatibility_header
if [ "$asan" = "ON" ]; then
clang -lcapstone -fsanitize=address src/test_arm64_compatibility_header.c -o test_arm64_compatibility_header
else
clang -lcapstone src/test_arm64_compatibility_header.c -o test_arm64_compatibility_header
fi
./test_arm64_compatibility_header
cd "$(git rev-parse --show-toplevel)"

- name: cstool - reaches disassembler engine
run: |
sh suite/run_invalid_cstool.sh

- name: cstest
- name: cstest (cmake)
if: startsWith(matrix.config.build-system, 'cmake')
run: |
python suite/cstest/cstest_report.py -D -d suite/MC
python suite/cstest/cstest_report.py -D -f suite/cstest/issues.cs
python suite/cstest/cstest_report.py -D -f tests/cs_details/issue.cs

- name: cstest (make)
if: startsWith(matrix.config.build-system, 'make')
run: |
cd suite/cstest && ./build_cstest.sh
python cstest_report.py -D -t build/cstest -d ../MC
python cstest_report.py -D -t build/cstest -f issues.cs; cd ..
python cstest_report.py -D -t build/cstest -f issues.cs
python cstest_report.py -D -t build/cstest -f ../../tests/cs_details/issue.cs
cd ../../

- name: verify python binding
if: matrix.config.enable-asan == 'OFF'
run: |
mkdir -p bindings/python/capstone/lib && cp libcapstone.so.5.* bindings/python/capstone/lib/libcapstone.so
cd bindings/python
Expand All @@ -136,6 +175,7 @@ jobs:
make check

- name: run python binding test
if: matrix.config.enable-asan == 'OFF'
run: |
cp libcapstone.* bindings/python/prebuilt
cd bindings/python
Expand All @@ -144,6 +184,7 @@ jobs:
BUILD_TESTS=no make tests

- name: run cython binding test
if: matrix.config.enable-asan == 'OFF'
run: |
pip install cython
cd bindings/python
Expand Down
27 changes: 23 additions & 4 deletions CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -68,6 +68,13 @@ option(CAPSTONE_USE_ARCH_REGISTRATION "Use explicit architecture registration" O
option(CAPSTONE_ARCHITECTURE_DEFAULT "Whether architectures are enabled by default" ON)
option(CAPSTONE_DEBUG "Whether to enable extra debug assertions" OFF)
option(CAPSTONE_INSTALL "Generate install target" ${PROJECT_IS_TOP_LEVEL})
option(ENABLE_ASAN "Enable address sanitizer" OFF)

if (ENABLE_ASAN)
add_definitions(-DASAN_ENABLED)
add_compile_options(-fsanitize=address)
add_link_options(-fsanitize=address)
endif()

# If building for OSX it's best to allow CMake to handle building both architectures
if(APPLE AND NOT CAPSTONE_BUILD_MACOS_THIN)
Expand Down Expand Up @@ -855,16 +862,28 @@ if(CAPSTONE_BUILD_CSTOOL)
endif()

if(CAPSTONE_BUILD_CSTEST)
find_package(PkgConfig REQUIRED)
pkg_check_modules(CMOCKA REQUIRED IMPORTED_TARGET cmocka)
include(ExternalProject)
ExternalProject_Add(cmocka_ext
PREFIX extern
GIT_REPOSITORY "https://git.cryptomilk.org/projects/cmocka.git"
GIT_TAG "origin/stable-1.1"
CONFIGURE_COMMAND cmake -DBUILD_SHARED_LIBS=OFF ../cmocka_ext/
BUILD_COMMAND cmake --build . --config Release
INSTALL_COMMAND ""
)
set(CMOCKA_INCLUDE_DIR ${CMAKE_CURRENT_BINARY_DIR}/extern/src/cmocka_ext/include)
set(CMOCKA_LIB_DIR ${CMAKE_CURRENT_BINARY_DIR}/extern/src/cmocka_ext-build/src/)
add_library(cmocka STATIC IMPORTED)
set_target_properties(cmocka PROPERTIES IMPORTED_LOCATION ${CMOCKA_LIB_DIR}/libcmocka.a)

file(GLOB CSTEST_SRC suite/cstest/src/*.c)
add_executable(cstest ${CSTEST_SRC})
target_link_libraries(cstest PUBLIC capstone PkgConfig::CMOCKA)
add_dependencies(cstest cmocka_ext)
target_link_libraries(cstest PUBLIC capstone cmocka)
target_include_directories(cstest PRIVATE
$<BUILD_INTERFACE:${PROJECT_SOURCE_DIR}/include>
${PROJECT_SOURCE_DIR}/suite/cstest/include
${CMOCKA_INCLUDE_DIRS}
${CMOCKA_INCLUDE_DIR}
)

if(CAPSTONE_INSTALL)
Expand Down
2 changes: 2 additions & 0 deletions COMPILE_CMAKE.TXT
Original file line number Diff line number Diff line change
Expand Up @@ -56,6 +56,8 @@ Get CMake for free from http://www.cmake.org.
- CAPSTONE_X86_REDUCE: change this to ON to make X86 binary smaller.
- CAPSTONE_X86_ATT_DISABLE: change this to ON to disable AT&T syntax on x86.
- CAPSTONE_DEBUG: change this to ON to enable extra debug assertions.
- CAPSTONE_BUILD_CSTEST: Build `cstest` in `suite/cstest/`
- ENABLE_ASAN: Compiles Capstone with the address sanitizer.

By default, Capstone use system dynamic memory management, and both DIET and X86_REDUCE
modes are disabled. To use your own memory allocations, turn ON both DIET &
Expand Down
12 changes: 11 additions & 1 deletion Mapping.h
Original file line number Diff line number Diff line change
Expand Up @@ -74,17 +74,27 @@ const cs_ac_type mapping_get_op_access(MCInst *MI, unsigned OpNum,
/// Macro for easier access of operand types from the map.
/// Assumes the istruction operands map is called "insn_operands"
/// Only usable by `auto-sync` archs!
#ifndef CAPSTONE_DIET
#define map_get_op_type(MI, OpNum) \
mapping_get_op_type(MI, OpNum, (const map_insn_ops *)insn_operands, \
sizeof(insn_operands) / sizeof(insn_operands[0]))
#else
#define map_get_op_type(MI, OpNum) \
CS_OP_INVALID
#endif

/// Macro for easier access of operand access flags from the map.
/// Assumes the istruction operands map is called "insn_operands"
/// Only usable by `auto-sync` archs!
#ifndef CAPSTONE_DIET
#define map_get_op_access(MI, OpNum) \
mapping_get_op_access(MI, OpNum, (const map_insn_ops *)insn_operands, \
sizeof(insn_operands) / \
sizeof(insn_operands[0]))
#else
#define map_get_op_access(MI, OpNum) \
CS_AC_INVALID
#endif

///< Map for ids to their string
typedef struct name_map {
Expand Down Expand Up @@ -212,4 +222,4 @@ bool map_use_alias_details(const MCInst *MI);

void map_set_alias_id(MCInst *MI, const SStream *O, const name_map *alias_mnem_id_map, int map_size);

#endif // CS_MAPPING_H
#endif // CS_MAPPING_H
2 changes: 1 addition & 1 deletion arch/ARM/ARMMapping.c
Original file line number Diff line number Diff line change
Expand Up @@ -762,11 +762,11 @@ void ARM_init_mri(MCRegisterInfo *MRI)
ARR_SIZE(ARMSubRegIdxLists), 0);
}

#ifndef CAPSTONE_DIET
static const map_insn_ops insn_operands[] = {
#include "ARMGenCSMappingInsnOp.inc"
};

#ifndef CAPSTONE_DIET
void ARM_reg_access(const cs_insn *insn, cs_regs regs_read,
uint8_t *regs_read_count, cs_regs regs_write,
uint8_t *regs_write_count)
Expand Down
2 changes: 2 additions & 0 deletions arch/HPPA/HPPAInstPrinter.c
Original file line number Diff line number Diff line change
Expand Up @@ -760,6 +760,7 @@ static void add_groups(MCInst *MI)
}
}

#ifndef CAPSTONE_DIET
static void update_regs_access(MCInst *MI, unsigned int opcode)
{
if (opcode == HPPA_INS_INVALID)
Expand Down Expand Up @@ -803,6 +804,7 @@ static void update_regs_access(MCInst *MI, unsigned int opcode)
break;
}
}
#endif

void HPPA_printInst(MCInst *MI, SStream *O, void *Info)
{
Expand Down
4 changes: 3 additions & 1 deletion arch/HPPA/HPPAMapping.c
Original file line number Diff line number Diff line change
Expand Up @@ -300,6 +300,7 @@ const char *HPPA_insn_name(csh handle, unsigned int id)
#endif
}

#ifndef CAPSTONE_DIET
/* Integer register names, indexed by the numbers which appear in the
opcodes. */
static const char *const reg_names[] = {
Expand Down Expand Up @@ -341,6 +342,7 @@ static const char *const sp_fp_reg[] = {
"fr16R", "fr17R", "fr18R", "fr19R", "fr20R", "fr21R", "fr22R", "fr23R",
"fr24R", "fr25R", "fr26R", "fr27R", "fr28R", "fr29R", "fr30R", "fr31R"
};
#endif

const char *HPPA_reg_name(csh handle, unsigned int reg)
{
Expand Down Expand Up @@ -437,4 +439,4 @@ void HPPA_reg_access(const cs_insn *insn, cs_regs regs_read,
sort_and_uniq(regs_write, write_count, regs_write_count);
}

#endif
#endif
4 changes: 4 additions & 0 deletions arch/PowerPC/PPCMapping.c
Original file line number Diff line number Diff line change
Expand Up @@ -225,8 +225,10 @@ void PPC_printer(MCInst *MI, SStream *O, void * /* MCRegisterInfo* */ info)
MI->fillDetailOps = detail_is_set(MI);
MI->flat_insn->usesAliasDetails = map_use_alias_details(MI);
PPC_LLVM_printInst(MI, MI->address, "", O);
#ifndef CAPSTONE_DIET
map_set_alias_id(MI, O, insn_alias_mnem_map,
ARR_SIZE(insn_alias_mnem_map));
#endif
}

bool PPC_getInstruction(csh handle, const uint8_t *bytes, size_t bytes_len,
Expand Down Expand Up @@ -264,9 +266,11 @@ bool PPC_getFeatureBits(unsigned int mode, unsigned int feature)
return true;
}

#ifndef CAPSTONE_DIET
static const map_insn_ops insn_operands[] = {
#include "PPCGenCSMappingInsnOp.inc"
};
#endif

/// @brief Handles memory operands.
/// @param MI The MCInst.
Expand Down
7 changes: 4 additions & 3 deletions cs.c
Original file line number Diff line number Diff line change
Expand Up @@ -697,7 +697,7 @@ CAPSTONE_EXPORT
cs_err CAPSTONE_API cs_open(cs_arch arch, cs_mode mode, csh *handle)
{
cs_err err;
struct cs_struct *ud;
struct cs_struct *ud = NULL;
if (!cs_mem_malloc || !cs_mem_calloc || !cs_mem_realloc || !cs_mem_free || !cs_vsnprintf)
// Error: before cs_open(), dynamic memory management must be initialized
// with cs_option(CS_OPT_MEM)
Expand Down Expand Up @@ -736,6 +736,7 @@ cs_err CAPSTONE_API cs_open(cs_arch arch, cs_mode mode, csh *handle)

return CS_ERR_OK;
} else {
cs_mem_free(ud);
*handle = 0;
return CS_ERR_ARCH;
}
Expand All @@ -744,8 +745,8 @@ cs_err CAPSTONE_API cs_open(cs_arch arch, cs_mode mode, csh *handle)
CAPSTONE_EXPORT
cs_err CAPSTONE_API cs_close(csh *handle)
{
struct cs_struct *ud;
struct insn_mnem *next, *tmp;
struct cs_struct *ud = NULL;
struct insn_mnem *next = NULL, *tmp = NULL;

if (*handle == 0)
// invalid handle
Expand Down
4 changes: 3 additions & 1 deletion cstool/cstool.c
Original file line number Diff line number Diff line change
Expand Up @@ -678,13 +678,15 @@ int main(int argc, char **argv)
}

cs_free(insn, count);
free(assembly);
} else {
printf("ERROR: invalid assembly code\n");
cs_close(&handle);
free(assembly);
return(-4);
}

cs_close(&handle);
free(assembly);

return 0;
}
2 changes: 2 additions & 0 deletions cstool/cstool_arm.c
Original file line number Diff line number Diff line change
Expand Up @@ -58,6 +58,8 @@ void print_insn_detail_arm(csh handle, cs_insn *ins)
printf("\t\t\toperands[%u].mem.scale: %d\n", i, op->mem.scale);
if (op->mem.disp != 0)
printf("\t\t\toperands[%u].mem.disp: 0x%x\n", i, op->mem.disp);
if (op->mem.align != 0)
printf("\t\t\toperands[%u].mem.align: 0x%x\n", i, op->mem.align);
if (op->mem.lshift != 0)
printf("\t\t\toperands[%u].mem.lshift: 0x%x\n", i, op->mem.lshift);

Expand Down
1 change: 1 addition & 0 deletions docs/cs_v6_release_guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -174,6 +174,7 @@ These features are only supported by `auto-sync`-enabled architectures.
**More code quality checks**

- `clang-tidy` is now run on all files changed by a PR.
- ASAN: All tests are now run with the address sanitizer enabled. This includes checking for leaks.

**Instruction formats for PPC**

Expand Down
2 changes: 1 addition & 1 deletion suite/cstest/Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ LIBRARY = -lcmocka -lcapstone -L../..
all:
rm -rf $(BUILD)
mkdir $(BUILD)
$(CC) $(SOURCE)/*.c $(INCLUDE:%=-I %) -g -o $(BUILD)/cstest $(LIBRARY)
$(CC) $(SOURCE)/*.c $(INCLUDE:%=-I %) ${CMAKE_C_FLAGS} -g -o $(BUILD)/cstest $(LIBRARY)
cstest:
$(BUILD)/cstest -d ../MC
clean:
Expand Down
17 changes: 3 additions & 14 deletions suite/cstest/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,23 +13,12 @@ brew install cmocka

- Build Cmocka

```
cd cmocka_dir
mkdir build
cd build
cmake ..
make
sudo make install
```

## Build

- Build `cstest`
You can build `cstest` with `cmake` when building Capstone. Just pass the `CAPSTONE_BUILD_CSTEST` flag
during configuration.

```
cd suite/cstest
make
```
Alternatively you can use the `build_cstest.sh` file in this directory.

## Usage

Expand Down
Loading
Loading