Skip to content

Commit

Permalink
Delete all Prefast tasks (microsoft#17522)
Browse files Browse the repository at this point in the history
### Description
Delete all Prefast tasks because the new VS 17.7 version crashes every
time when we run the task on our CI build servers. However, we cannot
reproduce it locally. And this problem blocks us installing security
patches to our CI build machines.

Will use [CodeQL](https://codeql.github.com/) instead. 

### Motivation and Context
Address some security alerts.
  • Loading branch information
snnn authored Sep 13, 2023
1 parent f923eec commit 9b755dc
Show file tree
Hide file tree
Showing 9 changed files with 0 additions and 318 deletions.
133 changes: 0 additions & 133 deletions .github/workflows/sca.yml

This file was deleted.

6 changes: 0 additions & 6 deletions tools/ci_build/github/azure-pipelines/post-merge-jobs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -74,7 +74,6 @@ stages:
isX86: false
job_name_suffix: x64_RelWithDebInfo
RunOnnxRuntimeTests: true
RunStaticCodeAnalysis: false
ORT_EP_NAME: CUDA
WITH_CACHE: true
MachinePool: onnxruntime-Win2022-GPU-MultiA10
Expand All @@ -95,7 +94,6 @@ stages:
isX86: false
job_name_suffix: x64_RelWithDebInfo
RunOnnxRuntimeTests: true
RunStaticCodeAnalysis: false
ORT_EP_NAME: TRT
WITH_CACHE: true
MachinePool: onnxruntime-Win2022-GPU-MultiA10
Expand All @@ -114,7 +112,6 @@ stages:
isX86: false
job_name_suffix: x64_mimalloc
RunOnnxRuntimeTests: true
RunStaticCodeAnalysis: false
isTraining: false
ORT_EP_NAME: CPU
GenerateDocumentation: false
Expand All @@ -134,7 +131,6 @@ stages:
isX86: false
job_name_suffix: x64_no_memory_profiling
RunOnnxRuntimeTests: false
RunStaticCodeAnalysis: false
isTraining: false
ORT_EP_NAME: CPU
GenerateDocumentation: false
Expand All @@ -154,7 +150,6 @@ stages:
isX86: false
job_name_suffix: x64_minimal_no_exception
RunOnnxRuntimeTests: true
RunStaticCodeAnalysis: false
isTraining: false
ORT_EP_NAME: CPU
GenerateDocumentation: false
Expand All @@ -174,7 +169,6 @@ stages:
isX86: false
job_name_suffix: x64_debug_node_input_output
RunOnnxRuntimeTests: true
RunStaticCodeAnalysis: false
isTraining: false
ORT_EP_NAME: CPU
GenerateDocumentation: false
Expand Down
21 changes: 0 additions & 21 deletions tools/ci_build/github/azure-pipelines/templates/compliance.yml
Original file line number Diff line number Diff line change
Expand Up @@ -18,27 +18,6 @@ steps:
AnalyzeTargetGlob: '+:file|$(Build.ArtifactStagingDirectory)\**\*.dll;-:file|$(Build.ArtifactStagingDirectory)\**\DirectML.dll'
continueOnError: true

- task: DeleteFiles@1
displayName: 'Delete files from $(Build.BinariesDirectory)\RelWithDebInfo'
inputs:
SourceFolder: '$(Build.BinariesDirectory)\RelWithDebInfo'
Contents: |
**/*.obj
**/*.pdb
**/*.dll
# Manually set msBuildCommandline so that we can also set CAExcludePath
- task: SDLNativeRules@3
displayName: 'Run the PREfast SDL Native Rules for MSBuild'
inputs:
userProvideBuildInfo: msBuildInfo
msBuildArchitecture: x64
msBuildVersion: 17.0
msBuildCommandline: '"C:\Program Files\Microsoft Visual Studio\2022\Enterprise\MSBuild\Current\Bin\amd64\msbuild.exe" "$(Build.BinariesDirectory)\RelWithDebInfo\onnxruntime.sln" /p:platform="${{parameters.msbuildPlatform}}" /p:configuration="RelWithDebInfo" /p:CAExcludePath="$(Build.BinariesDirectory);$(Build.SourcesDirectory)\cmake;C:\program files (x86)" /p:VisualStudioVersion="17.0" /m /p:PreferredToolArchitecture=x64'
excludedPaths: '$(Build.SourcesDirectory)\b#$(Build.SourcesDirectory)\cmake#C:\program files#C:\program files (x86)#C:\program files'
rulesetName: Custom
customRuleset: $(Build.SourcesDirectory)\cmake\Sdl.ruleset

- task: SdtReport@2
displayName: 'Create Security Analysis Report'
inputs:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -34,11 +34,6 @@ parameters:
type: boolean
default: true

- name: RunStaticCodeAnalysis
displayName: Run Static Code Analysis
type: boolean
default: true

- name: ORT_EP_NAME
type: string

Expand Down Expand Up @@ -220,49 +215,6 @@ jobs:
workingDirectory: '$(Build.BinariesDirectory)\${{ parameters.BuildConfig }}\${{ parameters.BuildConfig }}'
displayName: 'Run tests'
- ${{ if eq(parameters.RunStaticCodeAnalysis, true) }}:
- task: DeleteFiles@1
displayName: 'Delete binaries files from $(Build.BinariesDirectory)\RelWithDebInfo'
inputs:
SourceFolder: '$(Build.BinariesDirectory)\RelWithDebInfo'
Contents: |
**/*.obj
**/*.pdb
**/*.dll
# Manually set msBuildCommandline so that we can also set CAExcludePath
# build_dir must be a sub folder of $(Build.SourcesDirectory)
# TODO: move this step to a CPU-only machine to save GPU resources.
- task: SDLNativeRules@3
displayName: 'Run the PREfast SDL Native Rules for MSBuild'
inputs:
msBuildArchitecture: amd64
setupCommandlines: 'python $(Build.SourcesDirectory)\tools\ci_build\build.py --config RelWithDebInfo --build_dir $(Build.SourcesDirectory)\b --skip_submodule_sync --build_shared_lib --update --cmake_generator "Visual Studio 17 2022" --build_shared_lib --enable_onnx_tests ${{ parameters.additionalBuildFlags }} --cmake_extra_defines onnxruntime_ENABLE_STATIC_ANALYSIS=ON onnxruntime_ENABLE_LTO=OFF'
msBuildCommandline: '"C:\Program Files\Microsoft Visual Studio\2022\Enterprise\MSBuild\Current\Bin\amd64\msbuild.exe" "$(Build.SourcesDirectory)\b\RelWithDebInfo\onnxruntime.sln" /p:RunCodeAnalysis=true /p:platform=${{ parameters.msbuildPlatform }} /p:configuration=RelWithDebInfo /p:VisualStudioVersion="17.0" /m /p:PreferredToolArchitecture=x64'
excludedPaths: '$(Build.SourcesDirectory)\b#$(Build.SourcesDirectory)\cmake#C:\program files#C:\program files (x86)#C:\program files'
rulesetName: Custom
customRuleset: $(Build.SourcesDirectory)\cmake\Sdl.ruleset
publishXML: true

- task: SdtReport@2
displayName: 'Create Security Analysis Report'
inputs:
SDLNativeRules: true

- task: PublishSecurityAnalysisLogs@3
displayName: 'Publish Security Analysis Logs'
continueOnError: true

- task: PostAnalysis@2
displayName: 'Guardian Break v2'
inputs:
GdnBreakGdnToolSDLNativeRulesSeverity: Note
GdnBreakGdnToolSDLNativeRules: true


- ${{ if eq(parameters.RunOnnxRuntimeTests, true) }}:
- task: PublishTestResults@2
displayName: 'Publish unit test results'
inputs:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -246,24 +246,6 @@ stages:
workingDirectory: '$(Build.BinariesDirectory)\$(BuildConfig)\$(BuildConfig)'
displayName: 'Run Python Tests'
#Skip it for 32 bits x86 build. Currently the scan tool has a bug: it doesn't allow me use 64 bits link.exe
#in 32 bits Win32 build. I tried all the settings but they all don't work.
- task: SDLNativeRules@3
displayName: 'Run the PREfast SDL Native Rules for MSBuild'
condition: and (succeeded(), and(eq(variables['buildArch'], 'x64'), eq(variables['PythonVersion'], '3.8')))
inputs:
msBuildArchitecture: amd64
setupCommandlines: 'python $(Build.SourcesDirectory)\tools\ci_build\build.py --config Debug --build_dir $(Build.SourcesDirectory)\b --skip_submodule_sync --cmake_generator "Visual Studio 17 2022" --enable_pybind --enable_onnx_tests --parallel $(TelemetryOption) --update --cmake_extra_defines onnxruntime_ENABLE_STATIC_ANALYSIS=ON onnxruntime_ENABLE_LTO=OFF'
msBuildCommandline: '"C:\Program Files\Microsoft Visual Studio\2022\Enterprise\MSBuild\Current\Bin\amd64\msbuild.exe" "$(Build.SourcesDirectory)\b\Debug\onnxruntime.sln" /p:RunCodeAnalysis=true /p:platform="$(MsbuildPlatform)" /p:configuration=Debug /p:VisualStudioVersion="17.0" /m /p:PreferredToolArchitecture=x64'
excludedPaths: '$(Build.SourcesDirectory)\b#$(Build.SourcesDirectory)\cmake#C:\program files#C:\program files (x86)#C:\program files'
rulesetName: Custom
customRuleset: $(Build.SourcesDirectory)\cmake\Sdl.ruleset

- task: SdtReport@2
displayName: 'Create Security Analysis Report'
inputs:
SDLNativeRules: true

- task: TSAUpload@2
displayName: 'TSA upload'
condition: and(and (succeeded(), and(eq(variables['buildArch'], 'x64'), eq(variables['PythonVersion'], '3.8'))), eq(variables['Build.SourceBranch'], 'refs/heads/main'))
Expand Down
59 changes: 0 additions & 59 deletions tools/ci_build/github/azure-pipelines/templates/py-win-gpu.yml
Original file line number Diff line number Diff line change
Expand Up @@ -22,65 +22,6 @@ parameters:
default: ''

jobs:
- ${{ if eq(parameters.PYTHON_VERSION, '3.8') }}:
- job: Win_py_${{ parameters.EP_NAME }}_Wheels_StaticAnalysis
timeoutInMinutes: 240
workspace:
clean: all
pool: onnxruntime-Win-CPU-2022
steps:
- checkout: self
clean: true
submodules: none
- task: UsePythonVersion@0
inputs:
versionSpec: 3.8
addToPath: true
architecture: 'x64'
- task: onebranch.pipeline.tsaoptions@1
displayName: 'OneBranch TSAOptions'
inputs:
tsaConfigFilePath: '$(Build.SourcesDirectory)\.config\tsaoptions.json'
appendSourceBranchName: false

- template: download-deps.yml

- template: jobs/set-winenv.yml
parameters:
EnvSetupScript: ${{ parameters.ENV_SETUP_SCRIPT }}
DownloadCUDA: true

- task: PythonScript@0
displayName: 'Update deps.txt'
inputs:
scriptPath: $(Build.SourcesDirectory)/tools/ci_build/replace_urls_in_deps.py
arguments: --new_dir $(Build.BinariesDirectory)/deps
workingDirectory: $(Build.BinariesDirectory)

- task: SDLNativeRules@3
displayName: 'Run the PREfast SDL Native Rules for MSBuild'
inputs:
msBuildArchitecture: amd64
setupCommandlines: 'python $(Build.SourcesDirectory)\tools\ci_build\build.py --config Debug --build_dir $(Build.SourcesDirectory)\b --skip_submodule_sync --cmake_generator "Visual Studio 17 2022" --enable_pybind ${{ parameters.BUILD_PY_PARAMETERS }} ${{ parameters.EP_BUILD_FLAGS }} --update --cmake_extra_defines onnxruntime_ENABLE_STATIC_ANALYSIS=ON onnxruntime_ENABLE_LTO=OFF'
msBuildCommandline: '"C:\Program Files\Microsoft Visual Studio\2022\Enterprise\MSBuild\Current\Bin\amd64\msbuild.exe" "$(Build.SourcesDirectory)\b\Debug\onnxruntime.sln" /p:RunCodeAnalysis=true /p:platform=x64 /p:configuration=Debug /p:VisualStudioVersion="17.0" /m /p:PreferredToolArchitecture=x64'
excludedPaths: '$(Build.SourcesDirectory)\b#$(Build.SourcesDirectory)\cmake#C:\program files#C:\program files (x86)#C:\program files'
rulesetName: Custom
customRuleset: $(Build.SourcesDirectory)\cmake\Sdl.ruleset
publishXML: true

- task: SdtReport@2
displayName: 'Create Security Analysis Report'
inputs:
SDLNativeRules: true

- task: TSAUpload@2
displayName: 'TSA upload'
condition: and (succeeded(), eq(variables['Build.SourceBranch'], 'refs/heads/main'))
inputs:
GdnPublishTsaOnboard: false
GdnPublishTsaConfigFile: '$(Build.sourcesDirectory)\.gdn\.gdntsa'


- job: Win_py_${{ parameters.EP_NAME }}_Wheels_${{ replace(parameters.PYTHON_VERSION,'.','_') }}
timeoutInMinutes: 240
workspace:
Expand Down
19 changes: 0 additions & 19 deletions tools/ci_build/github/azure-pipelines/templates/win-ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -263,25 +263,6 @@ stages:
AnalyzeTargetGlob: '$(Build.BinariesDirectory)\RelWithDebInfo\RelWithDebInfo\**\*.dll'
continueOnError: true

- task: DeleteFiles@1
displayName: 'Delete files from $(Build.BinariesDirectory)\RelWithDebInfo'
inputs:
SourceFolder: '$(Build.BinariesDirectory)\RelWithDebInfo'
Contents: |
**/*.obj
**/*.pdb
**/*.dll
#Manually set msBuildCommandline so that we can also set CAExcludePath
- task: SDLNativeRules@3
displayName: 'Run the PREfast SDL Native Rules for MSBuild'
condition: and (succeeded(), eq(variables['msbuildPlatform'], 'x64'))
inputs:
msBuildArchitecture: amd64
setupCommandlines: 'python $(Build.SourcesDirectory)\tools\ci_build\build.py --config Debug --disable_rtti --build_dir $(Build.BinariesDirectory) --skip_submodule_sync --build_shared_lib --update --cmake_generator "$(VSGenerator)" --enable_onnx_tests $(TelemetryOption) ${{ parameters.buildparameter }} --cmake_extra_defines onnxruntime_ENABLE_STATIC_ANALYSIS=ON'
msBuildCommandline: '"C:\Program Files\Microsoft Visual Studio\2022\Enterprise\MSBuild\Current\Bin\amd64\msbuild.exe" "$(Build.BinariesDirectory)\Debug\onnxruntime.sln" /p:platform="$(MsbuildPlatform)" /p:configuration=Debug /p:VisualStudioVersion="17.0" /m /p:PreferredToolArchitecture=x64'
excludedPaths: '$(Build.BinariesDirectory)#$(Build.SourcesDirectory)\cmake#C:\program files (x86)'

- task: PostAnalysis@2
inputs:
GdnBreakAllTools: false
Expand Down
Loading

0 comments on commit 9b755dc

Please sign in to comment.