Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Windows Build #773

Open
wants to merge 29 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
c3cd476
initial windows build
RUIJIEZHONG66166 Aug 16, 2024
9df224e
add windows build in pull.yml
RUIJIEZHONG66166 Aug 16, 2024
b4669fd
set ut input as not required
RUIJIEZHONG66166 Aug 16, 2024
4170cee
remove unsupported cmd
RUIJIEZHONG66166 Aug 16, 2024
5c1ab12
add for conda activate
RUIJIEZHONG66166 Aug 16, 2024
1c42704
add call for all conda cmd
RUIJIEZHONG66166 Aug 19, 2024
8877db0
update rm to rmdir
RUIJIEZHONG66166 Aug 20, 2024
9505441
change pytorch check
RUIJIEZHONG66166 Aug 20, 2024
5ab9b63
update copy torch-xpu-ops
RUIJIEZHONG66166 Aug 20, 2024
66d6e50
add dependancy
RUIJIEZHONG66166 Aug 20, 2024
7c37368
add USE_XPU=1
RUIJIEZHONG66166 Aug 20, 2024
5215d4a
use USE_KINETO=0
RUIJIEZHONG66166 Aug 22, 2024
691f66c
update install_oneapi_bundle.bat
RUIJIEZHONG66166 Aug 22, 2024
db51cef
update VS2022INSTALLDIR
RUIJIEZHONG66166 Aug 22, 2024
bccc6a0
add upload windows binary
RUIJIEZHONG66166 Aug 23, 2024
b7834b7
update some name
RUIJIEZHONG66166 Aug 23, 2024
76fa0fd
update upload path
RUIJIEZHONG66166 Aug 23, 2024
1ecea60
add build log upload
RUIJIEZHONG66166 Aug 23, 2024
ead02a3
add build log upload
RUIJIEZHONG66166 Aug 23, 2024
b30feeb
update pti install
RUIJIEZHONG66166 Aug 27, 2024
bf5158b
update bundle install check and call
RUIJIEZHONG66166 Aug 27, 2024
82d6a59
add uninstall print
RUIJIEZHONG66166 Aug 27, 2024
06c4a56
add VS2022INSTALLDIR set
RUIJIEZHONG66166 Aug 28, 2024
628ab2c
update msvc call
RUIJIEZHONG66166 Aug 28, 2024
f00afcc
update msvc call
RUIJIEZHONG66166 Aug 28, 2024
d47060d
update '' to for msvc
RUIJIEZHONG66166 Aug 28, 2024
8b76c05
Merge branch 'main' into ruijie/add_windows_ci
RUIJIEZHONG66166 Sep 6, 2024
ade7744
rebase windows build
RUIJIEZHONG66166 Sep 6, 2024
fc7f59d
Merge branch 'main' into ruijie/add_windows_ci
RUIJIEZHONG66166 Sep 18, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
70 changes: 70 additions & 0 deletions .github/scripts/install_xpu.bat
Original file line number Diff line number Diff line change
@@ -0,0 +1,70 @@
REM Description: Install Intel Support Packages on Windows
REM BKM reference: https://www.intel.com/content/www/us/en/developer/articles/tool/pytorch-prerequisites-for-intel-gpu/2-5.html

:xpu_bundle_install_start

set XPU_BUNDLE_PARENT_DIR=C:\Program Files (x86)\Intel\oneAPI
set XPU_BUNDLE_URL=https://registrationcenter-download.intel.com/akdlm/IRC_NAS/9d1a91e2-e8b8-40a5-8c7f-5db768a6a60c/w_intel-for-pytorch-gpu-dev_p_0.5.3.37_offline.exe
set XPU_PTI_URL=https://registrationcenter-download.intel.com/akdlm/IRC_NAS/9d1a91e2-e8b8-40a5-8c7f-5db768a6a60c/w_intel-pti-dev_p_0.9.0.37_offline.exe
set XPU_BUNDLE_VERSION=0.5.3+31
set XPU_PTI_VERSION=0.9.0+36
set XPU_BUNDLE_PRODUCT_NAME=intel.oneapi.win.intel-for-pytorch-gpu-dev.product
set XPU_PTI_PRODUCT_NAME=intel.oneapi.win.intel-pti-dev.product
set XPU_BUNDLE_INSTALLED=0
set XPU_PTI_INSTALLED=0
set XPU_BUNDLE_UNINSTALL=0
set XPU_PTI_UNINSTALL=0

:: Check if XPU bundle is target version or already installed
if exist "%XPU_BUNDLE_PARENT_DIR%\Installer\installer.exe" goto xpu_bundle_ver_check
goto xpu_bundle_install

:xpu_bundle_ver_check

"%XPU_BUNDLE_PARENT_DIR%\Installer\installer.exe" --list-products > xpu_bundle_installed_ver.log

for /f "tokens=1,2" %%a in (xpu_bundle_installed_ver.log) do (
if "%%a"=="%XPU_BUNDLE_PRODUCT_NAME%" (
echo %%a Installed Version: %%b
set XPU_BUNDLE_INSTALLED=1
if not "%XPU_BUNDLE_VERSION%"=="%%b" (
echo "Old XPU Bundle uninstalling..."
start /wait "Installer Title" "%XPU_BUNDLE_PARENT_DIR%\Installer\installer.exe" --action=remove --eula=accept --silent --product-id %XPU_BUNDLE_PRODUCT_NAME% --product-ver %%b --log-dir uninstall_bundle
set XPU_BUNDLE_UNINSTALL=1
)
)
if "%%a"=="%XPU_PTI_PRODUCT_NAME%" (
echo %%a Installed Version: %%b
set XPU_PTI_INSTALLED=1
if not "%XPU_PTI_VERSION%"=="%%b" (
echo "Old PTI uninstalling..."
start /wait "Installer Title" "%XPU_BUNDLE_PARENT_DIR%\Installer\installer.exe" --action=remove --eula=accept --silent --product-id %XPU_PTI_PRODUCT_NAME% --product-ver %%b --log-dir uninstall_bundle
set XPU_PTI_UNINSTALL=1
)
)
)
if errorlevel 1 exit /b 1
if exist xpu_bundle_installed_ver.log del xpu_bundle_installed_ver.log
if "%XPU_BUNDLE_INSTALLED%"=="0" goto xpu_bundle_install
if "%XPU_BUNDLE_UNINSTALL%"=="1" goto xpu_bundle_install
if "%XPU_PTI_INSTALLED%"=="0" goto xpu_pti_install
if "%XPU_PTI_UNINSTALL%"=="1" goto xpu_pti_install
goto xpu_install_end

:xpu_bundle_install

curl -o xpu_bundle.exe --retry 3 --retry-all-errors -k %XPU_BUNDLE_URL%
echo "XPU Bundle installing..."
start /wait "Intel Pytorch Bundle Installer" "xpu_bundle.exe" --action=install --eula=accept --silent --log-dir install_bundle
if errorlevel 1 exit /b 1
del xpu_bundle.exe

:xpu_pti_install

curl -o xpu_pti.exe --retry 3 --retry-all-errors -k %XPU_PTI_URL%
echo "XPU PTI installing..."
start /wait "Intel PTI Installer" "xpu_pti.exe" --action=install --eula=accept --silent --log-dir install_bundle
if errorlevel 1 exit /b 1
del xpu_pti.exe

:xpu_install_end
7 changes: 7 additions & 0 deletions .github/workflows/pull.yml
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,13 @@ jobs:
abi: 0
ut: op_extended
runner: linux.idc.xpu

preci-windows-build:
# Don't run on forked repos and draft PRs
if: ${{ (github.repository_owner == 'intel') && (github.event.pull_request.draft == false) }}
uses: ./.github/workflows/windows.yml
with:
runner: win_build

Inductor-XPU-E2E-CI-Tests:
runs-on: pvc_e2e
Expand Down
142 changes: 142 additions & 0 deletions .github/workflows/windows.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,142 @@
name: Windows Test

on:
workflow_call:
inputs:
pytorch:
required: false
type: string
default: 'main'
description: Pytorch branch/commit
keep_torch_xpu_ops:
required: false
type: string
default: 'false'
description: Keep torch-xpu-ops pin. `true` means use pined commit
ut:
required: false
type: string
default: ''
description: UT scope. `op_example,op_extended,op_ut,torch_xpu` Delimiter is comma
abi:
required: false
type: string
default: 1
description: ABI version. Default abi as 1.
python:
required: false
type: string
default: '3.10'
description: Python version
runner:
required: true
type: string
default: 'linux.idc.xpu'
description: Runner label

permissions: read-all

env:
USE_XPU: 1
VS2022INSTALLDIR: 'C:\Program Files\Microsoft Visual Studio\2022\BuildTools\VC\Auxiliary\Build\vcvarsall.bat'

jobs:
Torch-XPU-Windows-Build:
runs-on: ${{ inputs.runner }}
timeout-minutes: 900
steps:
- name: Checkout torch-xpu-ops
uses: actions/checkout@v4
- name: Install oneAPI bundle and pti
shell: cmd
run: |
call .github\scripts\install_xpu.bat
if errorlevel 1 exit /b 1
- name: Prepare Stock Pytorch
shell: cmd
run: |
call "C:\ProgramData\miniforge3\Scripts\activate.bat"
call conda clean -ay
call conda remove --all -y -n windows_ci
call conda create -n windows_ci python=${{ inputs.python }} cmake ninja -y
call conda activate windows_ci
cd ..
if exist "pytorch" (
rmdir /s /q pytorch
)
git clone https://github.com/pytorch/pytorch pytorch
cd pytorch && git checkout ${{ inputs.pytorch }}
call pip install pyyaml requests
call conda install conda-forge::rust -y
call conda install typing_extensions -y
python ../torch-xpu-ops/.github/scripts/apply_torch_pr.py
git status && git show -s
git submodule sync && git submodule update --init --recursive
if ${{ inputs.keep_torch_xpu_ops }} == 'true' (
echo "Don't replace torch-xpu-ops!"
) else (
cd third_party
if exist "torch-xpu-ops" (
rmdir /s /q torch-xpu-ops
)
cd ..
Xcopy ..\torch-xpu-ops third_party\torch-xpu-ops /E/H/Y/F/I
)
- name: Build Pytorch XPU
shell: cmd
run: |
call "C:\ProgramData\miniforge3\Scripts\activate.bat"
call "C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Auxiliary\Build\vcvars64.bat"
call "C:\Program Files (x86)\Intel\oneAPI\setvars.bat"
call conda activate windows_ci
pip install mkl-static mkl-include
cd ../pytorch
pip install -r requirements.txt
set USE_KINETO=0
set USE_XPU=1
set CMAKE_SHARED_LINKER_FLAGS=/FORCE:MULTIPLE
set CMAKE_MODULE_LINKER_FLAGS=/FORCE:MULTIPLE
set CMAKE_EXE_LINKER_FLAGS=/FORCE:MULTIPLE
set CMAKE_PREFIX_PATH=${CMAKE_PREFIX_PATH}:${CONDA_PREFIX:-"$(dirname $(which conda))/../"}
if ${{ inputs.abi }} == '0' (
set _GLIBCXX_USE_CXX11_ABI=0
) else (
set _GLIBCXX_USE_CXX11_ABI=1
)
python setup.py bdist_wheel > build_torch_wheel_log.log
for %%f in (${{ github.workspace }}\..\pytorch\dist\torch-*.whl) do (
set "TORCH_WHL=%%f"
echo "INFO - torch whl found"
)
if defined TORCH_WHL (
pip install %TORCH_WHL%
echo "INFO - torch whl installed"
) else (
echo "INFO - No torch whl found."
exit /b 1
)
pip install --force-reinstall dist/*.whl
pip install -r .ci/docker/requirements-ci.txt
- name: Torch Config
shell: cmd
run: |
call "C:\ProgramData\miniforge3\Scripts\activate.bat"
call conda activate windows_ci
call "C:\Program Files (x86)\Intel\oneAPI\setvars.bat"
python -c "import torch; print(torch.__config__.show())"
python -c "import torch; print(torch.__config__.parallel_info())"
python -c "import torch; print(torch.__config__.torch.xpu.device_count())"

- name: Upload Windows build log
if: ${{ ! cancelled() }}
uses: actions/upload-artifact@v4
with:
name: Torch-XPU-Windows-Log-${{ github.event.pull_request.number || github.sha }}
path: 'C:\actions-runner\_work\torch-xpu-ops\pytorch\build_torch_wheel_log.log'

- name: Upload Windows binary
if: ${{ ! cancelled() }}
uses: actions/upload-artifact@v4
with:
name: Torch-XPU-Windows-Binary-${{ github.event.pull_request.number || github.sha }}
path: 'C:\actions-runner\_work\torch-xpu-ops\pytorch\dist'
Loading