Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding python wheels #18

Open
wants to merge 35 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
35 commits
Select commit Hold shift + click to select a range
ff44e77
add ci for building wheels
michaelfeil Jan 12, 2024
5c1cf2b
update ci rule
michaelfeil Jan 12, 2024
dd21cac
update release tag
michaelfeil Jan 12, 2024
573ebce
gh release iterate
michaelfeil Jan 12, 2024
489139e
fix release
michaelfeil Jan 12, 2024
36c07a2
update wheel policy
michaelfeil Jan 12, 2024
2d21fc1
update matrix
michaelfeil Jan 12, 2024
cb64488
update flashmm wheels
michaelfeil Jan 12, 2024
f500bb4
add cuda arch list
michaelfeil Jan 12, 2024
1096ddd
remove cudaarchlist
michaelfeil Jan 12, 2024
ce5b10f
update cuda arch path
michaelfeil Jan 12, 2024
378dd60
refine pipeline
michaelfeil Jan 12, 2024
ca34caa
update pipeline
michaelfeil Jan 12, 2024
4de0988
update windows build
michaelfeil Jan 12, 2024
2e50135
update python to python3
michaelfeil Jan 12, 2024
df8450e
abstract file path
michaelfeil Jan 12, 2024
5b2b8d2
release wheels
michaelfeil Jan 12, 2024
2eaf8c8
improve file path
michaelfeil Jan 12, 2024
5dddd1d
remove yaml temp
michaelfeil Jan 12, 2024
1877df3
add pipeline
michaelfeil Jan 12, 2024
f856910
improve script
michaelfeil Jan 12, 2024
9a30e0e
updated paths
michaelfeil Jan 12, 2024
9a2e671
update
michaelfeil Jan 12, 2024
96d7de4
even better file path
michaelfeil Jan 12, 2024
7914672
add python script
michaelfeil Jan 12, 2024
d4a0235
update flashmm
michaelfeil Jan 12, 2024
15e8e4e
build next wheels
michaelfeil Jan 12, 2024
89262b0
update path
michaelfeil Jan 12, 2024
42133ba
add missing yaml
michaelfeil Jan 12, 2024
15be2cc
fix setuppypath
michaelfeil Jan 12, 2024
0bc1564
update workflows
michaelfeil Jan 12, 2024
40e9fba
update paths
michaelfeil Jan 12, 2024
12f6286
update path
michaelfeil Jan 12, 2024
d702422
update setup
michaelfeil Jan 12, 2024
16d5e71
update setup
michaelfeil Jan 12, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 15 additions & 0 deletions .github/workflows/build_flashmm.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
name: Build Wheels

on:
push:
# push on github tags that start with "wheel"
tags:
- "wheel*"

jobs:
flashmm_wheels:
uses:
./.github/workflows/workflow_build_cuda_wheels.yaml
with: # ${{ github.workspace }}
setuppypath: "csrc/flashmm"
secrets: inherit
120 changes: 120 additions & 0 deletions .github/workflows/workflow_build_cuda_wheels.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,120 @@
name: install_wheels_cuda

on:
workflow_call:
inputs:
setuppypath:
required: true
type: string
description: "path to the dir of the setup.py that should be built."

env:
SETUPPYPATH: ${{ inputs.setuppypath }}
MAX_JOBS: "2"

jobs:
build_wheels:
name: Build Cuda Wheels for torch
runs-on: ${{ matrix.os }}

strategy:
matrix:
os: [ubuntu-20.04, windows-latest]
pyver: ["3.8", "3.9", "3.10", "3.11"]
cuda: ["11.8.0", "12.1.1"]
defaults:
run:
shell: pwsh
env:
CUDA_VERSION: ${{ matrix.cuda }}
# define where the setup.py is located

steps:
- name: Free Disk Space
uses: jlumbroso/[email protected]
if: runner.os == 'Linux'
with:
tool-cache: false
android: true
dotnet: true
haskell: true
large-packages: false
docker-images: true
swap-storage: false

- uses: actions/checkout@v3

- uses: actions/setup-python@v3
with:
python-version: ${{ matrix.pyver }}

- name: Setup Mamba
uses: conda-incubator/[email protected]
with:
activate-environment: "build"
python-version: ${{ matrix.pyver }}
miniforge-variant: Mambaforge
miniforge-version: latest
use-mamba: true
add-pip-as-python-dependency: true
auto-activate-base: false

- name: Install Dependencies
run: |
# Install CUDA toolkit
mamba install -y 'cuda' -c "nvidia/label/cuda-${env:CUDA_VERSION}"

# Env variables
$env:CUDA_PATH = $env:CONDA_PREFIX
$env:CUDA_HOME = $env:CONDA_PREFIX

# Install torch
$cudaVersion = $env:CUDA_VERSION.Replace('.', '')
$cudaVersionPytorch = $cudaVersion.Substring(0, $cudaVersion.Length - 1)
if ([int]$cudaVersionPytorch -gt 118) { $pytorchVersion = "torch==2.1.0" } else {$pytorchVersion = "torch==2.0.1"}
python3 -m pip install --upgrade --no-cache-dir $pytorchVersion+cu$cudaVersionPytorch --index-url https://download.pytorch.org/whl/cu$cudaVersionPytorch
python3 -m pip install build setuptools wheel ninja numpy

# Print version information
python3 --version
python3 -c "import torch; print('PyTorch:', torch.__version__)"
python3 -c "import torch; print('CUDA:', torch.version.cuda)"
python3 -c "import os; print('CUDA_HOME:', os.getenv('CUDA_HOME', None))"
python3 -c "from torch.utils import cpp_extension; print (cpp_extension.CUDA_HOME)"

- name: Build Wheel
run: |
$env:CUDA_PATH = $env:CONDA_PREFIX
$env:CUDA_HOME = $env:CONDA_PREFIX

# build the wheel
cd ${{ github.workspace }}/$env:SETUPPYPATH
python3 setup.py sdist bdist_wheel

# rename wheel to reflect cuda version and torch version bash: dist/*.whl
$WHEEL_PATH = python3 -c "if True: \
import os, shutil, glob, re, torch;\
torch_name = '+torch' + (torch.__version__.split('+')[0] + 'cu' + torch.version.cuda).replace('.', '');\
file = glob.glob(os.path.join('dist', '*.whl'))[0];\
current_dir = os.getcwd();\
insert_pos = re.search('-cp[0-9]+-cp[0-9]+-', file);\
torch_name = '' if insert_pos is None else torch_name;\
insert_pos = re.search('-py3', file).start() if insert_pos is None else insert_pos.start();\
new_name = f'{file[:insert_pos]}{torch_name}{file[insert_pos:]}';\
print(os.path.join(os.getcwd(), new_name));\
shutil.move(file, new_name);"

# append to powershell env
echo "WHEEL_PATH=$WHEEL_PATH" | Out-File -FilePath $env:GITHUB_ENV -Encoding utf8 -Append

- name: Release with Notes
uses: softprops/action-gh-release@v1
if: startsWith(github.ref, 'refs/tags/')
with:
files: |
${{ env.WHEEL_PATH }}
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
# from previous step
WHEEL_PATH: ${{ env.WHEEL_PATH }}