Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add --noarch-build-subdir option #172

Open
wants to merge 10 commits into
base: master
Choose a base branch
from

Conversation

mingwandroid
Copy link
Contributor

@mingwandroid mingwandroid commented Oct 31, 2020

It would be nice to know that we can still build noarch and noarch python on other
OSes (well, except Windows at present). This allows that.

Also, in combination with --output-dir it allows building locally as would happen on
c3i. A sample script to glue this together is:

#!/usr/bin/env bash

set -e
FS_STUB=$1

__conda_setup="$(/opt/conda/bin/conda 'shell.bash' 'hook' 2> /dev/null)"
if [ $? -eq 0 ]; then
    eval "$__conda_setup"
fi

if [[ $(uname) == Darwin ]]; then
  BUILD_SUBDIR=osx-64
elif [[ $(uname) =~ M.* ]]; then
  BUILD_SUBDIR=win-64
else
  BUILD_SUBDIR=linux-64
fi

mkdir /tmp/$$.c3i-locally
pushd /tmp/$$.c3i-locally
  conda activate
  cp -rf /opt/r/a/$FS_STUB-feedstock .
  cp -f /opt/r/a/conda_build_config.yaml .
  c3i one-off rdonnelly.$FS_STUB  \
              $FS_STUB-feedstock  \
              --config-root-dir=/opt/asrc/automated-build/c3i_configurations/github_public  \
              --dry-run  \
              --output-dir $PWD  \
              --noarch-build-subdir ${BUILD_SUBDIR}  \
              --no-skip-existing
  mkdir rsync-recipes
  cp -rf *-on-* rsync-recipes/
  mkdir stats
  cmds=$(cat plan.yml | yq -r '.jobs[0].plan[] | select(.task=="build").config.run.args[1]')
  echo $cmds
  conda deactivate
  eval ${cmds}
popd

It would be nice to know that we can still build noarch and noarch python on other
OSes (well, except Windows at present). This allows that.

Also, in combination with --output-dir it allows building locally as would happen on
c3i. A sample script to glue this together is:

```

set -e
FS_STUB=$1

__conda_setup="$(/opt/conda/bin/conda 'shell.bash' 'hook' 2> /dev/null)"
if [ $? -eq 0 ]; then
    eval "$__conda_setup"
fi

if [[ $(uname) == Darwin ]]; then
  BUILD_SUBDIR=osx-64
elif [[ $(uname) =~ M.* ]]; then
  BUILD_SUBDIR=win-64
else
  BUILD_SUBDIR=linux-64
fi

mkdir /tmp/$$.c3i-locally
pushd /tmp/$$.c3i-locally
  conda activate
  cp -rf /opt/r/a/$FS_STUB-feedstock .
  cp -f /opt/r/a/conda_build_config.yaml .
  c3i one-off rdonnelly.$FS_STUB  \
              $FS_STUB-feedstock  \
              --config-root-dir=/opt/asrc/automated-build/c3i_configurations/github_public  \
              --dry-run  \
              --output-dir $PWD  \
              --noarch-build-subdir ${BUILD_SUBDIR}  \
              --no-skip-existing
  mkdir rsync-recipes
  cp -rf *-on-* rsync-recipes/
  mkdir stats
  cmds=$(cat plan.yml | yq -r '.jobs[0].plan[] | select(.task=="build").config.run.args[1]')
  echo $cmds
  conda deactivate
  eval ${cmds}
popd
```
Ended up causing multiple builds inside of conda-build, one without the CBC file
being considered (we ended up with a meta.yaml and also a recipe/meta.yaml because
the cp -Rf fallback was incorrect).

Most people wouldn't see this problem as it only happens when shutil.copytree()
takes exception.
.. for local and docker build support. run_plan_yaml() does the work,
at the moment, osx builds will get launched via subprocess.check_output()
and gets called via submit_local()

We need a job loop with dependencies instead and we need to hook this up
to docker. Is there a good Python interface to docker?
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant