Update environment.yaml

.github/workflows/build.yml

@@ -1,6 +1,16 @@
 name: Build webpage
 
-on: [push, pull_request]
+concurrency:
+  group: ${{ github.ref }}
+  cancel-in-progress: true
+
+on:
+  pull_request:
+  push:
+    tags:
+      - "*"
+    branches:
+      - master
 
 jobs:
   build-pages:

.pre-commit-config.yaml

@@ -17,11 +17,11 @@ repos:
     rev: 1.7.0
     hooks:
       - id: nbqa-isort
-        additional_dependencies: [ isort==5.6.4 ]
+        additional_dependencies: [ isort ]
 
       - id: nbqa-pyupgrade
-        additional_dependencies: [ pyupgrade==2.7.4 ]
-        args: [ --py37-plus ]
+        additional_dependencies: [ pyupgrade ]
+        args: [ --py38-plus ]
 #
 #  - repo: https://github.com/ambv/black
 #    rev: 21.9b0

CONTRIBUTING.md

@@ -15,7 +15,7 @@ In exchange,
 we will address your issues and/or assess your change proposal as promptly as we can,
 and help you become a member of our community.
 Everyone involved in [HSF training][hsf-training]
-agrees to abide by our [code of conduct](CODE_OF_CONDUCT.md).
+agrees to abide by our [code of conduct](CONDUCT.md).
 
 ## How to Contribute
 
@@ -49,12 +49,12 @@ There are many ways to contribute,
 from writing new exercises and improving existing ones
 to updating or filling in the documentation
 and submitting [bug reports][issues]
-about things that do not work, aren not clear, or are missing.
+about things that do not work, are not clear, or are missing.
 If you are looking for ideas, please see the 'Issues' tab for
 a list of issues associated with this repository,
 or you may also look at all issues in [hsf-training][hsf-training-issues]
 
-There is also [a list](hsf-training-gfis) of all issues that are particularly easy and suitable
+There is also [a list][hsf-training-gfis] of all issues that are particularly easy and suitable
 for first contributions.
 
 Comments on issues and reviews of pull requests are just as welcome:

LICENSE.md

@@ -1,3 +1,4 @@
+
 # Instructional Material
 
 All instructional material is made available under the [Creative Commons 

README.md

@@ -19,29 +19,36 @@ There are two options for running these lessons. Running locally should be prefe
 
 ### Local
 
-This tutorial uses `Python 3.7` and requires some packages.
-It is recommended to use [Conda](https://docs.conda.io/en/latest/) to install the correct packages.
+This tutorial uses `Python 3.11` and requires some packages.
+It is recommended to use [mambaforge](https://github.com/conda-forge/miniforge#mambaforge) to install the correct packages.
+**Note:** `mamba` is like `conda` and can be used interchangeably. "forge" in the name refers to the [conda-forge](https://conda-forge.org/) channel, _the_ open-source maintained channel which contains a lot of packages.
 
-To install `Conda` you will need to do the following:
+To install `Conda`/`mamba` you will need to do the following:
 
- - Install `Conda` according to the instructions [here](https://docs.conda.io/projects/conda/en/latest/user-guide/install/index.html#installing-in-silent-mode)
- - You can add `source /my/path/for/miniconda/etc/profile.d/conda.sh` to your `.bashrc`
- - Add the channel:
+ - Install `mamba` according to the instructions [here](https://github.com/conda-forge/miniforge#install)
+ - To add `mamba`/`conda` to your shell, follow the instructions after the installation and execute
 ```bash
-conda config --add channels conda-forge
+mamba init
+```
+ - In order no not use the base environment (which you almost never should), do
+```bash
+conda config --set auto_activate_base false
 ```
 
-Now to use your first ```Conda``` environment:
- - Create an environment with some packages already installed:
+
+Now to use your first ```Conda/Mamba``` environment:
+
+ - This will install the above packages. In order to make sure that you install all of the packages needed in the tutorial, you can use the `environment.yml` file (make sure that the file `environment.yml` is in the current directory):
 ```bash
-conda create -n my-analysis-env python=3.7 jupyterlab ipython matplotlib uproot numpy pandas scikit-learn scipy tensorflow xgboost hep_ml wget
+mamba env create -f environment.yml
 ```
- - Activate your environment by doing: `conda activate my-analysis-env`
- - You can install additional packages by doing: `conda install package_name`
- - For the lessons to work fully you will also need to install a special helper package with pip:
+ - Alternatively, you could create an environment with some packages already in this way
 ```bash
-pip install git+https://github.com/hsf-training/python-lesson.git
+mamba create -n analysis-essentials python=3.11 jupyterlab ipython matplotlib uproot numpy pandas scikit-learn scipy tensorflow xgboost hep_ml wget
 ```
+ - Activate your environment by doing: `mamba activate analysis-essentials`
+ - You can install additional packages by doing: `mamba install package_name`
+
 
 You will also need [Jupyter](https://jupyterlab.readthedocs.io/) to run the examples in this tutorial.
 Jupyter was already installed in the previous command and can be ran by following the instructions [here](https://jupyterlab.readthedocs.io/en/stable/getting_started/starting.html).
@@ -86,4 +93,6 @@ If you have any problems or questions, you can [open an issue][issues] on this r
     snakemake/README.md
     git/README.md
     CONTRIBUTING.md
+    CONDUCT.md
+    LICENSE.md
 ```

advanced-python/11AdvancedPython.ipynb

@@ -299,6 +299,7 @@
     "def func(x):\n",
     "    yield x\n",
     "\n",
+    "\n",
     "with func(5) as var1:\n",
     "    print('inside')\n",
     "print(var1)"

advanced-python/12AdvancedClasses.ipynb

@@ -27,16 +27,19 @@
     "    def __init__(self, name):\n",
     "        self.name = name\n",
     "\n",
+    "\n",
     "class ValueLeft(NamedValue):    \n",
     "    def __add__(self, other):\n",
     "        print(f\"add called on {self.name}\")\n",
     "        return 42\n",
     "    \n",
+    "\n",
     "class ValueRight(NamedValue):\n",
     "    def __radd__(self, other):\n",
     "        print(\"radd called on {self.name}\")\n",
     "        return 24\n",
     "    \n",
+    "\n",
     "class Value(ValueRight, ValueLeft):\n",
     "    pass"
    ]
@@ -97,14 +100,17 @@
     "    def __init__(self, name):\n",
     "        self.name = name\n",
     "        \n",
+    "\n",
     "class NameRepr(Name):\n",
     "    def __repr__(self):\n",
     "        return self.name\n",
     "\n",
+    "\n",
     "class NameStr(Name):\n",
     "    def __str__(self):\n",
     "        return f'I am {self.name}'\n",
     "    \n",
+    "\n",
     "class NameStrRepr(NameStr, NameRepr):\n",
     "    pass"
    ]
@@ -135,6 +141,7 @@
     "    def __call__(self, *args, **kwargs):\n",
     "        print(f\"called with args {args} and kwargs {kwargs}\")\n",
     "        \n",
+    "\n",
     "class NotCallable:\n",
     "    pass"
    ]

advanced-python/20DataAndPlotting.ipynb

@@ -278,6 +278,7 @@
     "    plt.xlabel('$J/\\\\psi$ mass [GeV]')\n",
     "    plt.xlim(bins[0], bins[-1])\n",
     "\n",
+    "\n",
     "plot_mass(data_df)"
    ]
   },
@@ -380,6 +381,7 @@
     "    plt.xlabel('$J/\\\\psi$ mass [GeV]')\n",
     "    plt.xlim(bins[0], bins[-1])\n",
     "\n",
+    "\n",
     "plot_mass(data_df, label='No cuts', density=1)\n",
     "data_with_cuts_df = data_df.query('Jpsi_PT > 4')\n",
     "plot_mass(data_with_cuts_df, label='$J/\\\\psi$ p$_T$ only', density=1)\n",

advanced-python/30Classification.ipynb

@@ -258,6 +258,7 @@
     "    plt.xlim(bins[0], bins[-1])\n",
     "    plt.legend(loc='best')\n",
     "\n",
+    "\n",
     "def plot_mass(df, **kwargs):\n",
     "    h, bins = np.histogram(df['Jpsi_M'], bins=100, range=[2.75, 3.5])\n",
     "    mplhep.histplot(h, bins, yerr=True, **kwargs)  # feel free to adjust\n",

advanced-python/31ClassificationExtension.ipynb

@@ -66,6 +66,7 @@
     "    plt.xlim(bins[0], bins[-1])\n",
     "    plt.legend(loc='best')\n",
     "\n",
+    "\n",
     "def plot_significance(bdt, training_data, training_columns, label=None):\n",
     "    y_score = bdt.predict_proba(training_data[training_columns])[:,1]\n",
     "    fpr, tpr, thresholds = roc_curve(training_data['catagory'], y_score)\n",
@@ -84,6 +85,7 @@
     "    optimal_cut = thresholds[np.argmax(metric)]\n",
     "    plt.axvline(optimal_cut, color='black', linestyle='--')\n",
     "\n",
+    "\n",
     "def plot_roc(bdt, training_data, training_columns, label=None):\n",
     "    y_score = bdt.predict_proba(training_data[training_columns])[:,1]\n",
     "    fpr, tpr, thresholds = roc_curve(training_data['catagory'], y_score)\n",

advanced-python/32BoostingToUniformity.ipynb

@@ -80,6 +80,7 @@
     "    plt.ylabel(var_name2)\n",
     "    plt.colorbar()\n",
     "\n",
+    "\n",
     "plt.figure(figsize=(12, 6))\n",
     "plt.subplot(1, 2, 1), plt.title(\"signal\"),       plot_distribution(data[labels==1])\n",
     "plt.subplot(1, 2, 2), plt.title(\"background\"),   plot_distribution(data[labels==0]);"
@@ -181,7 +182,7 @@
     "        output = clf.predict_proba(trainX[train_features])\n",
     "    else:\n",
     "        output = clf.predict_proba(trainX[train_features + uniform_features])\n",
-    "    print('Area under curve: {}'.format(roc_auc_score(trainY, output[:,1])))"
+    "    print(f'Area under curve: {roc_auc_score(trainY, output[:,1])}')"
    ]
   },
   {
@@ -219,6 +220,7 @@
     "    # We can make the plot look nicer by forcing the grid to be square\n",
     "    plt.gca().set_aspect('equal', adjustable='box')\n",
     "\n",
+    "\n",
     "plt.figure(figsize=(8,8))\n",
     "plot_roc(classifiers['AdaBoost'], testY, testX, train_features, 'AdaBoost')\n",
     "plot_roc(classifiers['uGB+knnAda'], testY, testX, train_features+uniform_features, 'uGB+knnAda')\n",

advanced-python/45DemoReweighting.ipynb

@@ -104,6 +104,7 @@
     "\n",
     "hist_settings = {'bins': 100, 'density': True, 'alpha': 0.7}\n",
     "\n",
+    "\n",
     "def draw_distributions(original, target, new_original_weights):\n",
     "    plt.figure(figsize=[15, 7])\n",
     "    for id, column in enumerate(columns, 1):\n",

advanced-python/50LikelihoodInference.ipynb

@@ -190,7 +190,6 @@
     "    if ax is None:\n",
     "        ax = plt.gca()\n",
     "\n",
-    "\n",
     "    lower, upper = data.space.limit1d\n",
     "\n",
     "    # Creates and histogram of the data and plots it with mplhep.\n",

advanced-python/70ScikitHEPUniverse.ipynb

@@ -232,7 +232,7 @@
     "    \"phi\": [2.1, 2.2, 2.3, 2.4, 2.5],\n",
     "    \"eta\": [3.1, 3.2, 3.3, 3.4, 3.5],\n",
     "    \"M\": [4.1, 4.2, 4.3, 4.4, 4.5],\n",
-    "})\n"
+    "})"
    ]
   },
   {

conf.py

@@ -21,6 +21,7 @@
     'conf_py_path': '/',
 }
 
+
 html_static_path += [
     f'_static',
 ]

environment.yml

@@ -11,6 +11,7 @@ dependencies:
   - mplhep
   - nb_conda
   - nb_conda_kernels
+  - notebook<7.0.0  # fixes failed nb_conda install https://github.com/DeepLabCut/DeepLabCut/issues/2322
   - numpy
   - pandas
   - particle
@@ -23,11 +24,9 @@ dependencies:
   - vector
   - wget
   - xgboost
+  - zfit >=0.14.0
+  - hepstats
   - pip:
     - git+https://github.com/hsf-training/python-lesson.git
-    - tensorflow
-    - zfit >=0.8,<0.10
-    - keras <2.7  # temporary fix for https://github.com/keras-team/keras/issues/15579
-    - hepstats
     - formulate
     - starterkit-ci

python/classes.ipynb

@@ -42,6 +42,7 @@
     "pi1_pz = 30\n",
     "pi1_E = 100\n",
     "\n",
+    "\n",
     "def calc_mass_simple(px, py, pz, E):\n",
     "    return np.sqrt(E ** 2 - (px ** 2 + py ** 2 + pz ** 2))"
    ]
@@ -176,13 +177,15 @@
     "            'E': None,\n",
     "            'mass': calc_mass}\n",
     "\n",
+    "\n",
     "def initialize_particle(particle, px, py, pz, E):\n",
     "    particle['px'] = px\n",
     "    particle['py'] = py\n",
     "    particle['pz'] = pz\n",
     "    particle['E'] = E\n",
     "    return particle\n",
     "\n",
+    "\n",
     "particle1 = initialize_particle(make_particle(), px=20, py=30, pz=20, E=50)"
    ]
   },