Merge pull request #2079 from recommenders-team/staging

Staging to main: Support python 3.10 and 3.11

.github/PULL_REQUEST_TEMPLATE.md

@@ -15,7 +15,8 @@
 ### Checklist:
 <!--- Go over all the following points, and put an `x` in all the boxes that apply. -->
 <!--- If you're unsure about any of these, don't hesitate to ask. We're here to help! -->
-- [ ] I have followed the [contribution guidelines](../CONTRIBUTING.md) and code style for this project.
+- [ ] I have followed the [contribution guidelines](CONTRIBUTING.md) and code style for this project.
 - [ ] I have added tests covering my contributions.
 - [ ] I have updated the documentation accordingly.
-- [ ] This PR is being made to `staging branch` and not to `main branch`.
+- [ ] I have [signed the commits](https://github.com/recommenders-team/recommenders/wiki/How-to-sign-commits), e.g. `git commit -s -m "your commit message"`. 
+- [ ] This PR is being made to `staging branch` AND NOT TO `main branch`.

.github/actions/azureml-test/action.yml

@@ -69,56 +69,45 @@ runs:
   using: "composite"
   steps:
     - name: Setup python
-      uses: actions/setup-python@v4
+      uses: actions/setup-python@v5
       with:
         python-version: "3.8"
     - name: Install azureml-core and azure-cli on a GitHub hosted server
       shell: bash
       run: pip install --quiet "azureml-core>1,<2" "azure-cli>2,<3"
     - name: Log in to Azure
-      uses: azure/login@v1
+      uses: azure/login@v2
       with:
         creds: ${{inputs.AZUREML_TEST_CREDENTIALS}}
     - name: Install wheel package
       shell: bash
       run: pip install --quiet wheel
-    - name: Submit CPU tests to AzureML
+    - name: Submit tests to AzureML
       shell: bash
-      if: contains(inputs.TEST_GROUP, 'cpu')
       run: >-
-          python tests/ci/azureml_tests/submit_groupwise_azureml_pytest.py --clustername ${{inputs.CPU_CLUSTER_NAME}}
-          --subid ${{inputs.AZUREML_TEST_SUBID}} --reponame "recommenders" --branch ${{ github.ref }}
-          --rg ${{inputs.RG}} --wsname ${{inputs.WS}} --expname ${{inputs.EXP_NAME}}_${{inputs.TEST_GROUP}}
-          --testlogs ${{inputs.TEST_LOGS_PATH}} --testkind ${{inputs.TEST_KIND}}
-          --conda_pkg_python ${{inputs.PYTHON_VERSION}} --testgroup ${{inputs.TEST_GROUP}}
-          --disable-warnings --sha "${GITHUB_SHA}"
-    - name: Submit GPU tests to AzureML
-      shell: bash
-      if: contains(inputs.TEST_GROUP, 'gpu')
-      run: >-
-          python tests/ci/azureml_tests/submit_groupwise_azureml_pytest.py --clustername ${{inputs.GPU_CLUSTER_NAME}}
-          --subid ${{inputs.AZUREML_TEST_SUBID}} --reponame "recommenders" --branch ${{ github.ref }}
-          --rg ${{inputs.RG}} --wsname ${{inputs.WS}} --expname ${{inputs.EXP_NAME}}_${{inputs.TEST_GROUP}}
-          --testlogs ${{inputs.TEST_LOGS_PATH}} --add_gpu_dependencies --testkind ${{inputs.TEST_KIND}}
-          --conda_pkg_python ${{inputs.PYTHON_VERSION}} --testgroup ${{inputs.TEST_GROUP}}
-          --disable-warnings --sha "${GITHUB_SHA}"
-    - name: Submit PySpark tests to AzureML
-      shell: bash
-      if: contains(inputs.TEST_GROUP, 'spark')
-      run: >-
-          python tests/ci/azureml_tests/submit_groupwise_azureml_pytest.py --clustername ${{inputs.CPU_CLUSTER_NAME}}
-          --subid ${{inputs.AZUREML_TEST_SUBID}} --reponame "recommenders" --branch ${{ github.ref }}
-          --rg ${{inputs.RG}} --wsname ${{inputs.WS}} --expname ${{inputs.EXP_NAME}}_${{inputs.TEST_GROUP}}
-          --testlogs ${{inputs.TEST_LOGS_PATH}} --add_spark_dependencies --testkind ${{inputs.TEST_KIND}}
-          --conda_pkg_python ${{inputs.PYTHON_VERSION}} --testgroup ${{inputs.TEST_GROUP}}
-          --disable-warnings --sha "${GITHUB_SHA}"
+        python tests/ci/azureml_tests/submit_groupwise_azureml_pytest.py \
+          --subid ${{inputs.AZUREML_TEST_SUBID}} \
+          --reponame "recommenders" \
+          --branch ${{ github.ref }} \
+          --rg ${{inputs.RG}} \
+          --wsname ${{inputs.WS}} \
+          --expname ${{inputs.EXP_NAME}}_${{inputs.TEST_GROUP}} \
+          --testlogs ${{inputs.TEST_LOGS_PATH}} \
+          --testkind ${{inputs.TEST_KIND}} \
+          --conda_pkg_python ${{inputs.PYTHON_VERSION}} \
+          --testgroup ${{inputs.TEST_GROUP}} \
+          --disable-warnings \
+          --sha "${GITHUB_SHA}" \
+          --clustername $(if [[ ${{inputs.TEST_GROUP}} =~ "gpu" ]]; then echo "${{inputs.GPU_CLUSTER_NAME}}"; else echo "${{inputs.CPU_CLUSTER_NAME}}"; fi) \
+          $(if [[ ${{inputs.TEST_GROUP}} =~ "gpu" ]]; then echo "--add_gpu_dependencies"; fi) \
+          $(if [[ ${{inputs.TEST_GROUP}} =~ "spark" ]]; then echo "--add_spark_dependencies"; fi)
     - name: Get exit status
       shell: bash
       id: exit_status
       run: echo "code=$(cat ${{inputs.PYTEST_EXIT_CODE}})" >> $GITHUB_OUTPUT
     - name: Check Success/Failure
       if: ${{ steps.exit_status.outputs.code != 0 }}
-      uses: actions/github-script@v3
+      uses: actions/github-script@v7
       with:
         script: |
             core.setFailed('All tests did not pass!')

.github/workflows/azureml-cpu-nightly.yml

@@ -47,7 +47,7 @@ jobs:
     runs-on: ubuntu-latest
     steps:
       - name: Check out repository code
-        uses: actions/checkout@v3
+        uses: actions/checkout@v4
       - name: Get test group names
         id: get_test_groups
         uses: ./.github/actions/get-test-groups
@@ -67,11 +67,11 @@ jobs:
     strategy:
       max-parallel: 50 # Usage limits: https://docs.github.com/en/actions/learn-github-actions/usage-limits-billing-and-administration
       matrix:
-        python-version: ['"python=3.8"', '"python=3.9"']
+        python-version: ['"python=3.8"', '"python=3.9"', '"python=3.10"', '"python=3.11"']
         test-group:  ${{ fromJSON(needs.get-test-groups.outputs.test_groups) }}
     steps:
       - name: Check out repository code
-        uses: actions/checkout@v3
+        uses: actions/checkout@v4
       - name: Execute tests
         uses: ./.github/actions/azureml-test
         id: execute_tests

.github/workflows/azureml-gpu-nightly.yml

@@ -47,7 +47,7 @@ jobs:
     runs-on: ubuntu-latest
     steps:
       - name: Check out repository code
-        uses: actions/checkout@v3
+        uses: actions/checkout@v4
       - name: Get test group names
         id: get_test_groups
         uses: ./.github/actions/get-test-groups
@@ -67,11 +67,11 @@ jobs:
     strategy:
       max-parallel: 50 # Usage limits: https://docs.github.com/en/actions/learn-github-actions/usage-limits-billing-and-administration
       matrix:
-        python-version: ['"python=3.8"', '"python=3.9"']
+        python-version: ['"python=3.8"', '"python=3.9"', '"python=3.10"', '"python=3.11"']
         test-group:  ${{ fromJSON(needs.get-test-groups.outputs.test_groups) }}
     steps:
       - name: Check out repository code
-        uses: actions/checkout@v3
+        uses: actions/checkout@v4
       - name: Execute tests
         uses: ./.github/actions/azureml-test
         id: execute_tests

.github/workflows/azureml-release-pipeline.yml

@@ -33,9 +33,9 @@ jobs:
     needs: [unit-test-workflow, cpu-nightly-workflow, gpu-nightly-workflow, spark-nightly-workflow]
     steps:
       - name: Check out repository code
-        uses: actions/checkout@v3
+        uses: actions/checkout@v4
       - name: Setup python
-        uses: actions/setup-python@v4
+        uses: actions/setup-python@v5
         with: 
           python-version: "3.8"
       - name: Install wheel package

.github/workflows/azureml-spark-nightly.yml

@@ -46,7 +46,7 @@ jobs:
     runs-on: ubuntu-latest
     steps:
       - name: Check out repository code
-        uses: actions/checkout@v3
+        uses: actions/checkout@v4
       - name: Get test group names
         id: get_test_groups
         uses: ./.github/actions/get-test-groups
@@ -66,11 +66,11 @@ jobs:
     strategy:
       max-parallel: 50 # Usage limits: https://docs.github.com/en/actions/learn-github-actions/usage-limits-billing-and-administration
       matrix:
-        python-version: ['"python=3.8"', '"python=3.9"']
+        python-version: ['"python=3.8"', '"python=3.9"', '"python=3.10"', '"python=3.11"']
         test-group:  ${{ fromJSON(needs.get-test-groups.outputs.test_groups) }}
     steps:
       - name: Check out repository code
-        uses: actions/checkout@v3
+        uses: actions/checkout@v4
       - name: Execute tests
         uses: ./.github/actions/azureml-test
         id: execute_tests

.github/workflows/azureml-unit-tests.yml

@@ -36,7 +36,7 @@ jobs:
     runs-on: ubuntu-latest
     steps:
       - name: Check out repository code
-        uses: actions/checkout@v3
+        uses: actions/checkout@v4
       - name: Get test group names
         id: get_test_groups
         uses: ./.github/actions/get-test-groups
@@ -56,11 +56,11 @@ jobs:
     strategy:
       max-parallel: 50 # Usage limits: https://docs.github.com/en/actions/learn-github-actions/usage-limits-billing-and-administration
       matrix:
-        python-version: ['"python=3.8"', '"python=3.9"']
+        python-version: ['"python=3.8"', '"python=3.9"', '"python=3.10"', '"python=3.11"']
         test-group:  ${{ fromJSON(needs.get-test-groups.outputs.test_groups) }}
     steps:
       - name: Check out repository code
-        uses: actions/checkout@v3
+        uses: actions/checkout@v4
       - name: Execute tests
         uses: ./.github/actions/azureml-test
         id: execute_tests

.github/workflows/sarplus.yml

@@ -39,12 +39,12 @@ jobs:
     runs-on: ubuntu-22.04
     strategy:
       matrix:
-        python-version: ["3.8", "3.9"]
+        python-version: ["3.8", "3.9", "3.10", "3.11"]
     steps:
-      - uses: actions/checkout@v3
+      - uses: actions/checkout@v4
 
       - name: Set up Python ${{ matrix.python-version }}
-        uses: actions/setup-python@v4
+        uses: actions/setup-python@v5
         with:
           python-version: ${{ matrix.python-version }}
 
@@ -96,15 +96,15 @@ jobs:
       - name: Upload Python wheel as GitHub artifact when merged into main
         # Upload the whl file of the specific python version
         if: github.ref == 'refs/heads/main'
-        uses: actions/upload-artifact@v2
+        uses: actions/upload-artifact@v4
         with:
           name: pysarplus-${{ env.sarplus_version }}-cp${{ matrix.python-version }}-wheel
           path: ${{ env.PYTHON_ROOT }}/dist/*.whl
 
       - name: Upload Python source as GitHub artifact when merged into main
         # Only one pysarplus source tar file is needed
         if: github.ref == 'refs/heads/main' && matrix.python-version == '3.10'
-        uses: actions/upload-artifact@v2
+        uses: actions/upload-artifact@v4
         with:
           name: pysarplus-${{ env.sarplus_version }}-source
           path: ${{ env.PYTHON_ROOT }}/dist/*.tar.gz
@@ -131,7 +131,7 @@ jobs:
             hadoop-version: "3.3.1"
 
     steps:
-      - uses: actions/checkout@v3
+      - uses: actions/checkout@v4
 
       - name: Test
         run: |
@@ -180,7 +180,7 @@ jobs:
 
       - name: Upload Scala bundle as GitHub artifact when merged into main
         if: github.ref == 'refs/heads/main'
-        uses: actions/upload-artifact@v2
+        uses: actions/upload-artifact@v4
         with:
           name: sarplus-${{ env.sarplus_version }}-bundle_2.12-spark-${{ matrix.spark-version }}-jar
           path: ${{ env.SCALA_ROOT }}/target/scala-2.12/*bundle*.jar

.github/workflows/update_documentation.yml

@@ -8,20 +8,20 @@ name: Update Documentation
 on:
   push:
     branches:
-      - miguel/jupyter_book
+      - main
 
 jobs:
   build:
     runs-on: ubuntu-22.04
 
     steps:
       - name: Checkout repository
-        uses: actions/checkout@v3
+        uses: actions/checkout@v4
 
       - name: Set up Python
-        uses: actions/setup-python@v4
+        uses: actions/setup-python@v5
         with:
-          python-version: 3.9
+          python-version: "3.10"
 
       - name: Install dependencies
         run: |

CONTRIBUTING.md

@@ -23,15 +23,15 @@ Contributions are welcomed! Here's a few things to know:
 Here are the basic steps to get started with your first contribution. Please reach out with any questions.
 1. Use [open issues](https://github.com/Microsoft/Recommenders/issues) to discuss the proposed changes. Create an issue describing changes if necessary to collect feedback. Also, please use provided labels to tag issues so everyone can easily sort issues of interest.
 1. [Fork the repo](https://help.github.com/articles/fork-a-repo/) so you can make and test local changes.
-1. Create a new branch **from staging branch** for the issue (please do not create a branch from main). We suggest prefixing the branch with your username and then a descriptive title: (e.g. gramhagen/update_contributing_docs)
+1. Create a new branch **from staging branch** for the issue (please do not create a branch from main). We suggest prefixing the branch with your username and then a descriptive title: (e.g. `gramhagen/update_contributing_docs`)
 1. Install recommenders package locally using the right optional dependency for your test and the dev option. (e.g. gpu test: `pip install -e .[gpu,dev]`)
 1. Create a test that replicates the issue.
 1. Make code changes.
 1. Ensure unit tests pass and code style / formatting is consistent (see [wiki](https://github.com/Microsoft/Recommenders/wiki/Coding-Guidelines#python-and-docstrings-style) for more details).
 1. When adding code to the repo, make sure you sign the commits, otherwise the tests will fail (see [how to sign the commits](https://github.com/recommenders-team/recommenders/wiki/How-to-sign-commits)).
 1. Create a pull request against **staging** branch.
 
-Once the features included in a [milestone](https://github.com/microsoft/recommenders/milestones) are completed, we will merge staging into main. See the wiki for more detail about our [merge strategy](https://github.com/microsoft/recommenders/wiki/Strategy-to-merge-the-code-to-main-branch).
+See the wiki for more details about our [merging strategy](https://github.com/microsoft/recommenders/wiki/Strategy-to-merge-the-code-to-main-branch).
 
 ## Coding Guidelines
 

GLOSSARY.md

@@ -9,7 +9,7 @@ Licensed under the MIT License.
 
 * **Click-through rate (CTR)**: Ratio of the number of users who click on a link over the total number of users that visited the page. CTR is a measure of the user engagement.
 
-* **Cold-start problem**: The cold start problem concerns the recommendations for users with no or few past history (new users). Providing recommendations to users with small past history becomes a difficult problem for collaborative filtering models because their learning and predictive ability is limited. Multiple research have been conducted in this direction using content-based filtering models or hybrid models. These models use auxiliary information like user or item metadata to overcome the cold start problem.
+* **Cold-start problem**: The cold start problem concerns the recommendations for users with no or few past history (new users). Providing recommendations to users with small past history becomes a difficult problem for collaborative filtering models because their learning and predictive ability is limited. Multiple research have been conducted in this direction using content-based filtering models. These models use auxiliary information like user or item metadata to overcome the cold start problem.
 
 * **Collaborative filtering algorithms (CF)**: CF algorithms make prediction of what is the likelihood of a user selecting an item based on the behavior of other users [1]. It assumes that if user A likes item X and Y, and user B likes item X, user B would probably like item Y. See the [list of CF examples in Recommenders repository](examples/02_model_collaborative_filtering).
 
@@ -21,8 +21,6 @@ Licensed under the MIT License.
 
 * **Explicit interaction data**: When a user explicitly rate an item, typically between 1-5, the user is giving a value on the likeliness of the item. 
 
-* **Hybrid filtering algorithms**: This type of recommendation system can implement a combination of collaborative and content-based filtering models. See the [list of examples in Recommenders repository](examples/02_model_hybrid).
-
 * **Implicit interaction data**: Implicit interactions are views or clicks that show a certain interest of the user about a specific items. These kind of data is more common but it doesn't define the intention of the user as clearly as the explicit data.
 
 * **Item information**: These include information about the item, some examples can be name, description, price, etc.