Revise dependencies (LorenFrankLab#822)

* Change dependencies. See details.

Across `pyproject.toml` and both `environment.yml`:
- Alphabetical sort to allow for easier comparison across toml/yaml files.
- Remove dependencies required by other dependencies to reduce build time.
- Move `black` to `test` dependencies. Could be `dev` instead?
- Move `click` to `test` dependencies. CLI should be its own package.
- Add ruff config to `pyproject.toml`. Flake8 replacement to allow removal of
  `setup.cfg`.
- Remove `tqdm`, as it is required by `datajoint`

Standard environment:
- Add comments for all dependencies specific to position environment to allow
  for easier comparison across yaml files.
- Remove `dask`, as it is required by `ghostipy`
- Remove `hdmf`, as it is required by `pynwb`
- Remove `pymysql`, as it is required by `datajoint`
- Remove `pyyaml`, as it is required by `sortingview`
- Remove `pydotplus` from `pip` section, as it is already in `conda` section

Position environment:
- Rename `_position` or `-position` to `_dlc` or `-dlc` to be more precise.
- Remove `ipython`, `numba`, and `tensorflow`, as they are all required by
  `deeplabcut`
- Add items present in standard environment but not in position environment.

* Add pybind11 for std env.yml

* Change dependencies. See details.

- `trajectory_analysis_tools` and `replay_trajectory_classification` are
  superceded by `non_local_detector`.
- `dask-cuda` not in use, per @edeno

* Fixes for running notebooks up to 41

* Further dependency adjustments following PR review @edeno

* Depedencies for panel

* LorenFrankLab#810

CHANGELOG.md

@@ -21,18 +21,14 @@
 - Update docs to reflect new notebooks. #776
 - Add overview of Spyglass to docs. #779
 - Update linting for Black 24. #808
+- Steamline dependency management. #822
 
 ### Pipelines
 
 - Spike sorting:
     - Add SpikeSorting V1 pipeline. #651
     - Move modules into spikesorting.v0 #807
     - Add MUA analysis to spike sorting pipeline
-- LFP: Minor fixes to LFPBandV1 populator and `make`. #706, #795
-
-### Pipelines
-
-- Spike sorting: Add SpikeSorting V1 pipeline. #651
 - LFP:
     - Minor fixes to LFPBandV1 populator and `make`. #706, #795
     - LFPV1: Fix error for multiple lfp settings on same data #775
@@ -53,7 +49,7 @@
     - Add fetch class functionality to `Merge` table. #783, #786
     - Add ability to filter sorted units in decoding #807
     - Rename SortedSpikesGroup.SortGroup to SortedSpikesGroup.Units #807
-    - Change methods with load_... to fetch_... for consistency #807
+    - Change methods with load\_... to fetch\_... for consistency #807
     - Use merge table methods to access part methods #807
 
 ## [0.4.3] (November 7, 2023)

environment.yml

@@ -1,38 +1,50 @@
+# 1. Install a conda distribution.
+#    https://mamba.readthedocs.io/en/latest/installation/mamba-installation.html
+# 2. Run: `mamba env create -f environment.yml`
+# 3. Activate: `conda activate spyglass`
+#
+# (lines intentionally left blank)
+#
+#
 name: spyglass
 channels:
   - conda-forge
   - defaults
+  # - pytorch # dlc-only
   - franklab
   - edeno
 dependencies:
-  - python>=3.9,<3.10
+  - bottleneck
+  # - cudatoolkit=11.3 # dlc-only
+  # - ffmpeg # dlc-only
+  - ipympl
   - jupyterlab>=3.*
-  - pydotplus
-  - dask
+  # - libgcc # dlc-only
+  - matplotlib
+  - non_local_detector
+  - numpy<1.24
   - pip
   - position_tools
-  - numpy<1.24
-  - track_linearization>=2.3
+  - pybind11  # req by mountainsort4 -> isosplit5
+  - pydotplus
+  - pyfftw<=0.12.0  # ghostipy req. install from conda-forge for Mac ARM
+  - python>=3.9,<3.10
+  - pytorch<1.12.0
   - ripple_detection
-  - non_local_detector
-  - matplotlib
   - seaborn
-  - bottleneck
-  - ipympl
-  - tqdm
-  - pyfftw<=0.12.0  # used by ghostipy. install from conda-forge so that it works on Mac ARM processors
+  # - torchaudio # dlc-only
+  # - torchvision # dlc-only
+  - track_linearization>=2.3
   - pip:
-      - pubnub<6.4.0
-      - spikeinterface>=0.98.2,<0.99
-      - pynwb>=2.2.0,<3
-      - hdmf>=3.4.6
+      - "black[jupyter]"
       - datajoint>=0.13.6
-      - ghostipy
-      - pymysql
+      # - deeplabcut<2.3.0 # dlc-only
+      - ghostipy # for common_filter
+      - ndx-franklab-novela>=0.1.0
+      - mountainsort4
+      - panel<=1.3.5 # See panel #6325
+      - pubnub<=6.4.0
+      - pynwb>=2.2.0,<3
       - sortingview>=0.11
-      - figurl-jupyter
-      - git+https://github.com/LorenFrankLab/ndx-franklab-novela.git
-      - pyyaml
-      - click
-      - "black[jupyter]"
+      - spikeinterface>=0.98.2,<0.99
       - .

environment_dlc.yml

@@ -0,0 +1,50 @@
+# 1. INSTALL CORRECT DRIVER for your GPU-equipped machine.
+#    see https://stackoverflow.com/questions/30820513/what-is-the-correct-version-of-cuda-for-my-nvidia-driver/30820690
+# 2. install: mamba env create -f environment_dlc.yml
+#    update existing install:  mamba env update -f environment_dlc.yml
+# 2. After installing...
+#    run: conda activate spyglass-dlc
+#    run: mamba env config vars set LD_LIBRARY_PATH=~/path/to/<anaconda3 or mambaforge>/envs/spyglass-position/lib/
+#    run: mamba install -c conda-forge wxpython
+name: spyglass-dlc
+channels:
+  - conda-forge
+  - defaults
+  - pytorch   # dlc-only
+  - franklab
+  - edeno
+dependencies:
+  - bottleneck
+  - cudatoolkit=11.3   # dlc-only
+  - ffmpeg   # dlc-only
+  - ipympl
+  - jupyterlab>=3.*
+  - libgcc   # dlc-only
+  - matplotlib
+  - non_local_detector
+  - numpy<1.24
+  - pip>=20.2.*
+  - position_tools
+  - pybind11  # req by mountainsort4 -> isosplit5
+  - pydotplus>=2.0.*
+  - pyfftw<=0.12.0  # ghostipy req. install from conda-forge for Mac ARM
+  - python>=3.9,<3.10
+  - pytorch<1.12.0
+  - ripple_detection
+  - seaborn
+  - torchaudio  # dlc-only
+  - torchvision  # dlc-only
+  - track_linearization>=2.3
+  - pip:
+      - "black[jupyter]"
+      - datajoint>=0.13.6
+      - deeplabcut<2.3.0   # dlc-only
+      - ghostipy # for common_filter
+      - ndx-franklab-novela>=0.1.0
+      - mountainsort4
+      - panel<=1.3.5 # See panel #6325
+      - pubnub<=6.4.0
+      - pynwb>=2.2.0,<3
+      - sortingview>=0.11
+      - spikeinterface>=0.98.2,<0.99
+      - .[dlc]

notebooks/01_Insert_Data.ipynb

@@ -945,8 +945,9 @@
    "metadata": {},
    "outputs": [],
    "source": [
+    "team_name = \"My Team\"\n",
     "sgc.LabTeam().create_new_team(\n",
-    "    team_name=\"My Team\",  # Should be unique\n",
+    "    team_name=team_name,  # Should be unique\n",
     "    team_members=[\"Firstname Lastname\", \"Firstname2 Lastname2\"],\n",
     "    team_description=\"test\",  # Optional\n",
     ")"
@@ -2252,6 +2253,69 @@
     "session_entry"
    ]
   },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "`Session.Experimenter` is used for permissions checks when deleting. The\n",
+    "session will need to have an experimenter in order avoid an error being\n",
+    "raised during this check.\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "sess_key = (sgc.Session & {\"nwb_file_name\": nwb_copy_file_name}).fetch(\n",
+    "    \"KEY\", as_dict=True\n",
+    ")[0]\n",
+    "exp_key = (sgc.LabMember).fetch(\"KEY\", as_dict=True)[0]\n",
+    "sgc.Session.Experimenter.insert1(\n",
+    "    dict(**sess_key, **exp_key), skip_duplicates=True\n",
+    ")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Even with the experimenter specified, there are still delete protections\n",
+    "in place. To see an example, uncomment the cell below.\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# session_entry.delete()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "To delete, you'll need to share a team with the session experimenter.\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "your_name = \"YourFirst YourLast\"\n",
+    "parts = your_name.split(\" \")\n",
+    "sgc.LabMember.insert1([your_name, parts[0], parts[1]])\n",
+    "sgc.LabMember.LabMemberInfo.insert1(\n",
+    "    [your_name, \"your_gmail\", dj.config[\"database.user\"], 0]\n",
+    ")\n",
+    "sgc.LabTeam.LabTeamMember.insert1([team_name, your_name])"
+   ]
+  },
   {
    "cell_type": "markdown",
    "metadata": {},
@@ -2311,7 +2375,7 @@
     }
    ],
    "source": [
-    "session_entry.delete()"
+    "# session_entry.delete()"
    ]
   },
   {