Skip to content

Commit

Permalink
Fetch upstream
Browse files Browse the repository at this point in the history
  • Loading branch information
CBroz1 committed Nov 14, 2023
2 parents c228b82 + d3b5c93 commit e12c865
Show file tree
Hide file tree
Showing 141 changed files with 29,689 additions and 21,719 deletions.
19 changes: 19 additions & 0 deletions .github/pull_request_template.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
# Description

Please include a summary of the changes and the related issue. Please also
include relevant motivation and context. Please list issues fixed or closed by
This PR.

- Fixes #000: How this PR fixes the issue
- `path/file.py`: Description of the change
- `path/file.py`: Description of the change
- Fixes #000: How this PR fixes the issue
- `path/file.py`: Description of the change
- `path/file.py`: Description of the change

# Checklist:

- [ ] This PR should be accompanied by a release: (yes/no/unsure)
- [ ] (If release) I have updated the `CITATION.cff`
- [ ] I have updated the `CHANGELOG.md`
- [ ] I have added/edited docs/notebooks to reflect the changes
12 changes: 8 additions & 4 deletions .github/workflows/test-package-build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -80,15 +80,19 @@ jobs:
# - name: Run tests
# run: pytest --doctest-modules -v --pyargs spyglass
publish:
name: Upload release to PyPI
runs-on: ubuntu-latest
needs: [test-package]
environment:
name: pypi
url: https://pypi.org/p/spyglass-neuro
permissions:
id-token: write # IMPORTANT: this permission is mandatory for trusted publishing
if: github.event_name == 'push' && startsWith(github.ref, 'refs/tags/')
steps:
- uses: actions/download-artifact@v3
with:
name: dist
path: dist/
- uses: pypa/gh-action-pypi-publish@release/v1
with:
user: __token__
password: ${{ secrets.PYPI_API_TOKEN }}
- name: Publish package distributions to PyPI
uses: pypa/gh-action-pypi-publish@release/v1
23 changes: 4 additions & 19 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -6,23 +6,7 @@ mysql_config

# Notebooks
*.ipynb

!notebooks/00_intro.ipynb
!notebooks/01_spikesorting.ipynb
!notebooks/02_curation.ipynb
!notebooks/03_lfp.ipynb
!notebooks/04_Trodes_position.ipynb
!notebooks/05_DLC_from_scratch.ipynb
!notebooks/06_DLC_from_dir.ipynb
!notebooks/4_position_info.ipynb
!notebooks/07_linearization.ipynb
!notebooks/08_Extract_Mark_indicators.ipynb
!notebooks/09_Decoding_with_GPUs_on_the_GPU_cluster.ipynb
!notebooks/10_1D_Clusterless_Decoding.ipynb
!notebooks/11_2D_Clusterless_Decoding.ipynb
!notebooks/12_Ripple_Detection.ipynb
!notebooks/13_Theta_phase_and_power.ipynb

!notebooks/[0-9]*.ipynb

# Byte-compiled / optimized / DLL files
__pycache__/
Expand Down Expand Up @@ -159,7 +143,6 @@ dmypy.json
temp_nwb/*s

*.mp4
dj_local_conf.json
*.png
*.npy
*.nc
Expand All @@ -168,6 +151,8 @@ dj_local_conf.json
*.json
*.gz
*.pdf
dj_local_conf*
!dj_local_conf_example.json

!/.vscode/extensions.json
!/.vscode/settings.json
Expand All @@ -183,4 +168,4 @@ docs/src/notebooks/
temp*

# Version
_version.py
_version.py
71 changes: 71 additions & 0 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,71 @@
default_stages: [commit, push]
exclude: (^.github/|^docs/site/|^images/)

repos:
- repo: https://github.com/executablebooks/mdformat
# Do this before other tools "fixing" the line endings
rev: 0.7.16
hooks:
- id: mdformat
name: Format Markdown
entry: mdformat # Executable to run, with fixed options
language: python
types: [markdown]
args: [--wrap, "80", --number]
additional_dependencies:
- mdformat-toc
- mdformat-beautysh
- mdformat-config
- mdformat-black
- mdformat-web
- mdformat-gfm

- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v4.4.0
hooks:
- id: check-json
- id: check-toml
- id: check-yaml
args: [--unsafe]
- id: requirements-txt-fixer
- id: end-of-file-fixer
- id: mixed-line-ending
args: ["--fix=lf"]
description: Forces to replace line ending by the UNIX 'lf' character.
- id: trailing-whitespace
- id: debug-statements
- id: check-added-large-files # prevent giant files from being committed
- id: check-builtin-literals
- id: check-merge-conflict
- id: check-executables-have-shebangs
- id: check-shebang-scripts-are-executable
- id: check-docstring-first
- id: check-case-conflict
- id: fix-byte-order-marker

- repo: https://github.com/adrienverge/yamllint.git
rev: v1.29.0
hooks:
- id: yamllint
args:
- --no-warnings
- -d
- "{extends: relaxed, rules: {line-length: {max: 90}}}"

- repo: https://github.com/charliermarsh/ruff-pre-commit
rev: v0.0.254
hooks:
- id: ruff

- repo: https://github.com/PyCQA/autoflake
rev: v2.0.1
hooks:
- id: autoflake

- repo: https://github.com/codespell-project/codespell
rev: v2.2.2
hooks:
- id: codespell
args: [--toml, pyproject.toml]
additional_dependencies:
- tomli
11 changes: 9 additions & 2 deletions .vscode/settings.json
Original file line number Diff line number Diff line change
Expand Up @@ -4,11 +4,18 @@
"files.trimFinalNewlines": true,
"editor.multiCursorModifier": "ctrlCmd",
"autoDocstring.docstringFormat": "numpy",
"python.formatting.provider": "none",
"remote.SSH.remoteServerListenOnSocket": true,
"git.confirmSync": false,
"python.analysis.typeCheckingMode": "off",
"[python]": {
"editor.defaultFormatter": "ms-python.black-formatter"
"editor.defaultFormatter": "ms-python.black-formatter",
"editor.formatOnSave": true,
"editor.codeActionsOnSave": {
"source.organizeImports": true
},
},
"isort.args": [
"--profile",
"black"
],
}
40 changes: 40 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,42 @@
# Change Log

## [0.4.4] (Unreleased)

- Support multi-cam projects in DeepLabCut.

## [0.4.3] (November 7, 2023)

- Migrate `config` helper scripts to Spyglass codebase. #662
- Revise contribution guidelines. #655
- Minor bug fixes. #656, #657, #659, #651, #671
- Add setup instruction specificity.
- Reduce primary key varchar allocation aross may tables. #664

## [0.4.2] (October 10, 2023)

### Infrastructure / Support

- Bumped Python version to 3.9. #583
- Updated user management helper scripts for MySQL 8. #650
- Centralized config/path handling to permit setting via datajoint config. #593
- Fixed Merge Table deletes: error specificity and transaction context. #617

### Pipelines

- Common:
- Added support multiple cameras per epoch. #557
- Removed `common_backup` schema. #631
- Added support for multiple position objects per NWB in `common_behav` via
PositionSource.SpatialSeries and RawPosition.PosObject #628, #616.
_Note:_ Existing functions have been made compatible, but column labels for
`RawPosition.fetch1_dataframe` may change.
- Spike sorting:
- Added pipeline populator. #637, #646, #647
- Fixed curation functionality for `nn_isolation`. #597, #598
- Position: Added position interval/epoch mapping via PositionIntervalMap. #620,
#621, #627
- LFP: Refactored pipeline. #594, #588, #605, #606, #607, #608, #615, #629

## [0.4.1] (June 30, 2023)

- Add mkdocs automated deployment. #527, #537, #549, #551
Expand Down Expand Up @@ -62,6 +99,9 @@
- Allow creation and linkage of device metadata from YAML #400
- Move helper functions to utils directory #386

[0.4.4]: https://github.com/LorenFrankLab/spyglass/releases/tag/0.4.4
[0.4.3]: https://github.com/LorenFrankLab/spyglass/releases/tag/0.4.3
[0.4.2]: https://github.com/LorenFrankLab/spyglass/releases/tag/0.4.2
[0.4.1]: https://github.com/LorenFrankLab/spyglass/releases/tag/0.4.1
[0.4.0]: https://github.com/LorenFrankLab/spyglass/releases/tag/0.4.0
[0.3.4]: https://github.com/LorenFrankLab/spyglass/releases/tag/0.3.4
Expand Down
4 changes: 2 additions & 2 deletions CITATION.cff
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,6 @@ authors:
- family-names: "Frank"
given-names: "Loren"
title: "spyglass"
version: 0.4.0
date-released: 2023-05-22
version: 0.4.3
date-released: 2023-11-07
url: "https://github.com/LorenFrankLab/spyglass"
34 changes: 7 additions & 27 deletions config/add_dj_collaborator.py
Original file line number Diff line number Diff line change
@@ -1,32 +1,12 @@
#!/usr/bin/env python
import os
import sys
import tempfile

# shared_modules = [
# "common\_%",
# "spikesorting\_%",
# "decoding\_%",
# "position\_%",
# "lfp\_%",
# ]


def add_collab_user(user_name):
# create a tempoary file for the command
file = tempfile.NamedTemporaryFile(mode="w")

file.write(
f"GRANT ALL PRIVILEGES ON `{user_name}\_%`.* TO `{user_name}`@'%' IDENTIFIED BY 'temppass';\n"
)
# for module in shared_modules:
# file.write(f"GRANT ALL PRIVILEGES ON `{module}`.* TO `{user_name}`@'%';\n")
file.write(f"GRANT SELECT ON `%`.* TO `{user_name}`@'%';\n")
file.flush()

# run those commands in sql
os.system(f"mysql -p -h lmf-db.cin.ucsf.edu < {file.name}")
from warnings import warn

from spyglass.utils.database_settings import DatabaseSettings

if __name__ == "__main__":
add_collab_user(sys.argv[1])
warn(
"This script is deprecated. "
+ "Use spyglass.utils.database_settings.DatabaseSettings instead."
)
DatabaseSettings(user_name=sys.argv[1]).add_collab_user()
31 changes: 7 additions & 24 deletions config/add_dj_guest.py
Original file line number Diff line number Diff line change
@@ -1,29 +1,12 @@
#!/usr/bin/env python
import os
import sys
import tempfile

shared_modules = [
"common\_%",
"spikesorting\_%",
"decoding\_%",
"position\_%",
"lfp\_%",
]


def add_user(user_name):
# create a tempoary file for the command
file = tempfile.NamedTemporaryFile(mode="w")

file.write(
f"GRANT SELECT ON `%`.* TO `{user_name}`@'%' IDENTIFIED BY 'Data_$haring';\n"
)
file.flush()

# run that commands in sql
os.system(f"mysql -p -h lmf-db.cin.ucsf.edu < {file.name}")
from warnings import warn

from spyglass.utils.database_settings import DatabaseSettings

if __name__ == "__main__":
add_user(sys.argv[1])
warn(
"This script is deprecated. "
+ "Use spyglass.utils.database_settings.DatabaseSettings instead."
)
DatabaseSettings(user_name=sys.argv[1]).add_dj_guest()
36 changes: 7 additions & 29 deletions config/add_dj_module.py
Original file line number Diff line number Diff line change
@@ -1,34 +1,12 @@
#!/usr/bin/env python
import grp
import os
import sys
import tempfile

target_group = "kachery-users"


def add_module(module_name):
print(f"Granting everyone permissions to module {module_name}")

# create a tempoary file for the command
file = tempfile.NamedTemporaryFile(mode="w")

# find the kachery-users group
groups = grp.getgrall()
for group in groups:
if group.gr_name == target_group:
break

# get a list of usernames
for user in group.gr_mem:
file.write(
f"GRANT ALL PRIVILEGES ON `{module_name}\_%`.* TO `{user}`@'%';\n"
)
file.flush()

# run those commands in sql
os.system(f"mysql -p -h lmf-db.cin.ucsf.edu < {file.name}")
from warnings import warn

from spyglass.utils.database_settings import DatabaseSettings

if __name__ == "__main__":
add_module(sys.argv[1])
warn(
"This script is deprecated. "
+ "Use spyglass.utils.database_settings.DatabaseSettings instead."
)
DatabaseSettings().add_module(sys.argv[1])
Loading

0 comments on commit e12c865

Please sign in to comment.