Merge branch 'dev' into electrodes_table

.github/PULL_REQUEST_TEMPLATE.md

@@ -6,6 +6,8 @@
 
 For all schema changes:
 - [ ] Add release notes for the PR to `docs/format/source/format_release_notes.rst`.
+- [ ] Have you included the relevant issue number using "Fix #XXX" notation where XXX is the issue number? By including "Fix #XXX" you allow GitHub to close issue #XXX when the PR is merged.
+- [ ] Make sure that `hdmf-common-schema` points to the latest release and not the latest commit on the `main` branch.
 
 If this is the first schema change after a schema release (i.e., the version string in `core/nwb.namespace.yaml` does not
 end in "-alpha"), then:

.github/workflows/check_sphinx_links.yml

@@ -0,0 +1,34 @@
+name: Check Sphinx external links and references
+on:
+  pull_request:
+  schedule:
+    - cron: '0 5 * * *'  # once per day at midnight ET
+  workflow_dispatch:
+
+jobs:
+  check-external-links:
+    runs-on: ubuntu-latest
+    steps:
+      - name: Cancel non-latest runs
+        uses: styfle/[email protected]
+        with:
+          all_but_latest: true
+          access_token: ${{ github.token }}
+
+      - uses: actions/checkout@v4
+        with:
+          submodules: 'recursive'
+          fetch-depth: 0  # tags are required for versioneer to determine the version
+
+      - name: Set up Python
+        uses: actions/setup-python@v4
+        with:
+          python-version: '3.12'
+
+      - name: Install Sphinx dependencies and package
+        run: |
+          python -m pip install --upgrade pip
+          python -m pip install -r requirements-doc.txt
+
+      - name: Check Sphinx links and references
+        run: sphinx-build -n -b linkcheck ./docs/format/source ./test_build

.github/workflows/codespell.yml

@@ -14,6 +14,6 @@ jobs:
 
     steps:
       - name: Checkout
-        uses: actions/checkout@v3
+        uses: actions/checkout@v4
       - name: Codespell
-        uses: codespell-project/actions-codespell@v1
+        uses: codespell-project/actions-codespell@v2

.github/workflows/validate_schema.yml

@@ -8,11 +8,11 @@ jobs:
     if: github.event_name != 'pull_request' || github.event.pull_request.head.repo.full_name != github.event.pull_request.base.repo.full_name
     runs-on: ubuntu-latest
     steps:
-    - uses: actions/checkout@v3
-    - name: Set up Python 3.10
+    - uses: actions/checkout@v4
+    - name: Set up Python 3.12
       uses: actions/setup-python@v4
       with:
-        python-version: "3.10"
+        python-version: "3.12"
     - name: Install dev branch of HDMF
       run: |
         pip install git+https://github.com/hdmf-dev/hdmf.git

.gitignore

@@ -5,9 +5,7 @@ __pycache__/
 
 # Sphinx documentation
 docs/format/_build/
-docs/language/_build/
 docs/storage/_build/
-docs/general/_build
 
 # Autogenerated Sphinx sources
 docs/format/source/_format_auto_docs/
@@ -17,3 +15,5 @@ docs/format/source/_format_auto_docs/
 
 #PyCharm
 .idea/
+
+.DS_Store

Legal.txt

@@ -1,4 +1,4 @@
-“nwb-schema” Copyright (c) 2017-2023, The Regents of the University of California, through Lawrence Berkeley National Laboratory (subject to receipt of any required approvals from the U.S. Dept. of Energy).  All rights reserved.
+“nwb-schema” Copyright (c) 2017-2024, The Regents of the University of California, through Lawrence Berkeley National Laboratory (subject to receipt of any required approvals from the U.S. Dept. of Energy).  All rights reserved.
 
 If you have questions about your rights to use or distribute this software, please contact Berkeley Lab's Innovation & Partnerships Office at [email protected].
 

README.rst

@@ -1,9 +1,9 @@
 NWB Schema Format
-========================
+=================
 
 A format specification schema for the Neurodata Without Borders (NWB) data format.
 
-To get started using NWB, please go to the [NWB overview website](https://nwb-overview.readthedocs.io/en/latest/).
+To get started using NWB, please go to the `NWB overview website <https://nwb-overview.readthedocs.io/en/latest/>`_.
 
 This repo contains:
 
@@ -27,7 +27,7 @@ The NWB 1.0 format and API are archived in the [NeurodataWithoutBorders/api-pyth
 License
 ========================
 
-“nwb-schema” Copyright (c) 2017-2023, The Regents of the University of California, through Lawrence Berkeley National Laboratory (subject to receipt of any required approvals from the U.S. Dept. of Energy).  All rights reserved.
+“nwb-schema” Copyright (c) 2017-2024, The Regents of the University of California, through Lawrence Berkeley National Laboratory (subject to receipt of any required approvals from the U.S. Dept. of Energy).  All rights reserved.
 
 Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:
 
@@ -45,7 +45,7 @@ You are under no obligation whatsoever to provide any bug fixes, patches, or upg
 Copyright
 ========================
 
-“nwb-schema” Copyright (c) 2017-2023, The Regents of the University of California, through Lawrence Berkeley National Laboratory (subject to receipt of any required approvals from the U.S. Dept. of Energy).  All rights reserved.
+“nwb-schema” Copyright (c) 2017-2024, The Regents of the University of California, through Lawrence Berkeley National Laboratory (subject to receipt of any required approvals from the U.S. Dept. of Energy).  All rights reserved.
 
 If you have questions about your rights to use or distribute this software, please contact Berkeley Lab's Innovation & Partnerships Office at [email protected].
 

core/nwb.behavior.yaml

@@ -38,6 +38,24 @@ groups:
         is 'meters'. Actual stored values are not necessarily stored in these units.
         To access the data in these units, multiply 'data' by 'conversion' and add 'offset'.
       required: false
+    - name: bounds
+      dtype: numeric
+      doc: The boundary range (min, max) for each dimension of `data`. The units are the same as the units for the data.
+      shape:
+      - - 1
+        - 2
+      - - 2
+        - 2
+      - - 3
+        - 2 
+      dims:
+      - - x
+        - min,max
+      - - x,y
+        - min,max
+      - - x,y,z
+        - min,max
+      required: false
   - name: reference_frame
     dtype: text
     doc: Description defining what exactly 'straight-ahead' means.

core/nwb.file.yaml

@@ -7,7 +7,7 @@ groups:
   attributes:
   - name: nwb_version
     dtype: text
-    value: "2.6.0"
+    value: "2.7.0"
     doc: File version string. Use semantic versioning, e.g. 1.2.1. This will be the
       name of the format with trailing major, minor and patch numbers.
   datasets:
@@ -131,6 +131,16 @@ groups:
       - neurodata_type_inc: TimeSeries
         doc: TimeSeries objects containing data of presented stimuli.
         quantity: '*'
+      - neurodata_type_inc: NWBDataInterface
+        doc: 'Generic NWB data interfaces, usually from an extension,
+          containing data of presented stimuli.'
+        quantity: '*'
+      - neurodata_type_inc: DynamicTable
+        doc: DynamicTable objects containing data of presented stimuli.
+        quantity: '*'
+      # even though TimeSeries is a child type of NWBDataInterface, we do not remove TimeSeries
+      # in order to maintain backwards compatibility in the APIs that
+      # use the neurodata_type_inc from the schema to set the variable name
     - name: templates
       doc: 'Template stimuli. Timestamps in templates are based on stimulus
         design and are relative to the beginning of the stimulus. When templates are

core/nwb.icephys.yaml

@@ -286,6 +286,10 @@ groups:
   - name: stimulus
     neurodata_type_inc: TimeSeriesReferenceVectorData
     doc: Column storing the reference to the recorded stimulus for the recording (rows).
+  - name: stimulus_template
+    neurodata_type_inc: TimeSeriesReferenceVectorData
+    doc: Column storing the reference to the stimulus template for the recording (rows).
+    quantity: '?'
 
 - neurodata_type_def: IntracellularResponsesTable
   neurodata_type_inc: DynamicTable

core/nwb.namespace.yaml

@@ -57,4 +57,4 @@ namespaces:
   - doc: This source module contains neurodata_type for retinotopy data.
     source: nwb.retinotopy.yaml
     title: Retinotopy
-  version: "2.6.0-alpha"
+  version: "2.8.0-alpha"

core/nwb.ogen.yaml

@@ -6,10 +6,16 @@ groups:
   - name: data
     dtype: numeric
     dims:
-    - num_times
+    - - num_times
+    - - num_times
+      - num_rois
     shape:
-    - null
-    doc: Applied power for optogenetic stimulus, in watts.
+    - - null
+    - - null
+      - null
+    doc: Applied power for optogenetic stimulus, in watts. Shape can be 1D or 2D.
+      2D data is meant to be used in an extension of OptogeneticSeries that
+      defines what the second dimension represents.
     attributes:
     - name: unit
       dtype: text

core/nwb.ophys.yaml

@@ -340,12 +340,12 @@ groups:
     each point in time is assumed to be 2-D (has only x & y dimensions).'
   groups:
   - neurodata_type_inc: CorrectedImageStack
-    doc: Reuslts from motion correction of an image stack.
+    doc: Results from motion correction of an image stack.
     quantity: '+'
 
 - neurodata_type_def: CorrectedImageStack
   neurodata_type_inc: NWBDataInterface
-  doc: Reuslts from motion correction of an image stack.
+  doc: Results from motion correction of an image stack.
   groups:
   - name: corrected
     neurodata_type_inc: ImageSeries

core/nwb.retinotopy.yaml

@@ -2,9 +2,9 @@ groups:
 - neurodata_type_def: ImagingRetinotopy
   neurodata_type_inc: NWBDataInterface
   default_name: ImagingRetinotopy
-  doc: 'Intrinsic signal optical imaging or widefield imaging for measuring retinotopy.
-    Stores orthogonal maps (e.g., altitude/azimuth; radius/theta) of responses to
-    specific stimuli and a combined polarity map from which to identify visual areas.
+  doc: 'DEPRECATED. Intrinsic signal optical imaging or widefield imaging for measuring
+    retinotopy. Stores orthogonal maps (e.g., altitude/azimuth; radius/theta) of responses
+    to specific stimuli and a combined polarity map from which to identify visual areas.
     This group does not store the raw responses imaged during retinotopic mapping or the
     stimuli presented, but rather the resulting phase and power maps after applying a Fourier
     transform on the averaged responses.

docs/format/.readthedocs.yaml

@@ -0,0 +1,27 @@
+# Read the Docs configuration file for https://nwb-schema.readthedocs.io/
+# See https://docs.readthedocs.io/en/stable/config-file/v2.html for details
+
+# Required
+version: 2
+
+build:
+  os: ubuntu-22.04
+  tools:
+    python: "3.12"
+
+# Build documentation in the docs/ directory with Sphinx
+sphinx:
+  configuration: docs/format/source/conf.py
+
+# Optionally build your docs in additional formats such as PDF and ePub
+formats: all
+
+# Optionally set the version of Python and requirements required to build your docs
+python:
+  install:
+    - requirements: requirements-doc.txt
+
+# Optionally include all submodules
+submodules:
+  include: all
+  recursive: true

docs/format/Makefile

@@ -159,7 +159,7 @@ changes:
 	@echo "The overview file is in $(BUILDDIR)/changes."
 
 linkcheck:
-	$(SPHINXBUILD) -b linkcheck $(ALLSPHINXOPTS) $(BUILDDIR)/linkcheck
+	$(SPHINXBUILD) -n -b linkcheck $(ALLSPHINXOPTS) $(BUILDDIR)/linkcheck
 	@echo
 	@echo "Link check complete; look for any errors in the above output " \
 	      "or in $(BUILDDIR)/linkcheck/output.txt."

docs/format/make.bat

@@ -185,7 +185,7 @@ if "%1" == "changes" (
 )
 
 if "%1" == "linkcheck" (
-	%SPHINXBUILD% -b linkcheck %ALLSPHINXOPTS% %BUILDDIR%/linkcheck
+	%SPHINXBUILD% -n -b linkcheck %ALLSPHINXOPTS% %BUILDDIR%/linkcheck
 	if errorlevel 1 exit /b 1
 	echo.
 	echo.Link check complete; look for any errors in the above output ^

docs/format/source/conf.py

@@ -76,16 +76,16 @@ def setup(app):
 
 # General information about the project.
 project = u'NWB Format Specification'
-copyright = u'2017-2023, Neurodata Without Borders'
+copyright = u'2017-2024, Neurodata Without Borders'
 
 # The version info for the project you're documenting, acts as replacement for
 # |version| and |release|, also used in various other places throughout the
 # built documents.
 #
 # The short X.Y version.
-version = '2.6.0'
+version = '2.8.0-alpha'
 # The full version, including alpha/beta/rc tags.
-release = '2.6.0'
+release = '2.8.0-alpha'
 
 # The language for content autogenerated by Sphinx. Refer to documentation
 # for a list of supported languages.

docs/format/source/credits.rst

@@ -18,11 +18,11 @@ NWB: Version 2.0.0 and later
 Documentation for Version 2 of the NWB format and later have been created by
 Oliver Ruebel and Andrew Tritt et al. in collaboration with the NWB community.
 
-NWB:N: Version 1.0.x and earlier
+NWB: Version 1.0.x and earlier
 --------------------------------
 
 Version 1.0.5g (and earlier) of the NWB file format were created by Jeff Teeters et al. as
-part of the first NWB pilot project. The documents for NWB:N 2 have been adopted from the
+part of the first NWB pilot project. The documents for NWB 2 have been adopted from the
 final version of format docs released by the original NWB pilot project.
 
 

docs/format/source/format_description.rst

@@ -19,7 +19,7 @@ The NWB format uses the following main primitives to hierarchically organize neu
 
 The NWB format is formally described via formal specification documents using the `NWB specification language <http://schema-language.readthedocs.io/en/latest/>`_ .
 HDF5 currently serves as the main format for storing data in the NWB format (see http://nwb-storage.readthedocs.io/en/latest/ for details).
-The `PyNWB <http://pynwb.readthedocs.io/en/latest/>`_ API is available to enable users to efficiently interact with NWB format files.
+The `PyNWB <http://pynwb.readthedocs.io/en/stable/>`_ API is available to enable users to efficiently interact with NWB format files.
 
 The NWB format uses a modular design in which all main semantic components of the format have
 a unique *neurodata_type* (similar to a Class in object-oriented design)(:numref:`sec_neurodata_type_intro`).
@@ -172,7 +172,7 @@ To ease development of extensions, the PyNWB_
 (and `HDMF <https://github.com/hdmf-dev/hdmf>`_ used by PyNWB) API provides dedicated
 data structures that support programmatic creation and use of extensions. An
 example for extending NWB using PyNWB_ is available at
-https://pynwb.readthedocs.io/en/stable/extensions.html and
+https://pynwb.readthedocs.io/en/stable/tutorials/general/extensions.html and
 additional details are also available as part of the PyNWB_ tutorials
 https://pynwb.readthedocs.io/en/stable/tutorials/index.html .