Merge pull request #29 from datarootsio/assert-docs-and-change-deps

Assert docs and change deps

.github/codecov.yml

@@ -25,4 +25,4 @@ parsers:
 comment:
   layout: "reach,diff,flags,tree"
   behavior: default
-  require_changes: no
+  require_changes: no

.github/workflows/publish.yml

@@ -98,4 +98,4 @@ jobs:
       - name: Deploy docs
         run: |
           poetry run mike deploy --update-aliases ${{ needs.get-new-tag.outputs.tag-id }} latest
-          poetry run mike set-default --push latest
+          poetry run mike set-default --push latest

.github/workflows/tests.yml

@@ -20,10 +20,7 @@ jobs:
           poetry install --no-interaction --no-ansi
       - name: QA with flake8, black, isort and mypy
         run: |
-          black . --check
-          isort . --check
-          flake8 .
-          mypy
+          pre-commit run --all-files
       - name: Configure git user for git unit tests
         run: |
           git config --global user.name 'github-actions[bot]'

.gitignore

@@ -129,4 +129,4 @@ dmypy.json
 .pyre/
 
 # IDE files
-.idea/
+.idea/

.pre-commit-config.yaml

@@ -0,0 +1,55 @@
+default_language_version:
+    python: python3
+repos:
+  - repo: https://github.com/pre-commit/pre-commit-hooks
+    rev: v4.1.0
+    hooks:
+      - id: end-of-file-fixer
+      - id: trailing-whitespace
+  - repo: https://github.com/psf/black
+    rev: 22.1.0
+    hooks:
+      - id: black
+        files: (databooks/|tests/)
+  - repo: https://github.com/pycqa/isort
+    rev: 5.10.1
+    hooks:
+      - id: isort
+        files: (databooks/|tests/)
+  - repo: https://github.com/pycqa/flake8
+    rev: 4.0.1
+    hooks:
+      - id: flake8
+        files: (databooks/|tests/)
+        additional_dependencies: [flake8-docstrings==1.6.0]
+  - repo: https://github.com/pre-commit/mirrors-mypy
+    rev: v0.931
+    hooks:
+      - id: mypy
+        files: databooks/
+        additional_dependencies:
+          - types-all==1.0.0
+          - typer==0.3.2
+          - rich==10.16.2
+          - pydantic==1.9.0
+          - GitPython==3.1.26
+          - typing-extensions==4.0.1
+          - tomli==2.0.1
+  - repo: local
+    hooks:
+      - id: update-docs
+        name: update-docs
+        entry: cog -r
+        language: python
+        files: (.*\.md)
+        args: ["**/*.md"]
+        always_run: true
+        additional_dependencies:
+          - cogapp==3.3.0
+          - typer-cli==0.0.12
+          - typer==0.3.2
+          - rich==10.16.2
+          - pydantic==1.9.0
+          - GitPython==3.1.26
+          - typing-extensions==4.0.1
+          - tomli==2.0.1

.pre-commit-hooks.yaml

@@ -1,8 +1,16 @@
-- id: databooks
-  name: databooks
+- id: databooks-meta
+  name: databooks-meta
   description:
-    "Remove notebook metadata using `databooks`."
-  entry: databooks meta
+    "Remove Jupyter notebook metadata using `databooks`."
+  entry: databooks meta --overwrite
   language: python
   minimum_pre_commit_version: 2.9.2
-  types: [jupyter]
+  types: [jupyter]
+- id: databooks-assert
+  name: databooks-assert
+  description:
+    "Assert Jupyter notebook metadata values using `databooks`."
+  entry: databooks assert --expr "True"
+  language: python
+  minimum_pre_commit_version: 2.9.2
+  types: [ jupyter ]

CONTRIBUTING.md

@@ -73,6 +73,7 @@ As for documentation, the `databooks` documentation "lives" both on the code its
 supporting documentation (markdown) files.
 
 ### Code
+
 Code docs include annotating [type hints](#mypy) as well as function docstrings. For
 that, we use the [reStructuredText](https://www.python.org/dev/peps/pep-0287/) -like
 format.
@@ -94,9 +95,21 @@ mkdocs serve
 ```
 
 ### mike
+
 We also use the [`mike`](https://github.com/jimporter/mike) plugin in `MkDocs` to publish
 and keep different versions of documentation.
 
+### Cog
+
+We use [`cog`](https://nedbatchelder.com/code/cog) to dynamically generate parts of the
+documentation. That way, code changes trigger markdown changes as well.
+
+### Pre-commit
+
+[`Pre-commit`](https://pre-commit.com/) is the tool that automates everything, eases the
+workflow and run checks in CI/CD. It's highly recommended installing `pre-commit` and the
+hooks during development.
+
 ## Tests 🗳
 
 We use unit tests to ensure that our package works as expected. We use
@@ -132,4 +145,4 @@ maintained by [dataroots](https://github.com/datarootsio).
 Special thanks to:
 
 [Bart](https://github.com/Bart6114), [Nick](https://github.com/NickSchouten) and
-[Freddy](https://github.com/frederikdesmedt) for feedback and support.
+[Freddy](https://github.com/frederikdesmedt) for feedback and support.

LICENSE

@@ -18,4 +18,4 @@ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
 AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
 LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
 OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
-SOFTWARE.
+SOFTWARE.

README.md

@@ -46,7 +46,7 @@ pip install databooks
 
 ### Clear metadata
 
-Simply specify the paths for notebook files to remove metadata. By doing so, we can 
+Simply specify the paths for notebook files to remove metadata. By doing so, we can
 already avoid many of the conflicts.
 
 ```console
@@ -66,6 +66,21 @@ $ databooks fix [OPTIONS] PATHS...
 
 ![databooks fix demo](https://raw.githubusercontent.com/datarootsio/databooks/main/docs/images/databooks-fix.gif)
 
+### Assert notebook metadata
+
+Specify paths of notebooks to be checked, an expression or recipe of what you'd like to
+enforce. `databooks` will run your checks and raise errors if any notebook does not
+comply with the desired metadata values. This advanced feature allows users to enforce
+cell tags, sequential cell execution, maximum number of cells, among many other things!
+
+Check out our [docs](https://databooks.dev/latest/usage/overview/#databooks-assert) for more!
+
+```console
+$ databooks assert [OPTIONS] PATHS...
+```
+
+![databooks assert demo](https://raw.githubusercontent.com/datarootsio/databooks/main/docs/images/databooks-assert.gif)
+
 ## License
 
 This project is licensed under the terms of the MIT license.

databooks/cli.py

@@ -56,18 +56,19 @@ def _config_callback(ctx: Context, config_path: Optional[Path]) -> Optional[Path
     )
     logger.debug(f"Loading config file from: {config_path}")
 
-    ctx.default_map = ctx.default_map or {}  # initialize defaults
-
     if config_path is not None:  # config may not be specified
-        with config_path.open("r") as f:
+        with config_path.open("rb") as f:
             conf = (
                 tomli.load(f)
                 .get("tool", {})
                 .get("databooks", {})
                 .get(ctx.command.name, {})
             )
         # Merge configuration
-        ctx.default_map.update({k.replace("-", "_"): v for k, v in conf.items()})
+        ctx.default_map = {
+            **(ctx.default_map or {}),
+            **{k.replace("-", "_"): v for k, v in conf.items()},
+        }
     return config_path
 
 
@@ -128,7 +129,11 @@ def meta(
         help="Get CLI options from configuration file",
     ),
     help: Optional[bool] = Option(
-        None, is_eager=True, callback=_help_callback, help="Show this message and exit"
+        None,
+        "--help",
+        is_eager=True,
+        callback=_help_callback,
+        help="Show this message and exit",
     ),
 ) -> None:
     """Clear both notebook and cell metadata."""
@@ -186,8 +191,16 @@ def meta(
 def affirm_meta(
     paths: List[Path] = Argument(..., is_eager=True, help="Path(s) of notebook files"),
     ignore: List[str] = Option(["!*"], help="Glob expression(s) of files to ignore"),
-    expr: List[str] = Option((), help="Expressions to assert on notebooks"),
-    recipe: List[Recipe] = Option((), help="Common recipes of expressions"),
+    expr: List[str] = Option(
+        (), "--expr", "-x", help="Expressions to assert on notebooks"
+    ),
+    recipe: List[Recipe] = Option(
+        (),
+        "--recipe",
+        "-r",
+        help="Common recipes of expressions - see"
+        " https://databooks.dev/0.1.15/usage/overview/#recipes",
+    ),
     verbose: bool = Option(
         False, "--verbose", "-v", help="Log processed files in console"
     ),
@@ -202,7 +215,11 @@ def affirm_meta(
         help="Get CLI options from configuration file",
     ),
     help: Optional[bool] = Option(
-        None, is_eager=True, callback=_help_callback, help="Show this message and exit"
+        None,
+        "--help",
+        is_eager=True,
+        callback=_help_callback,
+        help="Show this message and exit",
     ),
 ) -> None:
     """
@@ -285,7 +302,11 @@ def fix(
         help="Get CLI options from configuration file",
     ),
     help: Optional[bool] = Option(
-        None, is_eager=True, callback=_help_callback, help="Show this message and exit"
+        None,
+        "--help",
+        is_eager=True,
+        callback=_help_callback,
+        help="Show this message and exit",
     ),
 ) -> None:
     """

docs/API.md

@@ -18,4 +18,4 @@ notebook = JupyterNotebook.parse_file(path="path/to/your/notebook")
 assert list(dict(notebook).keys()) == ['nbformat', 'nbformat_minor', 'metadata', 'cells']
 ```
 
-For more information on the internal workings and what is implemented, see [how it works]().
+For more information on the internal workings and what is implemented, see [how it works]().

docs/CLI.md

@@ -2,7 +2,29 @@
 hide:
   - navigation
 ---
-
+<!-- [[[cog
+import subprocess
+
+import cog
+from typer_cli.main import app
+
+result = subprocess.run(
+    [
+        "python",
+        "-m",
+        "typer_cli",
+        "databooks.cli",
+        "utils",
+        "docs",
+        "--name",
+        "databooks",
+    ],
+    stdout=subprocess.PIPE,
+    stderr=subprocess.PIPE,
+    encoding="utf-8",
+)
+cog.out(result.stdout)
+]]] -->
 # `databooks`
 
 CLI tool to resolve git conflicts and remove metadata in notebooks.
@@ -50,11 +72,11 @@ $ databooks assert [OPTIONS] PATHS...
 **Options**:
 
 * `--ignore TEXT`: Glob expression(s) of files to ignore  [default: !*]
-* `--check-expr TEXT`: Expressions to assert on notebooks  [default: ]
-* `--recipe [has-tags-code|max-cells|no-empty-code|seq-exec|seq-increase|startswith-md]`: Common recipes of expressions  [default: ]
+* `-x, --expr TEXT`: Expressions to assert on notebooks  [default: ]
+* `-r, --recipe [has-tags|has-tags-code|max-cells|no-empty-code|seq-exec|seq-increase|startswith-md]`: Common recipes of expressions - see https://databooks.dev/0.1.15/usage/overview/#recipes  [default: ]
 * `-v, --verbose`: Log processed files in console  [default: False]
 * `-c, --config PATH`: Get CLI options from configuration file
-* `--help / --no-help`: Show this message and exit
+* `--help`: Show this message and exit
 
 ## `databooks diff`
 
@@ -97,7 +119,7 @@ $ databooks fix [OPTIONS] PATHS...
 * `-i, --interactive`: Interactively resolve the conflicts (not implemented)  [default: False]
 * `--verbose / --no-verbose`: Log processed files in console  [default: False]
 * `-c, --config PATH`: Get CLI options from configuration file
-* `--help / --no-help`: Show this message and exit
+* `--help`: Show this message and exit
 
 ## `databooks meta`
 
@@ -127,4 +149,6 @@ $ databooks meta [OPTIONS] PATHS...
 * `--check`: Don't write files but check whether there is unwanted metadata  [default: False]
 * `-v, --verbose`: Log processed files in console  [default: False]
 * `-c, --config PATH`: Get CLI options from configuration file
-* `--help / --no-help`: Show this message and exit
+* `--help`: Show this message and exit
+
+<!-- [[[end]]] -->

docs/alternatives.md

@@ -25,4 +25,4 @@ conflicts.
 
 **Differences:** `nbdev` is closer to an opinionated template for developing packages
 with notebooks. `databooks` is a configurable CLI tool for metadata removal and conflict
-resolution.
+resolution.

docs/contrib.md

@@ -1 +1 @@
---8<-- "CONTRIBUTING.md"
+--8<-- "CONTRIBUTING.md"

docs/index.md

@@ -1 +1 @@
---8<-- "README.md"
+--8<-- "README.md"

docs/license.md

@@ -1 +1 @@
---8<-- "LICENSE"
+--8<-- "LICENSE"

docs/stylesheets/extra.css

@@ -1,3 +1,3 @@
 :root {
   --md-primary-fg-color:        #00b189;
-}
+}