diff --git a/README.md b/README.md index 999cbf1..66433d5 100644 --- a/README.md +++ b/README.md @@ -3,19 +3,54 @@ -The IAEA [Marine Radioactivity Information System -(**MARIS**)](https://maris.iaea.org) allows free access to users to -search and download the results of measurements of radioactivity in -seawater, biota, sediment and suspended matter. MARIS is maintained and -developed by the [IAEA Environmental +The [IAEA Marine Radioactivity Information System +(MARIS)](https://maris.iaea.org) provides open access to radioactivity +measurements in marine environments. Developed by the [IAEA +Environmental Laboratories](https://www.iaea.org/about/organizational-structure/department-of-nuclear-sciences-and-applications/division-of-iaea-environment-laboratories) -in Monaco. +in Monaco, MARIS offers data on seawater, biota, sediment, and suspended +matter. -The current Python package offers command-line utilities for encoding -MARIS harvested datasets into -[`NetCDF`](https://www.unidata.ucar.edu/software/netcdf/) or `.csv` -formats. This allows to convert **MARIS data into a format that is -compatible with a wide range of scientific and data analysis tools**. +This Python package includes command-line tools to convert MARIS +datasets into [`NetCDF`](https://www.unidata.ucar.edu/software/netcdf/) +or `.csv` formats, enhancing compatibility with various scientific and +data analysis software. + +## Core Concept: Handlers + +`marisco` is built around the concept of `handlers` - specialized +modules designed to convert MARIS datasets into NetCDF format. Each +handler is tailored to a specific data provider and implemented as a +dedicated Jupyter notebook. + +### Literate Programming Approach + +We’ve adopted a Literate Programming approach, which means: + +1. **Documentation**: Each handler serves as comprehensive + documentation. +2. **Code Reference**: The notebooks contain the actual implementation + code. +3. **Communication Tool**: They facilitate discussions with data + providers about discrepancies or inconsistencies. + +### Powered by nbdev + +To achieve this, we leverage +[nbdev](https://nbdev.readthedocs.io/en/latest/), a powerful tool that +allows us to: + +1. Write code within Jupyter notebooks +2. Automatically export relevant parts as dedicated Python modules + +This approach bridges the gap between documentation and implementation, +ensuring they remain in sync. + +### See It in Action + +For a concrete example of this approach, check out our [HELCOM dataset +handler +implementation](https://github.com/franckalbinet/marisco/blob/main/nbs/api/handlers/helcom.ipynb). ## Install diff --git a/nbs/api/nc_template.ipynb b/nbs/api/nc_template.ipynb index ecc90dd..1561853 100644 --- a/nbs/api/nc_template.ipynb +++ b/nbs/api/nc_template.ipynb @@ -284,6 +284,7 @@ "metadata": {}, "outputs": [], "source": [ + "#| eval: false\n", "# Example of suffix defined in the .cdl file\n", "suffix = {\n", " 'name': '_unc',\n", diff --git a/nbs/api/serializers.ipynb b/nbs/api/serializers.ipynb index 8122657..fb8d21f 100644 --- a/nbs/api/serializers.ipynb +++ b/nbs/api/serializers.ipynb @@ -36,6 +36,7 @@ } ], "source": [ + "#| hide\n", "%load_ext autoreload\n", "%autoreload 2" ] @@ -59,14 +60,6 @@ "import os" ] }, - { - "cell_type": "markdown", - "id": "c7952355-3118-4362-86d4-c0331b5ef4e1", - "metadata": {}, - "source": [ - "## NetCDF encoder" - ] - }, { "cell_type": "code", "execution_count": null, diff --git a/nbs/api/utils.ipynb b/nbs/api/utils.ipynb index aada9d8..81447a9 100644 --- a/nbs/api/utils.ipynb +++ b/nbs/api/utils.ipynb @@ -54,7 +54,7 @@ "metadata": {}, "outputs": [], "source": [ - "#| eval: false\n", + "#| hide\n", "import pandas as pd" ] }, diff --git a/nbs/index.ipynb b/nbs/index.ipynb index 428fb2c..f013523 100644 --- a/nbs/index.ipynb +++ b/nbs/index.ipynb @@ -15,10 +15,39 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "The IAEA [Marine Radioactivity Information System (**MARIS**)](https://maris.iaea.org) allows free access to users to search and download the results of measurements of radioactivity in seawater, biota, sediment and suspended matter. MARIS is maintained and developed by the [IAEA Environmental Laboratories](https://www.iaea.org/about/organizational-structure/department-of-nuclear-sciences-and-applications/division-of-iaea-environment-laboratories) in Monaco.\n", + "The [IAEA Marine Radioactivity Information System (MARIS)](https://maris.iaea.org) provides open access to radioactivity measurements in marine environments. Developed by the [IAEA Environmental Laboratories](https://www.iaea.org/about/organizational-structure/department-of-nuclear-sciences-and-applications/division-of-iaea-environment-laboratories) in Monaco, MARIS offers data on seawater, biota, sediment, and suspended matter.\n", "\n", + "This Python package includes command-line tools to convert MARIS datasets into [`NetCDF`](https://www.unidata.ucar.edu/software/netcdf/) or `.csv` formats, enhancing compatibility with various scientific and data analysis software." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Core Concept: Handlers\n", + "\n", + "`marisco` is built around the concept of `handlers` - specialized modules designed to convert MARIS datasets into NetCDF format. Each handler is tailored to a specific data provider and implemented as a dedicated Jupyter notebook.\n", + "\n", + "### Literate Programming Approach\n", + "\n", + "We've adopted a Literate Programming approach, which means:\n", + "\n", + "1. **Documentation**: Each handler serves as comprehensive documentation.\n", + "2. **Code Reference**: The notebooks contain the actual implementation code.\n", + "3. **Communication Tool**: They facilitate discussions with data providers about discrepancies or inconsistencies.\n", + "\n", + "### Powered by nbdev\n", + "\n", + "To achieve this, we leverage [nbdev](https://nbdev.readthedocs.io/en/latest/), a powerful tool that allows us to:\n", + "\n", + "1. Write code within Jupyter notebooks\n", + "2. Automatically export relevant parts as dedicated Python modules\n", + "\n", + "This approach bridges the gap between documentation and implementation, ensuring they remain in sync.\n", + "\n", + "### See It in Action\n", "\n", - "The current Python package offers command-line utilities for encoding MARIS harvested datasets into [`NetCDF`](https://www.unidata.ucar.edu/software/netcdf/) or `.csv` formats. This allows to convert **MARIS data into a format that is compatible with a wide range of scientific and data analysis tools**." + "For a concrete example of this approach, check out our [HELCOM dataset handler implementation](https://github.com/franckalbinet/marisco/blob/main/nbs/api/handlers/helcom.ipynb)." ] }, {