Skip to content

Commit

Permalink
Make the first working version
Browse files Browse the repository at this point in the history
* Added the first version of the extractor and analyser modules/scripts
* Added unittest
* Added a logo
* Updated the README file
  • Loading branch information
habedi committed Oct 20, 2024
1 parent 4b2155d commit 1e83ae1
Show file tree
Hide file tree
Showing 27 changed files with 1,026 additions and 495 deletions.
2 changes: 1 addition & 1 deletion .coveragerc
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ omit =

# Include source files only from certain directories
source =
src/
mongo_analyser/

# Set parallel to true if you run tests in parallel
parallel = True
Expand Down
22 changes: 21 additions & 1 deletion .github/workflows/publish_to_pypi.yml
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ jobs:
- name: Set Up Python
uses: actions/setup-python@v4
with:
python-version: "3.10"
python-version: "3.12"

- name: Install Poetry
run: |
Expand All @@ -34,6 +34,26 @@ jobs:
# run: |
# poetry version patch # Use 'minor' or 'major' for minor or major version bumps

# - name: Build Package
# run: |
# poetry build

# Automatically create a new tag for the release
# - name: Create Tag
# run: |
# VERSION=$(poetry version -s) # Get version from pyproject.toml
# git config user.name "github-actions"
# git config user.email "[email protected]"
# git tag v$VERSION
# git push origin v$VERSION

# - name: Upload Release Asset
# uses: softprops/action-gh-release@v2 # Uploads the wheel files to the GitHub release
# with:
# files: dist/*.whl
# env:
# GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

- name: Build and Publish Package
run: |
poetry config pypi-token.pypi ${{ secrets.PYPI_API_TOKEN }} # Make sure you've set `secrets.PYPI_API_TOKEN`
Expand Down
14 changes: 12 additions & 2 deletions .github/workflows/tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ jobs:
strategy:
matrix:
# Define the Python versions to test against
python-version: [ "3.10", "3.11", "3.12" ]
python-version: [ "3.9", "3.10", "3.11", "3.12", "3.13" ]

steps:
- name: Checkout Repository
Expand All @@ -30,10 +30,20 @@ jobs:
- name: Run Tests with Coverage
run: |
poetry run pytest tests/ --doctest-modules --cov=src --cov-report xml --cov-report html \
poetry run pytest tests/ --doctest-modules --cov=mongo_analyser/ --cov-report xml \
--junitxml=junit/test-results-${{ matrix.python-version }}.xml
continue-on-error: false

# Upload results to Codecov
- name: Upload results to Codecov
uses: codecov/codecov-action@v4
with:
token: ${{ secrets.CODECOV_TOKEN }}
files: ./coverage.xml
flags: unittests
name: codecov-umbrella
fail_ci_if_error: false # Don't fail the CI if the upload fails

- name: Upload Test Results and Coverage Reports
uses: actions/upload-artifact@v4
with:
Expand Down
6 changes: 6 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -73,3 +73,9 @@ poetry.lock

# Miscellaneous files and directories to ignore
# Add any additional file patterns a directory names that should be ignored down here
*.csv
*.json
*.log
*.gz
*.xlsx
*.zip
131 changes: 23 additions & 108 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,126 +1,41 @@
# Template for a Python Library
<img src="assets/logo.svg" align="right" width="25%"/>
# Mongo Analyser

[![Tests](https://github.com/habedi/template-python-library/actions/workflows/tests.yml/badge.svg)](https://github.com/habedi/template-python-library/actions/workflows/tests.yml)
[![PyPI version](https://badge.fury.io/py/template-python-library-placeholder.svg)](https://badge.fury.io/py/template-python-library-placeholder)
[![License](https://img.shields.io/github/license/habedi/template-python-library)](https://github.com/habedi/template-python-library/blob/main/LICENSE)
[![Python version](https://img.shields.io/badge/Python-%3E=3.10-blue)](https://github.com/habedi/template-python-library)
[![Pip downloads](https://img.shields.io/pypi/dm/template-python-library-placeholder.svg)](https://pypi.org/project/template-python-library-placeholder)
[![Ruff](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/ruff/main/assets/badge/v2.json)](https://github.com/astral-sh/ruff)
[![CodeFactor](https://www.codefactor.io/repository/github/habedi/template-python-library/badge)](https://www.codefactor.io/repository/github/habedi/template-python-library)

This is a template repository to make it easier to start developing Python libraries. The repository includes the basic
structure for a Python library, including the code for a dummy package, unit tests, and GitHub Actions workflows for
running tests and deploying the library to PyPI.

As said above, I've created this template to make it easier to create new Python libraries. I hope you find it useful
as well. If you have any suggestions or improvements, feel free to open an issue or a pull request.

## Getting Started

To get started with this template, you can click on the `Use this template` button at the top of the repository. This
will create a new repository with the same structure as this one. You can then clone the repository to your local
machine and start developing your own Python library.
<img src="logo.png" align="right" width="25%"/>

### Installing Poetry

We will use [Poetry](https://python-poetry.org/) to manage the dependencies and the Python virtual environment. To
get started, you need to install Poetry on your machine. We can install Poetry by running the following command in the
command line using pip.

```bash
pip install poetry
```

When the installation is finished, run the following command in the shell in the root folder of this repository to
install the dependencies, and create a virtual environment we can use for development.

```bash
poetry install
```

After that, enter the Poetry environment by invoking the poetry shell command.

```bash
poetry shell
```

### Unit Tests and Code Coverage

We use [pytest](https://docs.pytest.org/en/stable/) to run the tests. To run the tests with code coverage, you can
run the following command in the shell.
[![Tests](https://github.com/habedi/mongo-analyser/actions/workflows/tests.yml/badge.svg)](https://github.com/habedi/mongo-analyser/actions/workflows/tests.yml)
[![codecov](https://codecov.io/gh/habedi/mongo-analyser/graph/badge.svg?token=HOTAZKP3V7)](https://codecov.io/gh/habedi/mongo-analyser)
[![PyPI version](https://badge.fury.io/py/mongo-analyser.svg)](https://badge.fury.io/py/mongo-analyser)
[![License](https://img.shields.io/github/license/habedi/mongo-analyser)](https://github.com/habedi/mongo-analyser/blob/main/LICENSE)
[![Python version](https://img.shields.io/badge/Python-%3E=3.9-blue)](https://github.com/habedi/mongo-analyser)
[![Pip downloads](https://img.shields.io/pypi/dm/mongo-analyser.svg)](https://pypi.org/project/mongo-analyser)
[![Ruff](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/ruff/main/assets/badge/v2.json)](https://github.com/astral-sh/ruff)
[![CodeFactor](https://www.codefactor.io/repository/github/habedi/mongo-analyser/badge)](https://www.codefactor.io/repository/github/habedi/mongo-analyser)

```bash
poetry run pytest tests/ --cov
```
Mongo Analyser is a tool that helps you to analyse the structure of a MongoDB collection. It can help you extract the
schema of a collection, find the data types of the fields, and also extract sample data from the collection based on the
schema.

You can edit the entries under `[tool.pytest.ini_options]` in the `pyproject.toml` file to configure pytest behaviour.
You can also edit the `.coveragerc` file to configure the coverage report generation.
Mongo Analyser can be used as a command-line tool or as a Python library.

### Building and Publishing
## Installation

As said before, we use Poetry to manage the dependencies and virtual environment. To build the pip package, you can
run
the following command in the shell.
You can install Mongo Analyser using pip (mainly to use it as a library):

```bash
poetry build
pip install mongo-analyser
```

This command will create a `dist` folder in the root directory, which contains the built version of the
library. To publish the library to PyPI, you can run the following command in the shell.
You can also install it as a standalone executable using pipx:

```bash
poetry publish
```

Note that you need to have a PyPI account and a valid API token to publish the library on PyPI. You can find more
information
about
publishing pip packages to PyPI in the [Poetry documentation](https://python-poetry.org/docs/repositories/).

### Files and Folders

The main files and folders in this repository are:

- `.github`: Contains the GitHub Actions workflows for running tests and deploying the built version of the library to
PyPI.
- `assets`: Contains the asset files for the library, like images, logos, etc.
- `docs`: Contains the documentation for the library.
- `src`: Includes the source code for the library. You can create multiple packages and modules in this folder. The
`src/package` folder contains a dummy package with a dummy class and method.
- `tests`: Contains the unit tests for the functions and methods in the library.
- `.editorconfig`: Configuration file for the code editor settings for consistent coding style.
- `.gitignore`: Python-specific gitignore file to exclude the files generated by Python and Poetry.
- `.gitattributes`: Configuration file for Git LFS.
- `LICENSE`: The license file for the library.
- `pyproject.toml`: The configuration file for Poetry, which manages the dependencies and virtual environment.

### Example

Here is an example of how you can use the code in the library after building and installing it as a pip package.

```python
from src.package.module import DummyClass

dummy = DummyClass()

print(dummy.dummy_method())
pipx install mongo-analyser
```

Before running the code, you need to install the library in the current environment. You can use Poetry to do this by
running the following commands in the shell.
After installing it using pipx, you can run it from the command line:

```bash
poetry build
poetry install
mongo-analyser <command> [<args>]
```

### Notes

- To publish the library to PyPI, you need to have a PyPI account and a valid API token. You can find more
information about publishing pip packages to PyPI in
the [Poetry documentation](https://python-poetry.org/docs/repositories/).
- To run the `.github/workflows/publish_to_pypi.yml` workflow, you need to set the `PYPI_API_TOKEN` secret for
the repository. You can find more information about repository secrets in the
[GitHub documentation](https://docs.github.com/en/actions/reference/encrypted-secrets).
See the [documentation](https://github.com/habedi/mongo-analyser/blob/main/docs/index.md) for more information and
examples.
75 changes: 0 additions & 75 deletions assets/dummy_figure.dot

This file was deleted.

Loading

0 comments on commit 1e83ae1

Please sign in to comment.