Skip to content

Commit

Permalink
add contributing page
Browse files Browse the repository at this point in the history
  • Loading branch information
aaronsteers committed Feb 16, 2024
1 parent f800e81 commit 386a892
Show file tree
Hide file tree
Showing 2 changed files with 48 additions and 32 deletions.
35 changes: 35 additions & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
# Contributing to AirbyteLib

Learn how you can become a contributor to AirbyteLib.

## Development

- Make sure [Poetry is installed](https://python-poetry.org/docs/#).
- Run `poetry install`
- For examples, check out the `examples` folder. They can be run via `poetry run python examples/<example file>`
- Unit tests and type checks can be run via `poetry run pytest`

## Documentation

Regular documentation lives in the `/docs` folder. Based on the doc strings of public methods, we generate API documentation using [pdoc](https://pdoc.dev).

To generate the documentation, run:

```console
poetry run generate-docs
```

The `generate-docs` CLI command is mapped to the `run()` function of `docs.py` in the root `airbyte-lib` directory.

Documentation pages will be generated in the `docs/generated` folder. The `test_docs.py` test in pytest will automatically update generated content. This updates must be manually committed before docs tests will pass.

## Release

- In your PR:
- Bump the version in `pyproject.toml`
- Add a changelog entry to the table below
- Once the PR is merged, go to Github and trigger the `Publish AirbyteLib Manually` workflow. This will publish the new version to PyPI.

## Versioning

Versioning follows [Semantic Versioning](https://semver.org/). For new features, bump the minor version. For bug fixes, bump the patch version. For pre-releases, append `dev.N` to the version. For example, `0.1.0dev.1` is the first pre-release of the `0.1.0` version.
45 changes: 13 additions & 32 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,20 +1,6 @@
# PyAirbyte
# airbyte-lib

PyAirbyte is a library that allows to run Airbyte syncs embedded into any Python application, without the need to run Airbyte server.

## Development

- Make sure [Poetry is installed](https://python-poetry.org/docs/#).
- Run `poetry install`
- For examples, check out the `examples` folder. They can be run via `poetry run python examples/<example file>`
- Unit tests and type checks can be run via `poetry run pytest`

## Release

- In your PR:
- Bump the version in `pyproject.toml`
- Add a changelog entry to the table below
- Once the PR is merged, go to Github and trigger the `Publish AirbyteLib Manually` workflow. This will publish the new version to PyPI.
airbyte-lib is a library that allows to run Airbyte syncs embedded into any Python application, without the need to run Airbyte server.

## Secrets Management

Expand All @@ -25,12 +11,12 @@ AirbyteLib can auto-import secrets from the following sources:
3. [Google Colab secrets](https://medium.com/@parthdasawant/how-to-use-secrets-in-google-colab-450c38e3ec75).
4. Manual entry via [`getpass`](https://docs.python.org/3.9/library/getpass.html).

_Note: Additional secret store options may be supported in the future. [More info here.](https://github.com/airbytehq/PyAirbyte-private-beta/discussions/5)_
_Note: Additional secret store options may be supported in the future. [More info here.](https://github.com/airbytehq/airbyte-lib-private-beta/discussions/5)_

### Retrieving Secrets

```python
from airbyte import get_secret, SecretSource
from airbyte_lib import get_secret, SecretSource

source = get_connection("source-github")
source.set_config(
Expand All @@ -44,22 +30,13 @@ The `get_secret()` function accepts an optional `source` argument of enum type `

By default, AirbyteLib will prompt the user for any requested secrets that are not provided via other secret managers. You can disable this prompt by passing `prompt=False` to `get_secret()`.

### Versioning

Versioning follows [Semantic Versioning](https://semver.org/). For new features, bump the minor version. For bug fixes, bump the patch version. For pre-releases, append `dev.N` to the version. For example, `0.1.0dev.1` is the first pre-release of the `0.1.0` version.

## Documentation

Regular documentation lives in the `/docs` folder. Based on the doc strings of public methods, we generate API documentation using [pdoc](https://pdoc.dev). To generate the documentation, run `poetry run generate-docs`. The documentation will be generated in the `docs/generate` folder. This needs to be done manually when changing the public interface of the library.

A unit test validates the documentation is up to date.

## Connector compatibility

To make a connector compatible with PyAirbyte, the following requirements must be met:
* The connector must be a Python package, with a `pyproject.toml` or a `setup.py` file.
* In the package, there must be a `run.py` file that contains a `run` method. This method should read arguments from the command line, and run the connector with them, outputting messages to stdout.
* The `pyproject.toml` or `setup.py` file must specify a command line entry point for the `run` method called `source-<connector name>`. This is usually done by adding a `console_scripts` section to the `pyproject.toml` file, or a `entry_points` section to the `setup.py` file. For example:
To make a connector compatible with airbyte-lib, the following requirements must be met:

- The connector must be a Python package, with a `pyproject.toml` or a `setup.py` file.
- In the package, there must be a `run.py` file that contains a `run` method. This method should read arguments from the command line, and run the connector with them, outputting messages to stdout.
- The `pyproject.toml` or `setup.py` file must specify a command line entry point for the `run` method called `source-<connector name>`. This is usually done by adding a `console_scripts` section to the `pyproject.toml` file, or a `entry_points` section to the `setup.py` file. For example:

```toml
[tool.poetry.scripts]
Expand Down Expand Up @@ -101,6 +78,10 @@ The script will install the python package in the provided directory, and run th

For a more lightweight check, the `--validate-install-only` flag can be used. This will only check that the connector can be installed and returns a spec, no sample config required.

## Contributing

To learn how you can contribute to AirbyteLib, please see our [AirbyteLib Contributors Guide](./CONTRIBUTING.md).

## Changelog

| Version | PR | Description |
Expand Down

0 comments on commit 386a892

Please sign in to comment.