Dialect map private API

About

This repository contains the private API to modify the database information.

It will be used in combination with the set of I/O pipelines that populate information either from the public datasets or from the corpus of ArXiv PDFs, in order to perform write operations (create, update and delete) over the underlying database records.

Dependencies

Python dependencies are specified on the multiple files within the reqs directory.

In order to install all the development packages, as long as the defined commit hooks:

make install-dev

Formatting

All Python files are formatted using Black, and the custom properties defined in the pyproject.toml file.

make check

Testing

Project testing is performed using Pytest. In order to run the tests:

make test

Docker

The project is currently designed to be deployed in a Google Cloud Platform project, so the initial step involves using gcloud CLI tool to log into it:

gcloud login
gcloud auth configure-docker

In addition, macOS requires developers to have their identity keys added to the SSH agent in order for Docker to pick up those keys when building the image with the --ssh default flag.

ssh-add -K

To build a Docker image out of the project:

make build

To push a Docker image to the GCP registry:

export GCP_PROJECT="ds3-dialect-map"
export GCP_REGISTRY="us.gcr.io"
make push

Deployment

This project uses a set of env. variables to configure certain aspects of the API:

ENV VARIABLE	DEFAULT	REQUIRED	DESCRIPTION
DIALECT_MAP_DB_URL	...	No	Database connection URL
DIALECT_MAP_LOG_LEVEL	INFO	No	Log messages level