diff --git a/CONTRIB.md b/CONTRIB.md index 1387b0f9d..b902a3bb6 100644 --- a/CONTRIB.md +++ b/CONTRIB.md @@ -1,9 +1,7 @@ # Contributing Guide - First, thanks for being interested in helping us out! If you find an issue you're interested in, feel free to make a comment about how you're thinking of approaching implementing it in the issue and we can give you feedback. Please also read our [code of conduct](/CODE_OF_CONDUCT.md) before getting started. ## Submitting a Pull Request (PR) - When you come to implement your new feature, clone the repository and then create a branch off `develop` locally and add commits to implement your feature. If your git history is not so clean, please do rewrite before you submit your PR - if you're not sure if you need to do this, go ahead and submit and we can let you know when you submit. @@ -21,13 +19,11 @@ git config user.name "" This will make sure that all commits you make locally are associated with your github account and do not contain any additional identifying information. More detailed information on this topic can be found [here](https://docs.github.com/en/free-pro-team@latest/github/setting-up-and-managing-your-github-user-account/setting-your-commit-email-address). ### Linting / Style Checks +We use [pre-commit](https://pre-commit.com/) for automated linting and style checks. Be sure to [install pre-commit](https://pre-commit.com/#installation) and run `pre-commit install` in your local version of the repository to install our pre-commit checks. This will make sure your commits are always formatted correctly. - We use [pre-commit](https://pre-commit.com/) for automated linting and style checks. Be sure to [install pre-commit](https://pre-commit.com/#installation) and run `pre-commit install` in your local version of the repository to install our pre-commit checks. This will make sure your commits are always formatted correctly. - - You can run `pre-commit run --all-files` or `make lint` to run pre-commit over your local codebase, or `pre-commit run` to run it only over the currently stages files. +You can run `pre-commit run --all-files` or `make lint` to run pre-commit over your local codebase, or `pre-commit run` to run it only over the currently stages files. ## Development Environment - You can use our Docker-compose environment to stand up a development OpenOversight. You will need to have Docker installed in order to use the Docker development environment. @@ -53,8 +49,24 @@ $ docker exec -it openoversight_web_1 /bin/bash Once you're done, `make stop` and `make clean` to stop and remove the containers respectively. -## Testing S3 Functionality +## Gmail Requirements +**NOTE:** If you are running on dev and do not currently have a `service_account_key.json` file, create one and leave it empty. The email client will then default to an empty object and simulate emails in the logs. + +For the application to work properly, you will need a [Google Cloud Platform service account](https://cloud.google.com/iam/docs/service-account-overview) that is attached to a GSuite email address. Here are some general tips for working with service accounts: [Link](https://support.google.com/a/answer/7378726?hl=en). +We would suggest that you do not use a personal email address, but instead one that is used strictly for sending out OpenOversight emails. + +You will need to do these two things for the service account to work as a Gmail bot: +1. Enable domain-wide delegation for the service account: [Link](https://support.google.com/a/answer/162106?hl=en) +2. Enable the `https://www.googleapis.com/auth/gmail.send` scope in the Gmail API for your service account: [Link](https://developers.google.com/gmail/api/auth/scopes#scopes) +3. Save the service account key file in OpenOversight's base folder as `service_account_key.json`. The file is in the `.gitignore` file GitHub will not allow you to save it, provided you've named it correctly. +4. For production, save the email address associated with your service account to a variable named `OO_SERVICE_EMAIL` in a `.env` file in the base directory of this repository. For development and testing, update the `OO_SERVICE_EMAIL` variable in the `docker-compose.yml` file. +Example `.env` variable: +```bash +OO_SERVICE_EMAIL="sample_email@domain.com" +``` + +## Testing S3 Functionality We use an S3 bucket for image uploads. If you are working on functionality involving image uploads, then you should follow the "S3 Image Hosting" section in [DEPLOY.md](/DEPLOY.md) to make a test S3 bucket on Amazon Web Services. @@ -72,7 +84,6 @@ Now when you run `make dev` as usual in the same session, you will be able to su your test bucket. ## Database commands - Running `make dev` will create the database and persist it into your local filesystem. You can access your PostgreSQL development database via psql using: @@ -89,7 +100,6 @@ or `$ python test_data.py --cleanup` to delete the data ### Migrating the Database - If you e.g. add a new column or table, you'll need to migrate the database using the Flask CLI. First we need to 'stamp' the current version of the database: ```sh @@ -152,7 +162,6 @@ pip install -r dev-requirements.txt ``` ## OpenOversight Management Interface - In addition to generating database migrations, the Flask CLI can be used to run additional commands: ```sh diff --git a/Makefile b/Makefile index 97c4afa3d..8205486c5 100644 --- a/Makefile +++ b/Makefile @@ -3,19 +3,19 @@ export UID=$(shell id -u) default: build start create_db populate test stop clean .PHONY: build -build: ## Build containers +build: ## Build containers docker-compose build .PHONY: build_with_version -build_with_version: - docker-compose build --build-arg TRAVIS_PYTHON_VERSION=$(PYTHON_VERSION) +build_with_version: create_empty_secret create_empty_env + docker-compose build --build-arg MAKE_PYTHON_VERSION=$(PYTHON_VERSION) .PHONY: test_with_version test_with_version: build_with_version assets FLASK_ENV=testing docker-compose run --rm web pytest --cov=OpenOversight --cov-report xml:OpenOversight/tests/coverage.xml --doctest-modules -n 4 --dist=loadfile -v OpenOversight/tests/ .PHONY: start -start: build ## Run containers +start: build ## Run containers docker-compose up -d .PHONY: create_db @@ -36,7 +36,7 @@ assets: dev: build start create_db populate .PHONY: populate -populate: create_db ## Build and run containers +populate: create_db ## Build and run containers @until docker-compose exec postgres psql -h localhost -U openoversight -c '\l' postgres &>/dev/null; do \ echo "Postgres is unavailable - sleeping..."; \ sleep 1; \ @@ -46,27 +46,27 @@ populate: create_db ## Build and run containers docker-compose run --rm web python ./test_data.py -p .PHONY: test -test: start ## Run tests +test: start ## Run tests if [ -z "$(name)" ]; then \ FLASK_ENV=testing docker-compose run --rm web pytest --cov --doctest-modules -n auto --dist=loadfile -v OpenOversight/tests/; \ else \ - FLASK_ENV=testing docker-compose run --rm web pytest --cov --doctest-modules -v OpenOversight/tests/ -k $(name); \ + FLASK_ENV=testing docker-compose run --rm web pytest --cov --doctest-modules -v OpenOversight/tests/ -k $(name); \ fi .PHONY: lint lint: pre-commit run --all-files -.PHONY: cleanassets -cleanassets: +.PHONY: clean_assets +clean_assets: rm -rf ./OpenOversight/app/static/dist/ .PHONY: stop -stop: ## Stop containers +stop: ## Stop containers docker-compose stop .PHONY: clean -clean: cleanassets stop ## Remove containers +clean: clean_assets stop ## Remove containers docker-compose rm -f .PHONY: clean_all @@ -88,3 +88,18 @@ help: ## Print this message and exit .PHONY: attach attach: docker-compose exec postgres psql -h localhost -U openoversight openoversight-dev + +# TODO: These two commands are the same with the exception of the file name, this should be addressed at some point +.PHONY: create_empty_secret +create_empty_secret: # This is needed to make sure docker doesn't create an empty directory, or delete that directory first + touch service_account_key.json || \ + (echo "Need to delete that empty directory first"; \ + sudo rm -d service_account_key.json/; \ + touch service_account_key.json) + +.PHONY: create_empty_env +create_empty_env: # This is needed to make sure docker doesn't create an empty directory, or delete that directory first + touch .env || \ + (echo "Need to delete that empty directory first"; \ + sudo rm -d .env/; \ + touch .env) diff --git a/OpenOversight/app/__init__.py b/OpenOversight/app/__init__.py index 3fc64c451..f5cf12655 100644 --- a/OpenOversight/app/__init__.py +++ b/OpenOversight/app/__init__.py @@ -12,17 +12,17 @@ from flask_limiter import Limiter from flask_limiter.util import get_remote_address from flask_login import LoginManager -from flask_mail import Mail from flask_migrate import Migrate from flask_sitemap import Sitemap from flask_wtf.csrf import CSRFProtect from markupsafe import Markup -from .config import config +from OpenOversight.app.config import config +from OpenOversight.app.email_client import EmailClient +from OpenOversight.app.utils.constants import MEGABYTE, SERVICE_ACCOUNT_FILE bootstrap = Bootstrap() -mail = Mail() login_manager = LoginManager() login_manager.session_protection = "strong" @@ -43,12 +43,15 @@ def create_app(config_name="default"): from .models import db bootstrap.init_app(app) - mail.init_app(app) + csrf.init_app(app) db.init_app(app) - login_manager.init_app(app) + # This allows the application to run without creating an email client if it is + # in testing or dev mode and the service account file is empty. + service_account_file_size = os.path.getsize(SERVICE_ACCOUNT_FILE) + EmailClient(dev=app.debug and service_account_file_size == 0, testing=app.testing) limiter.init_app(app) + login_manager.init_app(app) sitemap.init_app(app) - csrf.init_app(app) from .main import main as main_blueprint @@ -58,7 +61,7 @@ def create_app(config_name="default"): app.register_blueprint(auth_blueprint, url_prefix="/auth") - max_log_size = 10 * 1024 * 1024 # start new log file after 10 MB + max_log_size = 10 * MEGABYTE # start new log file after 10 MB num_logs_to_keep = 5 file_handler = RotatingFileHandler( "/tmp/openoversight.log", "a", max_log_size, num_logs_to_keep diff --git a/OpenOversight/app/auth/utils.py b/OpenOversight/app/auth/utils.py index 8008579e1..2eb151816 100644 --- a/OpenOversight/app/auth/utils.py +++ b/OpenOversight/app/auth/utils.py @@ -4,6 +4,7 @@ from flask_login import current_user +# TODO: Move these functions to the utils package def admin_required(f): @wraps(f) def decorated_function(*args, **kwargs): diff --git a/OpenOversight/app/auth/views.py b/OpenOversight/app/auth/views.py index 4f75b1832..069ca544e 100644 --- a/OpenOversight/app/auth/views.py +++ b/OpenOversight/app/auth/views.py @@ -11,13 +11,20 @@ ) from flask_login import current_user, login_required, login_user, logout_user +from OpenOversight.app.email_client import ( + AdministratorApprovalEmail, + ChangeEmailAddressEmail, + ConfirmAccountEmail, + ConfirmedUserEmail, + EmailClient, + ResetPasswordEmail, +) +from OpenOversight.app.models import User, db from OpenOversight.app.utils.constants import HTTP_METHOD_GET, HTTP_METHOD_POST from OpenOversight.app.utils.forms import set_dynamic_default from OpenOversight.app.utils.general import validate_redirect_url from .. import sitemap -from ..email import send_email -from ..models import User, db from . import auth from .forms import ( ChangeDefaultDepartmentForm, @@ -99,7 +106,7 @@ def logout(): @sitemap_include @auth.route("/register", methods=[HTTP_METHOD_GET, HTTP_METHOD_POST]) def register(): - jsloads = ["js/zxcvbn.js", "js/password.js"] + js_loads = ["js/zxcvbn.js", "js/password.js"] form = RegistrationForm() if form.validate_on_submit(): user = User( @@ -113,12 +120,8 @@ def register(): if current_app.config["APPROVE_REGISTRATIONS"]: admins = User.query.filter_by(is_administrator=True).all() for admin in admins: - send_email( - admin.email, - "New user registered", - "auth/email/new_registration", - user=user, - admin=admin, + EmailClient.send_email( + AdministratorApprovalEmail(admin.email, user=user, admin=admin) ) flash( "Once an administrator approves your registration, you will " @@ -126,18 +129,14 @@ def register(): ) else: token = user.generate_confirmation_token() - send_email( - user.email, - "Confirm Your Account", - "auth/email/confirm", - user=user, - token=token, + EmailClient.send_email( + ConfirmAccountEmail(user.email, user=user, token=token) ) flash("A confirmation email has been sent to you.") return redirect(url_for("auth.login")) else: current_app.logger.info(form.errors) - return render_template("auth/register.html", form=form, jsloads=jsloads) + return render_template("auth/register.html", form=form, jsloads=js_loads) @auth.route("/confirm/", methods=[HTTP_METHOD_GET]) @@ -148,12 +147,8 @@ def confirm(token): if current_user.confirm(token): admins = User.query.filter_by(is_administrator=True).all() for admin in admins: - send_email( - admin.email, - "New user confirmed", - "auth/email/new_confirmation", - user=current_user, - admin=admin, + EmailClient.send_email( + ConfirmedUserEmail(admin.email, user=current_user, admin=admin) ) flash("You have confirmed your account. Thanks!") else: @@ -165,12 +160,8 @@ def confirm(token): @login_required def resend_confirmation(): token = current_user.generate_confirmation_token() - send_email( - current_user.email, - "Confirm Your Account", - "auth/email/confirm", - user=current_user, - token=token, + EmailClient.send_email( + ConfirmAccountEmail(current_user.email, user=current_user, token=token) ) flash("A new confirmation email has been sent to you.") return redirect(url_for("main.index")) @@ -179,7 +170,7 @@ def resend_confirmation(): @auth.route("/change-password", methods=[HTTP_METHOD_GET, HTTP_METHOD_POST]) @login_required def change_password(): - jsloads = ["js/zxcvbn.js", "js/password.js"] + js_loads = ["js/zxcvbn.js", "js/password.js"] form = ChangePasswordForm() if form.validate_on_submit(): if current_user.verify_password(form.old_password.data): @@ -192,7 +183,7 @@ def change_password(): flash("Invalid password.") else: current_app.logger.info(form.errors) - return render_template("auth/change_password.html", form=form, jsloads=jsloads) + return render_template("auth/change_password.html", form=form, jsloads=js_loads) @auth.route("/reset", methods=[HTTP_METHOD_GET, HTTP_METHOD_POST]) @@ -204,12 +195,8 @@ def password_reset_request(): user = User.by_email(form.email.data).first() if user: token = user.generate_reset_token() - send_email( - user.email, - "Reset Your Password", - "auth/email/reset_password", - user=user, - token=token, + EmailClient.send_email( + ResetPasswordEmail(user.email, user=user, token=token) ) flash("An email with instructions to reset your password has been sent to you.") return redirect(url_for("auth.login")) @@ -245,12 +232,8 @@ def change_email_request(): if current_user.verify_password(form.password.data): new_email = form.email.data token = current_user.generate_email_change_token(new_email) - send_email( - new_email, - "Confirm your email address", - "auth/email/change_email", - user=current_user, - token=token, + EmailClient.send_email( + ChangeEmailAddressEmail(new_email, user=current_user, token=token) ) flash( "An email with instructions to confirm your new email " @@ -338,7 +321,8 @@ def edit_user(user_id): db.session.add(user) db.session.commit() - # automatically send a confirmation email when approving an unconfirmed user + # automatically send a confirmation email when approving an + # unconfirmed user if ( current_app.config["APPROVE_REGISTRATIONS"] and not already_approved @@ -376,12 +360,6 @@ def admin_resend_confirmation(user): flash("User {} is already confirmed.".format(user.username)) else: token = user.generate_confirmation_token() - send_email( - user.email, - "Confirm Your Account", - "auth/email/confirm", - user=user, - token=token, - ) + EmailClient.send_email(ConfirmAccountEmail(user.email, user=user, token=token)) flash("A new confirmation email has been sent to {}.".format(user.email)) return redirect(url_for("auth.get_users")) diff --git a/OpenOversight/app/config.py b/OpenOversight/app/config.py index b4fb6099c..34bddcbae 100644 --- a/OpenOversight/app/config.py +++ b/OpenOversight/app/config.py @@ -1,14 +1,16 @@ import os +from OpenOversight.app.utils.constants import MEGABYTE + basedir = os.path.abspath(os.path.dirname(__file__)) class BaseConfig(object): - # DB SETUP + # DB Settings SQLALCHEMY_TRACK_MODIFICATIONS = False - # pagination + # Pagination Settings OFFICERS_PER_PAGE = os.environ.get("OFFICERS_PER_PAGE", 20) USERS_PER_PAGE = os.environ.get("USERS_PER_PAGE", 20) @@ -17,16 +19,8 @@ class BaseConfig(object): SECRET_KEY = os.environ.get("SECRET_KEY", "changemeplzorelsehax") # Mail Settings - MAIL_SERVER = os.environ.get("MAIL_SERVER", "smtp.googlemail.com") - MAIL_PORT = 587 - MAIL_USE_TLS = True - MAIL_USERNAME = os.environ.get("MAIL_USERNAME") - MAIL_PASSWORD = os.environ.get("MAIL_PASSWORD") OO_MAIL_SUBJECT_PREFIX = os.environ.get("OO_MAIL_SUBJECT_PREFIX", "[OpenOversight]") - OO_MAIL_SENDER = os.environ.get( - "OO_MAIL_SENDER", "OpenOversight " - ) - # OO_ADMIN = os.environ.get('OO_ADMIN') + OO_SERVICE_EMAIL = os.environ.get("OO_SERVICE_EMAIL") # AWS Settings AWS_ACCESS_KEY_ID = os.environ.get("AWS_ACCESS_KEY_ID") @@ -35,16 +29,17 @@ class BaseConfig(object): S3_BUCKET_NAME = os.environ.get("S3_BUCKET_NAME") # Upload Settings - MAX_CONTENT_LENGTH = 50 * 1024 * 1024 + MAX_CONTENT_LENGTH = 50 * MEGABYTE ALLOWED_EXTENSIONS = set(["jpeg", "jpg", "jpe", "png", "gif", "webp"]) - # User settings + # User Settings APPROVE_REGISTRATIONS = os.environ.get("APPROVE_REGISTRATIONS", False) # Use session cookie to store URL to redirect to after login # https://flask-login.readthedocs.io/en/latest/#customizing-the-login-process USE_SESSION_FOR_NEXT = True + # Misc. Settings SEED = 666 @staticmethod diff --git a/OpenOversight/app/email.py b/OpenOversight/app/email.py deleted file mode 100644 index cf8fc0f8f..000000000 --- a/OpenOversight/app/email.py +++ /dev/null @@ -1,30 +0,0 @@ -from threading import Thread - -from flask import current_app, render_template -from flask_mail import Message - -from . import mail - - -def send_async_email(app, msg): - with app.app_context(): - mail.send(msg) - - -def send_email(to, subject, template, **kwargs): - app = current_app._get_current_object() - msg = Message( - app.config["OO_MAIL_SUBJECT_PREFIX"] + " " + subject, - sender=app.config["OO_MAIL_SENDER"], - recipients=[to], - ) - msg.body = render_template(template + ".txt", **kwargs) - msg.html = render_template(template + ".html", **kwargs) - # Only send email if we're in prod or staging, otherwise log it so devs can see it - if app.env in ("staging", "production"): - thr = Thread(target=send_async_email, args=[app, msg]) - app.logger.info("Sent email.") - thr.start() - return thr - else: - app.logger.info("simulated email:\n%s\n%s", subject, msg.body) diff --git a/OpenOversight/app/email_client.py b/OpenOversight/app/email_client.py new file mode 100644 index 000000000..f89492a49 --- /dev/null +++ b/OpenOversight/app/email_client.py @@ -0,0 +1,125 @@ +import base64 +from email.mime.text import MIMEText + +from apiclient import errors +from flask import current_app, render_template +from google.oauth2 import service_account +from googleapiclient.discovery import build + +from OpenOversight.app.config import BaseConfig +from OpenOversight.app.utils.constants import SERVICE_ACCOUNT_FILE + + +class Email: + """Base class for all emails.""" + + def __init__(self, body: str, subject: str, receiver: str): + self.body = body + self.receiver = receiver + self.subject = subject + + def create_message(self): + message = MIMEText(self.body, "html") + message["to"] = self.receiver + message["from"] = BaseConfig.OO_SERVICE_EMAIL + message["subject"] = self.subject + return {"raw": base64.urlsafe_b64encode(message.as_bytes()).decode()} + + +class AdministratorApprovalEmail(Email): + def __init__(self, receiver: str, user, admin): + subject = ( + f"{current_app.config.get('OO_MAIL_SUBJECT_PREFIX')} New User Registered" + ) + body = render_template( + "auth/email/new_registration.html", user=user, admin=admin + ) + super().__init__(body, subject, receiver) + + +class ChangeEmailAddressEmail(Email): + def __init__(self, receiver: str, user, token: str): + subject = ( + f"{current_app.config.get('OO_MAIL_SUBJECT_PREFIX')} Confirm Your Email " + f"Address" + ) + body = render_template("auth/email/change_email.html", user=user, token=token) + super().__init__(body, subject, receiver) + + +class ConfirmAccountEmail(Email): + def __init__(self, receiver: str, user, token: str): + subject = ( + f"{current_app.config.get('OO_MAIL_SUBJECT_PREFIX')} Confirm Your Account" + ) + body = render_template("auth/email/confirm.html", user=user, token=token) + super().__init__(body, subject, receiver) + + +class ConfirmedUserEmail(Email): + def __init__(self, receiver: str, user, admin): + subject = ( + f"{current_app.config.get('OO_MAIL_SUBJECT_PREFIX')} New User Confirmed" + ) + body = render_template( + "auth/email/new_confirmation.html", user=user, admin=admin + ) + super().__init__(body, subject, receiver) + + +class ResetPasswordEmail(Email): + def __init__(self, receiver: str, user, token: str): + subject = ( + f"{current_app.config.get('OO_MAIL_SUBJECT_PREFIX')} Reset Your Password" + ) + body = render_template("auth/email/reset_password.html", user=user, token=token) + super().__init__(body, subject, receiver) + + +class EmailClient(object): + """ + EmailClient is a Singleton class that is used for the Gmail client. + This can be fairly easily switched out with another email service, but it is + currently defaulted to Gmail. + """ + + SCOPES = ["https://www.googleapis.com/auth/gmail.send"] + + _instance = None + + def __new__(cls, dev=False, testing=False): + if (testing or dev) and cls._instance is None: + cls._instance = {} + + if cls._instance is None: + credentials = service_account.Credentials.from_service_account_file( + SERVICE_ACCOUNT_FILE, scopes=cls.SCOPES + ) + delegated_credentials = credentials.with_subject( + BaseConfig.OO_SERVICE_EMAIL + ) + cls.service = build("gmail", "v1", credentials=delegated_credentials) + cls._instance = super(EmailClient, cls).__new__(cls) + return cls._instance + + @classmethod + def send_email(cls, email: Email): + """ + Deliver the email from the parameter list using the Singleton client. + + :param email: the specific email to be delivered + """ + if not cls._instance: + current_app.logger.info( + "simulated email:\n%s\n%s", email.subject, email.body + ) + else: + try: + ( + cls.service.users() + .messages() + .send(userId="me", body=email.create_message()) + .execute() + ) + except errors.HttpError as error: + print("An error occurred: %s" % error) diff --git a/OpenOversight/app/models.py b/OpenOversight/app/models.py index 99a08ba3d..32d3ed410 100644 --- a/OpenOversight/app/models.py +++ b/OpenOversight/app/models.py @@ -426,6 +426,7 @@ class LicensePlate(BaseModel): id = db.Column(db.Integer, primary_key=True) number = db.Column(db.String(8), nullable=False, index=True) state = db.Column(db.String(2), index=True) + # for use if car is federal, diplomat, or other non-state # non_state_identifier = db.Column(db.String(20), index=True) diff --git a/OpenOversight/app/templates/auth/email/change_email.txt b/OpenOversight/app/templates/auth/email/change_email.txt deleted file mode 100644 index 0604e669a..000000000 --- a/OpenOversight/app/templates/auth/email/change_email.txt +++ /dev/null @@ -1,11 +0,0 @@ -Dear {{ user.username }}, - -To confirm your new email address click on the following link: - -{{ url_for('auth.change_email', token=token, _external=True) }} - -Sincerely, - -The OpenOversight Team - -Note: replies to this email address are not monitored. diff --git a/OpenOversight/app/templates/auth/email/confirm.txt b/OpenOversight/app/templates/auth/email/confirm.txt deleted file mode 100644 index 41abe7c57..000000000 --- a/OpenOversight/app/templates/auth/email/confirm.txt +++ /dev/null @@ -1,13 +0,0 @@ -Dear {{ user.username }}, - -Welcome to OpenOversight! - -To confirm your account please click on the following link: - -{{ url_for('auth.confirm', token=token, _external=True) }} - -Sincerely, - -The OpenOversight Team - -Note: replies to this email address are not monitored. diff --git a/OpenOversight/app/templates/auth/email/new_confirmation.txt b/OpenOversight/app/templates/auth/email/new_confirmation.txt deleted file mode 100644 index a6328ac0e..000000000 --- a/OpenOversight/app/templates/auth/email/new_confirmation.txt +++ /dev/null @@ -1,16 +0,0 @@ -Dear {{ admin.username }}, - -A new user account with the following information has been confirmed: - -Username: {{ user.username }} -Email: {{ user.email }} - -To view or delete this user, click on the following link: - -{{ url_for('auth.edit_user', user_id=user.id, _external=True) }} - -Sincerely, - -The OpenOversight Team - -Note: replies to this email address are not monitored. diff --git a/OpenOversight/app/templates/auth/email/new_registration.txt b/OpenOversight/app/templates/auth/email/new_registration.txt deleted file mode 100644 index 5840085a9..000000000 --- a/OpenOversight/app/templates/auth/email/new_registration.txt +++ /dev/null @@ -1,16 +0,0 @@ -Dear {{ admin.username }}, - -A new user has registered with the following information: - -Username: {{ user.username }} -Email: {{ user.email }} - -To approve or delete this user, click on the following link: - -{{ url_for('auth.edit_user', user_id=user.id, _external=True) }} - -Sincerely, - -The OpenOversight Team - -Note: replies to this email address are not monitored. diff --git a/OpenOversight/app/templates/auth/email/reset_password.txt b/OpenOversight/app/templates/auth/email/reset_password.txt deleted file mode 100644 index d14796c51..000000000 --- a/OpenOversight/app/templates/auth/email/reset_password.txt +++ /dev/null @@ -1,13 +0,0 @@ -Dear {{ user.username }}, - -To reset your password click on the following link: - -{{ url_for('auth.password_reset', token=token, _external=True) }} - -If you have not requested a password reset simply ignore this message. - -Sincerely, - -The OpenOversight Team - -Note: replies to this email address are not monitored. diff --git a/OpenOversight/app/utils/constants.py b/OpenOversight/app/utils/constants.py index eb74a08d0..5745d69ad 100644 --- a/OpenOversight/app/utils/constants.py +++ b/OpenOversight/app/utils/constants.py @@ -1,14 +1,20 @@ import os -# Encoding constants +# File Handling Constants ENCODING_UTF_8 = "utf-8" +SAVED_UMASK = os.umask(0o077) # Ensure the file is read/write by the creator only -# HTTP Method constants +# File Name Constants +SERVICE_ACCOUNT_FILE = "service_account_key.json" + +# HTTP Method Constants # TODO: Remove these constants and use HTTPMethod in http package when we # migrate to version 3.11 HTTP_METHOD_GET = "GET" HTTP_METHOD_POST = "POST" -# Ensure the file is read/write by the creator only -SAVED_UMASK = os.umask(0o077) +# Numerical Constants +BYTE = 1 +KILOBYTE = 1024 * BYTE +MEGABYTE = 1024 * KILOBYTE diff --git a/OpenOversight/tests/routes/test_user_api.py b/OpenOversight/tests/routes/test_user_api.py index 6f9de8147..7fe28cbf2 100644 --- a/OpenOversight/tests/routes/test_user_api.py +++ b/OpenOversight/tests/routes/test_user_api.py @@ -5,6 +5,7 @@ from flask import current_app, url_for from OpenOversight.app.auth.forms import EditUserForm, LoginForm, RegistrationForm +from OpenOversight.app.email_client import EmailClient from OpenOversight.app.models import User, db from OpenOversight.app.utils.constants import ( ENCODING_UTF_8, @@ -267,8 +268,9 @@ def test_admin_can_resend_user_confirmation_email(mockdata, client, session): def test_register_user_approval_required(mockdata, client, session): current_app.config["APPROVE_REGISTRATIONS"] = True + EmailClient(testing=True) with current_app.test_request_context(): - diceware_password = "operative hamster perservere verbalize curling" + diceware_password = "operative hamster persevere verbalize curling" form = RegistrationForm( email="jen@example.com", username="redshiftzero", @@ -327,11 +329,13 @@ def test_admin_can_approve_user(mockdata, client, session): @pytest.mark.parametrize( - "currently_approved, currently_confirmed, approve_registration_config, should_send_email", + "currently_approved, currently_confirmed, approve_registration_config, " + "should_send_email", [ # Approving unconfirmed user sends email (False, False, True, True), - # Approving unconfirmed user does not send email if approve_registration config is not set + # Approving unconfirmed user does not send email if approve_registration config + # is not set (False, False, False, False), # Updating approved user does not send email (True, False, True, False), @@ -349,6 +353,7 @@ def test_admin_approval_sends_confirmation_email( session, ): current_app.config["APPROVE_REGISTRATIONS"] = approve_registration_config + EmailClient(testing=True) with current_app.test_request_context(): login_admin(client) diff --git a/docker-compose.prod-img.yml b/docker-compose.prod-img.yml index b2d7d48dc..fe0fff747 100644 --- a/docker-compose.prod-img.yml +++ b/docker-compose.prod-img.yml @@ -1,5 +1,9 @@ version: "3.4" +secrets: + service-account-key: + file: ./service_account_key.json + services: web: image: ghcr.io/lucyparsons/openoversight:${DOCKER_IMAGE_TAG} @@ -16,3 +20,6 @@ services: networks: - default restart: always + secrets: + - source: service-account-key + target: /usr/src/app/service_account_key.json diff --git a/docker-compose.yml b/docker-compose.yml index 8ce6c5f09..aff18769b 100644 --- a/docker-compose.yml +++ b/docker-compose.yml @@ -1,9 +1,13 @@ -version: "3" +version: "3.2" volumes: postgres: driver: local +secrets: + service-account-key: + file: ./service_account_key.json + services: postgres: restart: always @@ -23,18 +27,22 @@ services: context: . args: - DOCKER_BUILD_ENV - - TRAVIS_PYTHON_VERSION + - MAKE_PYTHON_VERSION dockerfile: ./dockerfiles/web/Dockerfile environment: + APPROVE_REGISTRATIONS: "${APPROVE_REGISTRATIONS}" AWS_ACCESS_KEY_ID: "${AWS_ACCESS_KEY_ID}" - AWS_SECRET_ACCESS_KEY: "${AWS_SECRET_ACCESS_KEY}" AWS_DEFAULT_REGION: "${AWS_DEFAULT_REGION}" - S3_BUCKET_NAME: "${S3_BUCKET_NAME}" - APPROVE_REGISTRATIONS: "${APPROVE_REGISTRATIONS}" + AWS_SECRET_ACCESS_KEY: "${AWS_SECRET_ACCESS_KEY}" FLASK_APP: OpenOversight.app FLASK_ENV: "${FLASK_ENV:-development}" + OO_SERVICE_EMAIL: "openoversightchi@lucyparsonslabs.com" + S3_BUCKET_NAME: "${S3_BUCKET_NAME}" volumes: - ./OpenOversight/:/usr/src/app/OpenOversight/:z + secrets: + - source: service-account-key + target: /usr/src/app/service_account_key.json user: "${UID:?Docker-compose needs UID set to the current user id number. Try 'export UID' and run docker-compose again}" links: - postgres:postgres diff --git a/dockerfiles/web/Dockerfile b/dockerfiles/web/Dockerfile index 820e4c5e6..827ea2b2f 100644 --- a/dockerfiles/web/Dockerfile +++ b/dockerfiles/web/Dockerfile @@ -1,6 +1,6 @@ -ARG TRAVIS_PYTHON_VERSION +ARG MAKE_PYTHON_VERSION ARG DOCKER_BUILD_ENV -FROM python:${TRAVIS_PYTHON_VERSION:-3.8}-buster +FROM python:${MAKE_PYTHON_VERSION:-3.8}-buster WORKDIR /usr/src/app diff --git a/mypy.ini b/mypy.ini index bd20fc711..3f39ef6da 100644 --- a/mypy.ini +++ b/mypy.ini @@ -22,9 +22,6 @@ ignore_missing_imports = True [mypy-flask_login.*] ignore_missing_imports = True -[mypy-flask_mail.*] -ignore_missing_imports = True - [mypy-flask_migrate.*] ignore_missing_imports = True diff --git a/requirements.txt b/requirements.txt index 90acad994..4234e36ef 100644 --- a/requirements.txt +++ b/requirements.txt @@ -1,5 +1,5 @@ -# TODO: There are currently A LOT of packages that we are using that are not included in this file. alembic~=1.11.1 +apiclient==1.0.4 attrs~=23.1.0 Authlib==1.2.1 Babel~=2.12.1 @@ -22,12 +22,13 @@ Flask==2.2.5 # Updating this breaks the build for python 3.8 Flask-Bootstrap==3.3.7.1 Flask-Limiter==3.3.1 Flask-Login==0.6.2 -Flask-Mail==0.9.1 Flask-Migrate==4.0.4 Flask-Sitemap==0.4.0 Flask-SQLAlchemy==3.0.5 Flask-WTF==1.1.1 +google-api-python-client==2.92.0 gunicorn==20.1.0 +httplib2==0.22.0 idna~=3.4 itsdangerous~=2.1.2 Jinja2~=3.1.2