Add Docker image creation for crossreference #412

vladbogo · 2024-04-11T13:55:17Z

No description provided.

grooverdan · 2024-04-11T14:51:47Z

cross-reference/Dockerfile

+WORKDIR /app
+
+# Install system dependencies
+RUN apt-get update && apt-get install --no-install-recommends -y build-essential mariadb-client libmariadbclient-dev


build-essential and mariadb-client looks odd - does it directly use mariadb executable? Could append ; rm -rf /var/lib/apt/lists/* for size reduction.

Also:

please one package by line;

this lack of cleaning should have been catched by hadolint, can you make sure that it is run against this Dockerfile.

hadolint runs

probably the mariadb-client and build-essential are not needed. some leftovers from other dockerfiles. I'll remove them and we'll see after testing

hadolint runs

Nope, it does not, it runs on the wrong PATH...

~/buildbot/cross-reference remotes/vlad/crossref_docker ≡ .venv ❯ podman run -it -i -v $(pwd):/mnt -w /mnt ghcr.io/hadolint/hadolint:latest hadolint /mnt/Dockerfile WARN[0000] "/" is not a shared mount, this could cause issues or missing mounts with rootless containers Trying to pull ghcr.io/hadolint/hadolint:latest... Getting image source signatures Copying blob db4123164570 done | Copying config da13a5ec2e done | Writing manifest to image destination /mnt/Dockerfile:12 DL3008 warning: Pin versions in apt get install. Instead of `apt-get install <package>` use `apt-get install <package>=<version>` /mnt/Dockerfile:12 DL3009 info: Delete the apt-get lists after installing something /mnt/Dockerfile:18 DL3013 warning: Pin versions in pip. Instead of `pip install <package>` use `pip install <package>==<version>` or `pip install --requirement <requirements file>`

DL3008 are disabled but DL3009 is not.

The paths are too cryptic. I am not sure what it wants from pip

DL3009 is also disabled. Anyway not sure what it wants now

cross-reference/Dockerfile

fauust · 2024-04-12T11:57:59Z

cross-reference/Dockerfile

+WORKDIR /app
+
+# Install system dependencies
+RUN apt-get update && apt-get install --no-install-recommends -y build-essential mariadb-client libmariadbclient-dev


Also:

please one package by line;

this lack of cleaning should have been catched by hadolint, can you make sure that it is run against this Dockerfile.

fauust · 2024-04-12T11:58:35Z

cross-reference/Dockerfile

@@ -0,0 +1,27 @@
+# Use an official Python runtime as a parent image
+FROM python:3.11-slim-buster


Maybe a good moment to try to upgrade to bullseye (or better alpine :-) )

cross-reference/Dockerfile

cvicentiu · 2024-04-12T19:07:34Z

0.0.0.0 as far as I am aware

…

On Fri, 12 Apr 2024, 15:01 faust, ***@***.***> wrote: ***@***.**** commented on this pull request. ------------------------------ In cross-reference/Dockerfile <#412 (comment)>: > @@ -0,0 +1,27 @@ +# Use an official Python runtime as a parent image +FROM python:3.11-slim-buster Maybe a good moment to try to upgrade to bullseye (or better alpine :-) ) ------------------------------ In cross-reference/Dockerfile <#412 (comment)>: > @@ -0,0 +1,27 @@ +# Use an official Python runtime as a parent image +FROM python:3.11-slim-buster + +# Set environment variables +ENV PYTHONDONTWRITEBYTECODE 1 +ENV PYTHONUNBUFFERED 1 + +# Set work directory +WORKDIR /app + +# Install system dependencies +RUN apt-get update && apt-get install --no-install-recommends -y build-essential mariadb-client libmariadbclient-dev Also: - please one package by line; - this lack of cleaning should have been catched by hadolint, can you make sure that it is run against this Dockerfile. ------------------------------ In cross-reference/Dockerfile <#412 (comment)>: > +RUN apt-get update && apt-get install --no-install-recommends -y build-essential mariadb-client libmariadbclient-dev + +# Install python dependencies +COPY requirements.txt /app +RUN pip install --no-cache-dir --upgrade pip && \ + pip install --no-cache-dir -r requirements.txt + +# Copy project +COPY ./crossreference /app/ + +# Add and run as non-root user +RUN adduser --disabled-password --gecos '' cr +USER cr + +# Run gunicorn +CMD ["bash", "-c", "python manage.py collectstatic --noinput && gunicorn crossreference.wsgi:application --bind 1.0.0.0:25432"] yep, that's probably 0.0.0.0/0 (not sure if there is a best practice here, it maybe localhost too since the port is going to be exposed on localhost host by docker-compose). — Reply to this email directly, view it on GitHub <#412 (review)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AARWRPKS7NLQLBXG33CIGWDY47EJRAVCNFSM6AAAAABGCLOOFSVHI2DSMVQWIX3LMV43YUDVNRWFEZLROVSXG5CSMV3GSZLXHMYTSOJWGYZDQNRZGM> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

fauust · 2024-04-15T12:32:43Z

.github/workflows/bbm_build_crossreference.yml

+      - uses: actions/checkout@v4
+      - name: Check Dockerfile with hadolint
+        run: |
+          docker run -i -v $(pwd):/mnt -w /mnt ghcr.io/hadolint/hadolint:latest hadolint /mnt/Dockerfile


Here is how to check the correct Dockerfile:

- docker run -i -v $(pwd):/mnt -w /mnt ghcr.io/hadolint/hadolint:latest hadolint /mnt/Dockerfile + docker run -i -v $(pwd):/mnt -w /mnt ghcr.io/hadolint/hadolint:latest hadolint /mnt/cross-reference/Dockerfile

note that $(pwd) the current repo root is mounted in /mnt in the container, also since the working directory is /mnt (that's what the -w does), this should also work:

docker run -i -v $(pwd):/mnt -w /mnt ghcr.io/hadolint/hadolint:latest hadolint cross-reference/Dockerfile

This doesn't work because of line 16. All the next steps need to run in the cross-reference dir. So we either make this run in the parent, or copy the file

Oh, I see, sorry missed that one. I am happy with any one of the mentioned solution.

fauust

Do you think that you could implement a very basic check step in the CI, just to make sure that the image seems to look good.

fauust · 2024-04-29T13:35:31Z

.github/workflows/bbm_build_crossreference.yml

+          podman build . --tag ${{ env.REPO }}:crossreference
+      - name: Push images to local registry
+        run: |
+          for img in crossreference; do


this for loop is useless

fauust · 2024-04-29T13:39:09Z

cross-reference/Dockerfile

+    pip install --no-cache-dir -r requirements.txt
+
+# Copy project
+COPY ./crossreference /app/


It's probably cleaner to use this as a volume mount with the generated image. It would avoid regenerating the image anytime we change app code source...

I think it's easier to have all contained in the image, for local Development purposes.
If this sounds sane, then also making use of DEV_ *container tags will help separate the two environments.

RazvanLiviuVarzaru · 2024-08-14T10:26:08Z

What is the status of this PR?
Is it still work in progress / abandoned ?

Let's handle it appropriately so won't hang in the PR queue forever.
I can help with a review, but please help me with a quick recap of what is the purpose << No description provided. >> and what has happened so far + if there is any other information split over other channels (Zulip, Slack, e-mail, etc).

RazvanLiviuVarzaru · 2024-11-08T09:18:42Z

cross-reference/Dockerfile

+USER cr
+
+# Run gunicorn
+CMD ["bash", "-c", "python manage.py collectstatic --noinput && exec gunicorn crossreference.wsgi:application --bind 0.0.0.0:25432"]


I would really like to see a gunicorn configuration file, stored in the repository and exposed to the container as a mount.
https://docs.gunicorn.org/en/stable/settings.html

For example, we need to add the 60s timeout.

definitively the way to proceed, yes!

RazvanLiviuVarzaru · 2024-11-08T09:34:17Z

Another thing that is worth mentioning.
This PR needs a deployment pipeline, something like *deploy_crossreference*

Based on the current Git branching strategy,
--> Changes in DEV will trigger a deploy on DEV host. Here I propose to use hz-dev-bbm and use the MariaDB instance there (leverage the fact that now we have host networking everywhere, DEV is not under high load and CR is configured to look for DB on localhost anyway). @fauust sounds ok to you?
--> Changes on Main will trigger a deploy on hz-bbw5.

vladbogo force-pushed the crossref_docker branch 3 times, most recently from 5b72332 to 1cef242 Compare April 11, 2024 14:09

grooverdan reviewed Apr 11, 2024

View reviewed changes

fauust reviewed Apr 12, 2024

View reviewed changes

vladbogo force-pushed the crossref_docker branch from 1cef242 to 7844576 Compare April 15, 2024 09:22

vladbogo requested a review from fauust April 15, 2024 09:23

vladbogo force-pushed the crossref_docker branch 5 times, most recently from 91555d5 to 6392884 Compare April 15, 2024 10:44

fauust reviewed Apr 15, 2024

View reviewed changes

vladbogo force-pushed the crossref_docker branch 6 times, most recently from b4c4e0f to 990998e Compare April 22, 2024 13:11

Add Docker image creation for crossreference

c177b12

vladbogo force-pushed the crossref_docker branch from 990998e to c177b12 Compare April 22, 2024 13:13

vladbogo requested a review from fauust April 22, 2024 13:14

fauust reviewed Apr 29, 2024

View reviewed changes

RazvanLiviuVarzaru reviewed Nov 8, 2024

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add Docker image creation for crossreference #412

Add Docker image creation for crossreference #412

vladbogo commented Apr 11, 2024

grooverdan Apr 11, 2024

fauust Apr 12, 2024

vladbogo Apr 12, 2024

vladbogo Apr 12, 2024

fauust Apr 15, 2024

fauust Apr 15, 2024

vladbogo Apr 15, 2024

vladbogo Apr 15, 2024

fauust Apr 12, 2024

fauust Apr 12, 2024

cvicentiu commented Apr 12, 2024 via email

fauust Apr 15, 2024 •

edited

Loading

vladbogo Apr 15, 2024

fauust Apr 15, 2024

fauust left a comment

fauust Apr 29, 2024

fauust Apr 29, 2024

RazvanLiviuVarzaru Nov 8, 2024

RazvanLiviuVarzaru commented Aug 14, 2024 •

edited

Loading

RazvanLiviuVarzaru Nov 8, 2024

fauust Nov 8, 2024

RazvanLiviuVarzaru commented Nov 8, 2024 •

edited

Loading

		@@ -0,0 +1,27 @@
		# Use an official Python runtime as a parent image
		FROM python:3.11-slim-buster

Add Docker image creation for crossreference #412

Are you sure you want to change the base?

Add Docker image creation for crossreference #412

Conversation

vladbogo commented Apr 11, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

cvicentiu commented Apr 12, 2024 via email

fauust Apr 15, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

fauust left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

RazvanLiviuVarzaru commented Aug 14, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

RazvanLiviuVarzaru commented Nov 8, 2024 • edited Loading

fauust Apr 15, 2024 •

edited

Loading

RazvanLiviuVarzaru commented Aug 14, 2024 •

edited

Loading

RazvanLiviuVarzaru commented Nov 8, 2024 •

edited

Loading