From 00e3d0dd7a30ae01cecac0e6d0a2d74cd7271833 Mon Sep 17 00:00:00 2001 From: Mukul Chandrakant Mahadik Date: Thu, 15 Aug 2024 14:32:13 -0700 Subject: [PATCH] Redesign Build Deployment Process (External) (#113) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Create image_build_push.yml * Corrected Dockerfile path * Added Dockerfile in repo root. Need to make it available outside the docker folder as the build context was specified in the docker compose as the repo root directory and corresponding paths in the Dockerfile were relative to the root directory. Additionally, added Environment variables defined in docker compose file. The image wasn't running directly after building because these things were missing: - needed to add db container as well - ports needed to be exposed for both db and dashboard container - common network needed to be added under which both the containers had to be linked - all environment variables had to be added, especially important ones like DB_HOST * Replaced config.py with config-fake.py in external Dockerfile. Encountered error while building and pushing docker image as a part of the CI. Due to config.py mentioned in COPY command in Dockerfile not being there in the repo online. Added config-fake.py in its place for now, so image is pushed to Dockerhub. But eventually, will need to handle as per decision to consolidate differences in external and internal repos. * Copy config-fake.py as config.py In the external public Docker image, make the config-fake.py available as config.py by copying it over into the container as config.py Internally, if custom config.py is needed, can copy it over in the internal Dockerfile. But might not even need the config.py * Removed expose port duplication Found usage of two variables referring to same port. Had exposed using both variable names. But since same port exposed, should be fine to expose just once. * Multiple changes for external repo differences 1. Changes in related files including Dockerfile for loading OpenPATH logo from local image rather than URL link. 2. Changed sed to jq in start.sh. 3. Changed dash version to latest v 2.16.1 4. Added new Dockerfile for image_build_push yml. Removed assets/ from COPY command as only contents of assets/ were being copied to root app directory. In COPY assets command, copy entire directory instead of just css file. * Added TODO to change image push branch Currently the branch specified - "image-push-merge" is available locally on my system. I use it to test the automated docker image push mechanism whenever any changes are merged to this branch. Once, everything looks good, need to change this to master or main as per the repo. * Removed printing Docker username Had added it initially for testing purposes. Can remove now so it doesn't expose any sensitive info. * Replacing CognitoConfig class with ENV variables - Option 1: Can we read in these values directly from environment variables instead of reading from a class? - Option 2: Create a class that makes all environment variables available as python variables For now, I’m going ahead with reading in these variables directly. So, will need to replace all uses of CognitoConfig class with variable name directly in files that use it. Shankari mentioned in a commit whether we even need config.py? https://github.com/e-mission/op-admin-dashboard/commit/927817a95a9f7f9c4307f98cbfe2ef02584c047d Also, removed references to both config-fake.py and config.py in the Dockerfiles. * Replaced dictionary in config.py Added actual template value directly where the variable was being used through the config.py file. * Removing ENV variables from Dockerfile The other ENV vars that were added in Docker-compose files are currently present in the Dockerfile in the external repo root. Removing them from there and can add them as per requirement. For local testing, add them when “docker run” is used using the -e flag. For usage in stage / production, can be set by cloud team in AWS Codebuild as they currently do. * Update requirements.txt Changed pillow version to 10.3.0 Editing online manually to test if Github action still works after re-adding image_build_push.yml file. * Removed sed / jq usage from start scripts Can directly set DB_HOST since we can now use environment variables. No need to use jq or sed to replace file contents and copy over files. * Changing base image to build from redesign server image Created a new branch image-push-merge for e-mission-server in my forked repo. Also modified image push workflow to build and push docker image to docker hub on push to image-push-merge branch. Doing this since admin-dash was failing when was building from internal repo since this was still referring to old server code. Changed Dockerfile and docker/Dockerfile.dev in public-dash and admin-dash to build from this new image mukuflash03/e-mission-server:image-push-merge_2024-04-12--01-01 Redesigned server image is built from the image-push-merge branch on my personal forked repository. This branch has a workflow run set up to build the docker image and push it to Dockerhub whenever a push or merge happens to image-push-merge branch. Currently, I've been pushing to image-push and then creating a PR to merge into image-push-merge. Doing this, so admin-dash and public-dash can build from the latest redesigned server code. This is not the latest server code but the latest changes from my redesign PR. * Bumped up base server image tag Changing to build from base server image from my personal Dockerhub repository with redesigned server code. Will need to change to build from Shankari's Dockerhub repository, once all changes are final. * Bump up base server image tag * Artifact download test - 1 Added working code from join repo to fetch docker image tags using artifact download. * Bumped up server image tag Bumped up after fixing "url" KeyError bug in this commit in server repo: https://github.com/MukuFlash03/e-mission-server/commit/e778b3f3cf050eec33ef439a1b6763b6aaf533e0 * Artifact + Matrix - 1 Combining both artifact and matrix methods since both workflow is triggered by both push event and workflow_dispatch event. Workflow dispatch event receives tag via input parameter sent via server workflow. Push event does not receive any tag and the DOCKER_IMAGE_TAG_2 would have empty value since workflow triggering event and hence input parameter would be empty. So, was facing the issue of having empty timestamp being suffixed to the docker image tag in the Dockerfiles. Hence, using the logic of fetching the latest run id and then downloading the artifact uploaded by server workflow to ensure that a non-empty value is also retrieved for the timestamp. This value is stored in DOCKER_IMAGE_TAG_1 and can be used for building docker image. Now, depending on the trigger event, the appropriate docker image tag can be used in the docker build command for the --build-arg flag. Additionally, Dockerfiles now use ARG to use the tags passed from the docker build command, hence using environment variables. Docker-compose files similarly have the args config parameter set. Developers would have to set the correct server image tags manually here for the docker-compose command to pass the value to the ARG in the Dockerfile and correctly set the image tag to point to the appropriate server image. Need to mention somewhere in the ReadME.md to refer to the server image list in Dockerhub to choose the latest image tag. While pushing the image in the GitHub actions, it'll be done manually. Todo: Change branch name to tags-combo-approach in fetch_runID.py * Artifact + Matrix - 2 Was unable to see docker_image_tag in echo statement in logs. Added docker_image_tag print statement to see retrieved image tag. Ah! Was using run_id instead of docker_image_tag as the output value in outputs section in fetch_tag job. * Artifact + Matrix - 3 Still not seeing the env vars echo-ed. Debugging by printing various log statements. * Artifact + Matrix - 4 Still not seeing the env vars echo-ed. Debugging by printing various log statements. * Artifact + Matrix - 5 Working finally! Changed the deprecated set-output command to use {key}={value} format with echo command. Also, adding commented out build and push commands to see if build, push is successful. For this commit, not changing branch = tags-artifact in fetch_runID.py Once, server code has latest working yml file, then will come back and push another commit to change branch to tags-combo-approach. * Artifact + Matrix - 6 Removed an extra echo statement. * Artifact + Matrix - 7 Updating Dockerfiles to use ARG environment variable with latest timestamp that will be passed through: - `docker build --build-arg` command in Github actions in the workflow for automated pushing to Docker hub. - `args: ` config field in docker-compose which will need to be set manually by developers locally. Also, changing branch in fetch_runID and Dockerfiles to tags-combo-approach. * Artifact + Matrix - 8 For public-dash, admin-dash where ARGS are now being used, need to add the args under build command in the docker compose files. Gives error if arg is at the same hierarchical level as build. Also, public-dash docker-compose.yml (non-dev) version changed to have build: context, dockerfile ; unlike only build: frontend. This allows adding args under build. Similar to how currently being built in docker-compose.dev.yml. Also, args to be added under notebook-server and not dashboard since viz_scripts builds off of server image and not frontend, which is a node image. * Artifact + Matrix - 9 Adding .env file which stores only docker image timestamp for the latest dockerhub e-mission-server image already pushed. .env file overwritten in both types of trigger events - push and workflow_dispatch. Added commit and push github actions as well for pushing latest changes to the .env file made via the workflow. Lastly, docker-compose now also mentions the ENV variable name to be read from the .env file for the ARG value in the Dockerfile. No changes required in the Dockerfiles. * Updated docker image tag in .env to the latest timestamp: * Updated docker image tag in .env to the latest timestamp: 2024-05-03--14-37 * Added TODOs in github actions workflow YAML file. Reminder for things to change as per master branch of e-mission-server once changes are finalized. * Artifact + Matrix - 10 Added another TODO. evious Push event triggers run failed Error occurred in GitHub actions git add, commit, push step. If file with no changes operated upon, it leaves an error: “nothing to commit, working tree clean Error: Process completed with exit code 1.” Need to fix. —— Quick fix is to make changes to .env file only if workflow_dispatch event is the trigger. Don’t do anything for push event. So, in case anyone modifies .env file on their own by using their own timestamp during testing, and pushes it as a part of their PR, then Shankari will have to ask them to revert the changes. Else, their custom timestamp will make it to the repo code base. Found something: https://www.reddit.com/r/github/comments/ju3ipr/commit_from_github_action_only_when_changes_exist/ It should work but there’s a drawback of using “exit 0” - it will mask all errors generated during “git commit”. This is bad and we won’t be able to see the reason why something wrong happened as the workflow would be shown as successful with a green tick. Found a solution with git diff: https://github.com/simonw/til/blob/main/github-actions/commit-if-file-changed.md $ git diff --quiet || (git add README.md && git commit -m "Updated README") However, I won’t be able to log any message saying that no changes to commit, tag not modified. Hence, will possibly use just “git diff —quiet” with an if-else block. Expected results: - Push event triggers workflow. - It writes DOCKER_IMAGE_TAG_1 fetched from last successful completed run to .env file. - It sees that there is a difference in the latest committed .env file in the dashboard repo which includes older timestamp. - Hence it runs git commit part of the script to reset to latest server timestamp. * Updated docker image tag in .env file to the latest timestamp * Updated docker image tag in .env file to the latest timestamp * Updated docker image tag in .env file to the latest timestamp * Update Dockerfile Reverting the Dockerfile tag for testing so that checks can run properly * Syntax Removing a space. * Updated docker image tag in .env file to the latest timestamp * Update Dockerfile again Also editing this file so that checks pass for now * Finalize Dockerfiile Changing this to build from actual e-mission server in hopes of merging! * Update Dockerfile in docker folder Potentially finalizing the dockerfile in the docker repo. * Update fetch_runID.py Removing prints/comments, updating head_branch and download_ url. * Cleanup image_build_push.yml Removing comments and changing branches so it's ready for merge. * Removing docker build, updating DOCKER_IMAGE_TAG --> SERVER IMAGE TAG, removing redundant db setting * Reverting for testing Need to test some changes to the workflows, so temporarily reverting these changes to work with mukul's branches. * Trying to trigger run without matrix Changing branches to try to trigger run and see if I can test docker compose * Changing repo name * Updated docker image tag in .env file to the latest timestamp * So close to trying docker compose with actions Runner couldn't find the file (because I used the wrong filename) * Build context change * dockerhub repos are case sensitive. * Updating tag * Rename docker tag Docker image created locally is not taking the name that I passed in for it. Hopefully, every new image created in an individual runner is named the same thing, so that this workaround is successful. * Add artifact Adding artifact upload for internal repo to be able to pull image tag. * Trying to use admin dash tag Trying to do this the way I originally planned. * Reverting changes Reverting the changes I made to test docker compose * Remove extra dockerfile * server image tag update * Update .env Adding other env vars as a template * Update image_build_push.yml Adding other env vars to .env. Otherwise, the file will get overwritten with the server tag only every time. * Update start.sh Remove extraneous fi * Switching to build prod compose Because that makes sense. * Updated username to clarify that env file is being updated * Modified tag variable names to be more relevant These store tags differently depending on the trigger event - Push OR Workflow dispatch * Removing redundant pip install * Certificates added in Dockerfile Similarly done with public-dashboard. See comment: https://github.com/e-mission/em-public-dashboard/pull/125#issuecomment-2101651604 * Fixed indentation * Added .env.cognito.template + Removed cognito variables from .env and workflow file The .env file is meant to be used only for the docker image tag to store the latest server image tag. If other variables are stored there, the workflow will overwrite the .env file to store only the image tag whenever the CI/CD runs. Hence, separating out the Cognito variables in a separate template file. I am expecting the users would either set them directly in the system or use the docker compose prod yml file. * Removing artifact method This was mainly needed for Push event but since Workflow dispatch event would be setting the latest server image tag in .env file, the push event can read from this file directly. * Updating latest server image tag This is probably the first and only time we'll need to manually change the image tag. Hence forth, the CI/CD should automatically update the .env file with the latest tag whenever the workflow dispatch is triggered on new merges to server. * Moving .env file update and git commit to the end This will ensure that if the CI/CD pipeline fails in any prior steps such as the docker related ones, the .env file isn't updated. This is because the the docker failures can include errors image not found which can occur due to incorrect tags. --------- Co-authored-by: Mahadik, Mukul Chandrakant Co-authored-by: GitHub Action Co-authored-by: Natalie Schultz <90212258+nataliejschultz@users.noreply.github.com> --- .env | 1 + .env.cognito.template | 7 ++ .github/workflows/image_build_push.yml | 98 +++++++++++++++++++++++++ app_sidebar_collapsible.py | 12 ++- assets/openpath-logo.jpg | Bin 0 -> 9970 bytes config-fake.py | 21 ------ docker-compose-dev.yml | 2 + docker-compose-prod-nginx.yml | 2 + docker-compose-prod.yml | 2 + docker/Dockerfile | 10 ++- docker/start.sh | 8 +- utils/cognito_utils.py | 12 +-- utils/decode_jwt.py | 15 ++-- 13 files changed, 141 insertions(+), 49 deletions(-) create mode 100644 .env create mode 100644 .env.cognito.template create mode 100644 .github/workflows/image_build_push.yml create mode 100644 assets/openpath-logo.jpg delete mode 100644 config-fake.py diff --git a/.env b/.env new file mode 100644 index 0000000..9305039 --- /dev/null +++ b/.env @@ -0,0 +1 @@ +SERVER_IMAGE_TAG=2024-08-12--15-15 \ No newline at end of file diff --git a/.env.cognito.template b/.env.cognito.template new file mode 100644 index 0000000..437bd68 --- /dev/null +++ b/.env.cognito.template @@ -0,0 +1,7 @@ +COGNITO_CLIENT_ID='' +COGNITO_CLIENT_SECRET='' +COGNITO_REDIRECT_URL='' +COGNITO_TOKEN_ENDPOINT='' +COGNITO_USER_POOL_ID='' +COGNITO_REGION='' +COGNITO_AUTH_URL='' diff --git a/.github/workflows/image_build_push.yml b/.github/workflows/image_build_push.yml new file mode 100644 index 0000000..2b780f8 --- /dev/null +++ b/.github/workflows/image_build_push.yml @@ -0,0 +1,98 @@ +name: docker-image-push-admin + +on: + push: + branches: [ master ] + + workflow_dispatch: + inputs: + docker_image_tag: + description: "Latest Docker image tags passed from e-mission-server repository on image build and push" + required: true + +env: + DOCKER_USER: ${{secrets.DOCKER_USER}} + DOCKER_PASSWORD: ${{secrets.DOCKER_PASSWORD}} + +jobs: + build: + runs-on: ubuntu-latest + + env: + DOCKER_TAG_FROM_WORKFLOW_DISPATCH: ${{ github.event.inputs.docker_image_tag }} + + steps: + - uses: actions/checkout@v4 + + - name: Set docker image tag from .env file + run: | + set -a; source .env; set +a + echo "DOCKER_TAG_FROM_PUSH=${SERVER_IMAGE_TAG}" >> $GITHUB_ENV + + - name: Print input docker image tag + run: | + echo "Event name: ${{ github.event_name }}" + echo "Latest docker image tag (push): ${{ env.DOCKER_TAG_FROM_PUSH }}" + echo "Latest docker image tag (workflow_dispatch): ${{ env.DOCKER_TAG_FROM_WORKFLOW_DISPATCH }}" + + - name: docker login + run: | # log into docker hub account + docker login -u $DOCKER_USER -p $DOCKER_PASSWORD + + - name: Get current date # get the date of the build + id: date + run: echo "::set-output name=date::$(date +'%Y-%m-%d--%M-%S')" + + - name: Run a one-line script + run: echo running in repo ${GITHUB_REPOSITORY#*/} branch ${GITHUB_REF##*/} on ${{ steps.date.outputs.date }} + + - name: build docker image + run: | + if [ "${{ github.event_name }}" == "workflow_dispatch" ]; then + SERVER_IMAGE_TAG=$DOCKER_TAG_FROM_WORKFLOW_DISPATCH docker compose -f docker-compose-prod.yml build + else + SERVER_IMAGE_TAG=$DOCKER_TAG_FROM_PUSH docker compose -f docker-compose-prod.yml build + fi + docker images + + - name: rename docker image + run: | + docker image tag e-mission/opdash:0.0.1 $DOCKER_USER/${GITHUB_REPOSITORY#*/}:${GITHUB_REF##*/}_${{ steps.date.outputs.date }} + + - name: push docker image + run: | + docker push $DOCKER_USER/${GITHUB_REPOSITORY#*/}:${GITHUB_REF##*/}_${{ steps.date.outputs.date }} + + - name: Update .env file + run: | + if [ "${{ github.event_name }}" == "workflow_dispatch" ]; then + echo "Workflow_dispatch: New server image built and pushed, Updating image tag in .env" + echo "SERVER_IMAGE_TAG=$DOCKER_TAG_FROM_WORKFLOW_DISPATCH" > .env + else + echo "Push event: Restoring latest server image tag from .env" + fi + + - name: Add, Commit, Push changes to .env file + run: | + git config --local user.email "action@github.com" + git config --local user.name "Github Actions bot to update .env with latest tags" + if git diff --quiet; then + echo "Latest timestamp already present in .env file, no changes to commit" + else + git add .env + git commit -m "Updated docker image tag in .env file to the latest timestamp" + git push origin + fi + + - name: Create artifact text file + run: | + echo ${{ steps.date.outputs.date }} > admin_dash_tag_file.txt + echo "Created tag text file" + + - name: Upload Artifact + uses: actions/upload-artifact@v4 + with: + name: admin-dash-image-tag + path: admin_dash_tag_file.txt + overwrite: true + diff --git a/app_sidebar_collapsible.py b/app_sidebar_collapsible.py index 9bd342a..da74ade 100644 --- a/app_sidebar_collapsible.py +++ b/app_sidebar_collapsible.py @@ -17,6 +17,7 @@ from dash import Input, Output, dcc, html, Dash import dash_auth import logging +import base64 # Set the logging right at the top to make sure that debug # logs are displayed in dev mode # until https://github.com/plotly/dash/issues/532 is fixed @@ -29,15 +30,17 @@ import flask_talisman as flt - -OPENPATH_LOGO = "https://www.nrel.gov/transportation/assets/images/openpath-logo.jpg" +OPENPATH_LOGO = os.path.join(os.getcwd(), "assets/openpath-logo.jpg") +encoded_image = base64.b64encode(open(OPENPATH_LOGO, 'rb').read()).decode("utf-8") auth_type = os.getenv('AUTH_TYPE') if auth_type == 'cognito': from utils.cognito_utils import authenticate_user, get_cognito_login_page elif auth_type == 'basic': - from config import VALID_USERNAME_PASSWORD_PAIRS + VALID_USERNAME_PASSWORD_PAIRS = { + 'hello': 'world' + } app = Dash( external_stylesheets=[dbc.themes.BOOTSTRAP, dbc.icons.FONT_AWESOME], @@ -59,7 +62,8 @@ [ # width: 3rem ensures the logo is the exact width of the # collapsed sidebar (accounting for padding) - html.Img(src=OPENPATH_LOGO, style={"width": "3rem"}), + # html.Img(src=OPENPATH_LOGO, style={"width": "3rem"}), + html.Img(src=f"data:image/png;base64,{encoded_image}", style={"width": "3rem"}), # Working html.H2("OpenPATH"), ], className="sidebar-header", diff --git a/assets/openpath-logo.jpg b/assets/openpath-logo.jpg new file mode 100644 index 0000000000000000000000000000000000000000..1e13c576eefa039050c6412a498b35079138944b GIT binary patch literal 9970 zcma)iWmFu^^7rBrAh;7e=pw=0ZE=TPLXhANVX;7P3$DR~1^2~62p-%QcM0wm^iQ69 z@AK*X@YXrgHK(hmx@W5DH{D(RH2bsw2nHY{{iWv;85Q~YLPJ4DMnOYIMMXnJMaM+P zcqVjAOsv23Tw-HmVd4EH!hZ2LWjA)q|J4k8jdE&u@u2@x3)1qBTU1N&Kw@Z27Nf`^KaMnKCgrAbI<>JorX^hP?K zn1nttw_`#^R!dvg%-k(7p$;}a$-v03=la3DrjwM1S3u`|P;h?j+R5qjpq{m$pkkpQ zBO?9n@^AOZcqsUPHJZA_qY}_b2aFTH(K2&Q$gQdESUVvk(srAm*9ko3;bmaVdzuGe zJwJzlhlmG|0JKlZQSZmF&VMKiopW;ft$Y^CE@zz(3a<=cV?j=~y5x;s_S^*z7dc5M zi!KPe@L^M29grSDRFnH}>H_aZ1zs9X@Y|oqP)s6tEV=vEiIg~cd;?z95}S9(tn*Y6 z!!=;E1>Sc7$fmYS2Nj;fY6`*T;xu+H`F0=DEdivbd=SZ`_|zyGnNZ_<*8N8RN}2@c zRpyq74D%O}WYgHgoPtS;V4+zH3y&1gUSF6L0C0& z3C?mCcy+lH?Gp+-jt{eQQNIQE^;DX@Gq>b;`U1-${IVewRq1&gyZMv0Ji72mI_B_y*wa|kM|F4+ zXkLu`MTQfBmJEnWSrsSJZDlpb=^f^@myF#bCN`XUPXZXEbaV?$=&X|4>e&6Rc=LiO%) zUM-taXC*nVD+m^OeSB*}fqf=+9+}@vyZX^M?&C@8A@rhU+4(8@+?OmfsLq8=`OUIP z(E|+E3jsPOI7z;H+FXSo0nxN~+8+&g_C|4z2Bt6e!@0EukU1`|E zrsC3GOD=so$p$m&EajHV*B!aH?&dh-p^N(+ra$_!#hztJY3tB%JL=Upepp3!1EUso z0LzyzOS|0zM7<9;F5Vux_{cQIp0MPgYCmjQHmszOQ+4gtyzMX9fICs=9XLgMQ&E@y z{z#^9ot&_&1;WNz6`AQi1AAVkbzgJLX4;NV2_?8eF?G7i6mk}hIh_0)-Ye^OB-Nt}$|x}Qu@ukU{0cYC`zv!HwMC$(%c#l26Z5($2C+}o$OG%oroRy&|q%Ub`F zh)n)R8Ry)kZsffebl1ix>V2O9ur*+1T8}5z()Lk$E-9;%pu;1qKEU#a5ukWym2Dedw9~@-okS+i{q=h<%O8YVq^y(SNAI2{a!gCtY3$TxZ3=5Q=qHz5Y-Pgj zTiXA?g5&aBT?w_1Pbx{+@HzQ}oDo?j7}ilX*4JV(cUp|p?0*=a@}cFigjn-TA1bwk zWb0$+Rfr+3*m*Q^lt0?uap&Bqcr063DGE0vHc~rl)|&vw31v0$WZ&v7*xwCpS2Q&3 zCkri?*uau;9dijyY!eM~>rKuV9h}6)74&cTKzyiB5%) z8hG^bhHcM_NNh`H{dm1vu8p%~dF0}=`#5>t$b3#RavPZtt2b-&D#E%Jz<|2H)Nn*y zecE~c8)ft=mXX~S_M=4aIVJdn(s2Rsnp zux632=E>AkVQS7tIroubdz!31Tqyh^T(iwsDhf?sdM&JxjDfz*iO#Aq;1aXD4{tJJ zvCf=YNWZH@2c{MF%8e>QJ*mYr8byv7sZI=r=GLru&2lgKxnFl^utc z7Q>V#4r%uZlHuHA3gzY|i=sTcwtNi39)<5*vNg;&61J0J5(IGjCK967#17a$dPxwQ z7k{ZCjBwa51|6=WRg!5LGWuv`BcUoIy$&xdBlZJcua2)b$|(5N6$(j}>8iEmaZ2Se zuNdzju#5Os^F-v&q@mRKpPWL>J4U%L+aCV3vk1MRgPR$AICn#XSfAI&G!|x0TYzU+ zFj;nXpcP?Otx!!any8rRe)#6IGFF|u@S4|!py9f)yZwWq)C7{JDVP()VbAQ$;vz*l z3a?pdFr+Kn-@ghcw3%H=kukpNW=TcOBlgvFa|2IT?$TgIO9X)tcmJ9S_ZDO9z~25Y zKGopsSEt9U@ILo_&E?N+9Ws379FdW9TGa-Q-q{B|DPuElAj`FjdS3Q09K!uzD9CTZ z8>c>%g$3W1ISk9p6QTUz380Mg92y1m#4Py=`0m(04hCRg0eSET-MTS{udR#|B*H%a zZaL04zo`<$OVMOw9hLUs?irXMD!7@NW8I(MZa7g3PtO8t z^_C@hIvP3Rw1ijFzzB5?!rWlqxe5Tpe1A0*)k2XQ1A#8(OjUB!1RX;yL|3uS)n0$2 z-&EMtA2efEJR+#KO-E08KDAaTbdo&*4j*EVAUQ#LPS@gak1D^rMa3ro&&zeoCxC}K zI@1%N@k*jul)SFJ`Kqo>h*tss66NYk9_Ahr6f8!dg^MvpahgI8?DW2&FqfU&AueOd~Y=;AnhUiA30pO0#??P39QQU2Gx zS*TAVa~y7N{CqR(5_4OLTm&Dh9>Y|XBzuOKINy~e2=i&Wnbm~$bH>L)&djXVpU6xz zBdRuPVe)c>6>D8YAF1@YW;ixS*Ud#MNvd;LG@k&4y8822trjqcS7!|`5T;?HdB?7S zYzlA1m}l+v#gyGGhyN2acK@|{SO?2}(bs)L*M#`jdEw4IN~@@Kbr0-9{lB#0q71%8 zQ!1UwPaX`FN#w(P^W_$NQD+l2!U&lOHfn1UFA`AK5Qy=I-&jvAn0_ex1X6r(oo1rZC z=0iVOFB>U%GGUyYiMGDnt8hCrW>I~)-jUr#sLAP#3*}A}Nu{7xzU4db#43$X$-)W` zW=@w(w%bmRL)gbRm<5T?u*AJnw_)o#UHi>ANLI7AeT>co`e15xZ<=*(n*Xo$kB>*Z zl>2K4$s#N-&fOHALEs1Q&mq5VnQL5Lc1%#2>DUzUGut)Q<5E{#2Buc z9k%ig`u=-d`xE^83sci(DFhKcv5&t6m)3ODLalFYMkA|nh%h30djEXnlm1xP`xrOP z2wP~dzQ{O+NW6}A;5_wcmV$~fj%qHE)y zFiyXa_||Wv&h#%zjz%4FNO)nth?s{K*xWR(o1jH5VN>0XByFHp5P@c~@%S$z25yPzU4Z)@-z=__MKQgTSO$<}DtHl@dIW+Lj`s71A zZ)`mIMX9k@Us5^Ys1m(MY+2YpGEdiGFBj0lZdEN9~3 zi=#@gbm8xDZQp@A5T^tsET>VRD4;tabn2{+25JeX7u6T6tAO|kxg*?sxQ7lfA3L)` z*!azQc&+UkGr5p#-h$OZ;13}%$%Z1I{myTr1_--2FUy~8S^SSJ_4ItRo&mep^7)77 zax}%ux&WGd2JCwkz$qm6FJPx^4%kxaZ;Ly3L#Ot?9M>Z;k)86TaZ^>(3e(q`aEL{= zP+%#9MlD+BqpUPFdWSw1RMWBB$w1)UD7h~n!}VDmBB5IwRM$*QJ2Mg~Fo%d=^0uUQ zraO)asQ2o`Q=5$`N^`icXcRcyMjT7NGO<@v;!NF)rPsIF6dANJ<|xFsX>N02uqYpH zh$!vXONKCsdBZm>z=%*Ixk4OShZm3r_DR||rg|rZU?`Q3yqrbZ$h(R5!`Uam^rn7> zvfw{1XJ4NGIQZi=vwrvs^(CrPkxrsrN0S_+$*Rt#tWt$-yeEKjWr+F99hnPhnVH*k zoWgbimk8S5<^p)(B^R*5M#MQZuTCXWO_8?E>7F9-{^E{9hR+pL>^Nkn!$z|coXP$M zs|Xx!ErL<5YYAU?4N?0$&x8Xb3?%*tlzpsgUw?)T&VWIw`cK>0%PY~P+7%r(FUG7t z&(mFy(}jDsep_3}aMF>sev@jfETEc&*Ss9==`W05r3<^4vA-X$6!_Ejz z0rVX*=Q9nFn4ApLuW5CeBQ)o8aAZkh$de}K43GV5qV7fDyhE*)G(QMxeUv>zE<$j- zJUxdCUSZ-UR=PVzhw&)z4=VeH2$6;9B$Lc@WM_WGA!&tfpa0%4O3{L6)vXCD2I%3B zxfEAcxDU5pROVy$CFb~8M)2~wkiO-eDNWjS)T(;P+X)Lzj?GrqZ?G{JvgIqwp z3?MkEOg2~a!_loCjdFpWFC~mu49cpjw$Pj^$o&YPTu7hpRxVd&V8h;q9!m###hl;6 zlB=8;H!l?Tyi0%ZPeFgaXO20A;1!&>-p#%_{t`hqC}yY zE8=z8DDx&p3EGq%eL{o(uf`ehWbX*Mpa5+ICRwj6`(-g@LER;_rjKX}EHveIpY0Pa z?{W>C+!8xjJh{uLFH?Dl7@vZhIG|BXq^#aM$aZi>Oqx>q#YDDf@H6>WT{Z-<6X{w@N#l7s4wSj*|fIIt)2>Mz2)hwPKOLSzN- zc9QZYE56UawXt!jkIJ>CRiYol8-tr0QL=KsAQ}ep3KC_3&f@9|+gcmq+ZL9{l*e*@ zKAdUjz5^qJF($*Eo?~mry3%tlwBP1GK4M4`!~H51%nvH@@Tt~3 zEn7bfwJ8BdK?c&d)V`i39eTu7Pvg&fL>*AUi2Is9I}uc@c_Gbgc}XGe6I>kWp(~~6 z>LdJcIEBSeFHX_;FZ*teIOLj4MV~^nRSdcERcqhh3{ieed=$LnV~g<7V<$66I2?GL zWvuZ8U>9qdBvD`S4EoAx$w_hvY*5Q-X(6pu1c3^E<$^@s^d z%EIOJky-7UrO`>U&%3Q1Ui4eoQI*{HK4A>*h4-zp@=yv=$ikKEH&0!;KY-_z=~sj z@oQ-2_5DaoK!v%hpH$K9G0~Y_9nWGq&{;uy5%~C}urq|V`&h2+B=2`MP!WZNX%Mr= zY|vHBW~_>$f~nN)P*|+YM6@&|1pkn7Rk&*t$FDo}E1%p-SWB@%Wyi0pqW9J%U*i}b9jbji){pqRv5CYZ!L*JYOFe7gIKw z7@DA<5gaH^)R__YF4x#wj=3l4+`%51Flf?w1TEPc3&m76c*3^*X`+2fHp^Qu>>l>$ zO(zX1&UIPlqsme1lg${)QLyRMng!{sPbs+K$El%3lj8%}xW}M!0GwlDgCI^is z#>Mil&jP>8dycuVIZxd>VAaqrOc5IXr? zaP`;?)Pv9Z0%v%~g1#2-N4dc)3zNS!`8u~KT0Xp4?{2b$ce&{eBxUlADrvCDea35zqr9olT+woQU7+(nepLUBGopaZ7jHKAEkDsyfN z7|xAe%os~y1B#*j03qt+E&OS1y`7a@<(+J1_m+)+>AcLtdbk^7@mOpM*6`~nIad}} zA=y2g7*=3lv|+}7G$}0*@~yjeJB=WNYf-HOe8k$a*Z*r~QBFIBki}t<2%boQ26~tM^2`YV zh2YIs;y1y2wywx*$9f`NS0UTQC7X&dYpfV9HG!%41g)-WldYr`;DzdTE;l;9Ys!Hr zIVrSl%vpOo6Gfq!4ByHrYuij7PwK9PMMbQ48%jB^k);4iDj|K#%a=_`(!y9Jwl}() zNo5u${4Z2xLH>R|g%1=-7EY&u)O6f4FmE>+CwTYsD*#D|NGt(`h zxFLzzed{+)+kGBC)9lNv4?mRWHj&;O3JPrdxKfqLZaEZSGaBX>Jls;ur7bjr$n$mP zFOqBY+0r8xyeT2BpVC7++*Y%_dzA&SV};o| z$b6Xg(C3)Zz1_5rB||>@Vkd+y?>EiX65H9dc$rJ=$`V0PZEZaz?mc4~ml(o?E)yz{ z#UVWU%UZFprK7Y4#}Kws>U()*+TrDv`ZV?I35mrqz(0?864{ZM4*aXXMx7|Vbc!`A z@_v_G7=QOrmMEfQk~P+AJS2I+>p~R1Yg*W{kwCQXV39Mwt(AO>5Cunsy5Qi{niAuw1#W=M&%+@7ppp!%LBB4p4xf zw~=Q>ZbNgwZkrU28-g)_(YO*DzieT2fI@E80j`|+i$s95T~a{N#-<^1tkVd1=*EG? zM&HE?s+f=6Ldl1f@0?gkHn-8KkT&v1LDRLocr=V0nB0Qk0Pkf&S$J|j-9Fnx-d5ov zlfquZa6kS;rZ4y zGH!&s-_FVRJIrA;n>*^xj?c6r7c1^Lu@gVX2*c%BK_Vr~17Ce{rXV;$69Qh6_x#ydi!Ke~%Em@o&y z$?+(~BImL3;*TTA69rpqgECDclLcAro(v&81_%Q`#cZ|t4upJ>;DtQKT{B~1mO~aL z4229)aFsWT8aqkvp5V{lJ*;#DX!uA(eZ;Gw&DY4P!;!L>$C1Pl-kAGAuI)v!)?fZS z*Gz6^&EpW6J|hZ5JR*o-K{IDB+HF}o2!3>1Wv!fqt^Rr*KNOooIBfX^hnDU+HwO=F z2MHW&x$<+Z%%_)e6-;k&1q=V^48d&9SVl5xE? zd&d`xsOsuZ0P6@UVWEKA-A}{R3vz@If`U2q{p2SAuO@RtkqRtl2J_K6Pt%h3T;|gK z({p7V@$0_5fjboZ&)gRFe&townxG-DhJ>z!Ngn&8gXDw8Ly|1K&H3P+Wm?_QD05Wd zv(w5r&^+j1Jrp%pa7=8DZfv;Kn2VTv&6m$T7fSYJv-2ZMCGCLC;6n?Z0BwXXmjv30 za}G;K5mq0|nov^LknRI|sJ)q~Tb!OVWDDy5R3o1RJW8&|{@c39|KyYIBBJMWJkq#G zt`W71IjWoAz@7jOqG&<4m8*pRk*Ki|hg2IFsic5)B8vCY@X^5upS%GWukNiDAVgXH z{X$G=FcMMstY`siO164ERdD=9B!T=M#vp{X^xgAwwTH|uqzh)qVd!14810gRQ)G3zpx|t!nEA+wY~@mWcbZqNZnDpbwM0J+j%Mh3eeE!}1ulgp1v=SJ z(0gI!$m`4v@qr6MBHK0`ny_m7u-2 ztza>Q44Tjjj*Mp>GS%QWV%}txe8`&7uu(O+tDA-Oq~BS`twAaj8yP_mMHfTH7Z@~1 zW0dC2Lqse%OT`@DrEGVL-6ilIw|_U(zaF=Cm?5=u)ox?5vs0874n{Eri7AZ;4}{E- zM3Oh$byBQTwT8jpCK3}Ei61>_xOkb>1x8Qt;bCjW-{5| z05R(0JET@%bYyh2Hf;THtjr=7HBd(kYa?dXE;Cfxi>u0?sT5m}BzaY1*@G4a#=ZkC z)87Wpjp$s_i!LNk81>w0fhH0Zx0vn46$r%3d-Uf-U78jB>v6oX&ewSC?~aBi73$;evf4IMIIap+Sn)nQ=p4{b9a zOPnZ_p6kwLaG}@U^>Fac0I^QmY99i%Rg~-jFnq17QT!Zm__xu-;|E>CCqPu{gUx(k z0q4_X&G(OM7AtyRyz^Cfc1b*eq-X0i!7ck3nqb0Z#eq0boAKtWEP1RD6dHlPVXt$6 zp)iGh<0|Z1!4%vpp{72r-Yv(ayQQ1axB+ zNlXhhpc<~?3qy?EOn>1^L1dK}9J__bpn7AAp2;fXg9IuE4iVC%k_aW~29!h4gD%p# zsIZq#GKNQ(DDSlQ$X3avV_|riEvLp2wyaJPOI~v-VX>34)eLrHs}Oyaid7}1wq+km z1>m5J2~QAuxVCI#WYf=)D`|q5RDqz{5|f6)lsozfG5DxWIt04Lcx(J+{LP2OG!Jz9 zF99#VZx$ag197l_p#dr+4@qBH=7QX8_AUCW65qGl#jVJ7_hcf=@m8O*rmP{EQnUnR zPfbJ$!)B|ZotZQ`T<3rUMr{fZm!z(MX&T!2CYmCK566+on69h7xwtnj=0?8|#fA~p zE`>*dr)eA;BL4cCu_01ggf4xb{l3D}ueu5=su`Odq9|xEUy{zWgxG2*n<}Bm3X+VAn}w%5s$8{Zm8l!gH+T9Gd`=Ic#Ro%J zwY((XqXfiD2MFj}wLSp`-Wxq2st^B2dil-OXw!WAp?TZf$uDa+moVTgD1?8lD|JIM z=AZ=C$c)?+)5Q~0NJIAcxU%%%;qt09DwR<+zQVT}yrFD8{M+5XSs zLlggXEd?y}t9n+0$rE7gEuPFL#w3TDJ%Y?_olgVf64?dCn4gNZ{yq2e-{U|3u=c09 F{{ysm_00eP literal 0 HcmV?d00001 diff --git a/config-fake.py b/config-fake.py deleted file mode 100644 index bcfc85f..0000000 --- a/config-fake.py +++ /dev/null @@ -1,21 +0,0 @@ -# This is a template for the "config.py" file. Put all of your valid data in this file and then change the name of this -# file to "config.py". -import os - -# Fill these variables when using AWS Cognito authentication; otherwise, keep them empty. -# All the variables of this class must be filled in using your credential data from your AWS Cognito panel. -class CognitoConfig: - CLIENT_ID = os.getenv("COGNITO_CLIENT_ID", '') - CLIENT_SECRET = os.getenv("COGNITO_CLIENT_SECRET", '') - REDIRECT_URL = os.getenv("COGNITO_REDIRECT_URL", '') - TOKEN_ENDPOINT = os.getenv("COGNITO_TOKEN_ENDPOINT", '') - USER_POOL_ID = os.getenv("COGNITO_USER_POOL_ID", '') - REGION = os.getenv("COGNITO_REGION", '') - AUTH_URL = os.getenv("COGNITO_AUTH_URL", '') - - -# Fill this variable when using basic authentication; otherwise, keep it empty. -# This dictionary contains all the valid usernames and passwords that users can authenticate with. -VALID_USERNAME_PASSWORD_PAIRS = { - 'hello': 'world' -} diff --git a/docker-compose-dev.yml b/docker-compose-dev.yml index 75f8df2..df44e28 100644 --- a/docker-compose-dev.yml +++ b/docker-compose-dev.yml @@ -5,6 +5,8 @@ services: build: context: . dockerfile: docker/Dockerfile + args: + SERVER_IMAGE_TAG: ${SERVER_IMAGE_TAG} image: e-mission/opdash:0.0.1 ports: - "8050:8050" diff --git a/docker-compose-prod-nginx.yml b/docker-compose-prod-nginx.yml index d4d6abf..55fbde6 100644 --- a/docker-compose-prod-nginx.yml +++ b/docker-compose-prod-nginx.yml @@ -6,6 +6,8 @@ services: build: context: . dockerfile: docker/Dockerfile + args: + SERVER_IMAGE_TAG: ${SERVER_IMAGE_TAG} image: e-mission/opdash:0.0.1 environment: DASH_DEBUG_MODE: "True" diff --git a/docker-compose-prod.yml b/docker-compose-prod.yml index 8b50892..eb284fd 100644 --- a/docker-compose-prod.yml +++ b/docker-compose-prod.yml @@ -5,6 +5,8 @@ services: build: context: . dockerfile: docker/Dockerfile + args: + SERVER_IMAGE_TAG: ${SERVER_IMAGE_TAG} image: e-mission/opdash:0.0.1 ports: - "8050:8050" diff --git a/docker/Dockerfile b/docker/Dockerfile index 188b1ac..8224c25 100644 --- a/docker/Dockerfile +++ b/docker/Dockerfile @@ -1,4 +1,8 @@ -FROM shankari/e-mission-server:master_2024-07-19--34-43 +ARG SERVER_IMAGE_TAG + +FROM shankari/e-mission-server:master_${SERVER_IMAGE_TAG} + +ADD https://s3.amazonaws.com/rds-downloads/rds-combined-ca-bundle.pem /etc/ssl/certs/ ENV DASH_DEBUG_MODE True ENV SERVER_PORT 8050 @@ -17,10 +21,10 @@ COPY ./pages ./ WORKDIR /usr/src/app/utils COPY ./utils ./ WORKDIR /usr/src/app -COPY app.py config.py app_sidebar_collapsible.py assets globals.py globalsUpdater.py Procfile ./ +COPY app.py app_sidebar_collapsible.py globals.py globalsUpdater.py Procfile ./ WORKDIR /usr/src/app/assets -COPY assets/style.css ./ +COPY assets/ ./ RUN mkdir qrcodes # copy over test data diff --git a/docker/start.sh b/docker/start.sh index 05b2f12..8500a69 100755 --- a/docker/start.sh +++ b/docker/start.sh @@ -2,13 +2,7 @@ source setup/activate.sh # change the db host -echo "DB host = "${DB_HOST} -if [ -z ${DB_HOST} ] ; then - local_host=`hostname -i` - sed "s_localhost_${local_host}_" conf/storage/db.conf.sample > conf/storage/db.conf -else - sed "s_localhost_${DB_HOST}_" conf/storage/db.conf.sample > conf/storage/db.conf -fi +echo ${DB_HOST} # run the app # python app.py diff --git a/utils/cognito_utils.py b/utils/cognito_utils.py index 59203d4..17f2e30 100644 --- a/utils/cognito_utils.py +++ b/utils/cognito_utils.py @@ -4,16 +4,16 @@ import flask import requests import dash +import os -from config import CognitoConfig from utils import decode_jwt def get_tokens(code): - client_id = CognitoConfig.CLIENT_ID - client_secret = CognitoConfig.CLIENT_SECRET - redirect_uri = CognitoConfig.REDIRECT_URL - token_endpoint = CognitoConfig.TOKEN_ENDPOINT + client_id = os.getenv("COGNITO_CLIENT_ID", '') + client_secret = os.getenv("COGNITO_CLIENT_SECRET", '') + redirect_uri = os.getenv("COGNITO_REDIRECT_URL", '') + token_endpoint = os.getenv("COGNITO_TOKEN_ENDPOINT", '') encoded_data = base64.b64encode(f'{client_id}:{client_secret}'.encode('ascii')).decode('ascii') headers = { @@ -59,7 +59,7 @@ def get_cognito_login_page(text='Welcome to the dashboard', color='black'): dash.html.Label(text, style={ 'font-size': '15px', 'display': 'block', 'verticalAlign': 'top', 'padding': '15px', 'color': color }), - dbc.Button('Login with AWS Cognito', id='login-button', href=CognitoConfig.AUTH_URL, style={ + dbc.Button('Login with AWS Cognito', id='login-button', href=os.getenv("COGNITO_AUTH_URL", ''), style={ 'font-size': '14px', 'display': 'block', 'padding': '15px', 'verticalAlign': 'top', 'background-color': 'green', 'color': 'white' }), diff --git a/utils/decode_jwt.py b/utils/decode_jwt.py index 8f565bb..dca5c02 100644 --- a/utils/decode_jwt.py +++ b/utils/decode_jwt.py @@ -14,15 +14,14 @@ import urllib.request from jose import jwk, jwt from jose.utils import base64url_decode +import os -from config import CognitoConfig - -client_id = CognitoConfig.CLIENT_ID -client_secret = CognitoConfig.CLIENT_SECRET -redirect_uri = CognitoConfig.REDIRECT_URL -token_endpoint = CognitoConfig.TOKEN_ENDPOINT -user_pool_id = CognitoConfig.USER_POOL_ID -region = CognitoConfig.REGION +client_id = os.getenv("COGNITO_CLIENT_ID", '') +client_secret = os.getenv("COGNITO_CLIENT_SECRET", '') +redirect_uri = os.getenv("COGNITO_REDIRECT_URL", '') +token_endpoint = os.getenv("COGNITO_TOKEN_ENDPOINT", '') +user_pool_id = os.getenv("COGNITO_USER_POOL_ID", '') +region = os.getenv("COGNITO_REGION", '') keys_url = f'https://cognito-idp.{region}.amazonaws.com/{user_pool_id}/.well-known/jwks.json' # instead of re-downloading the public keys every time