Elastic Agent ACI compliant image #3778

michalpristas · 2023-11-16T08:53:45Z

What does this PR do?

Bringing back work which allows

image to run on Azure Container Instances (no root group)
run cloud defend (heartbeat capabilities set)
smaller size (got rid of double chown and reduced size of npm modules setup for synthethics)
allow container to run as different user (get rid of world permission stripping)

Size comparison

branch
docker images | grep cloud
docker.elastic.co/beats-ci/elastic-agent-cloud. 8.12.0-SNAPSHOT  331c23d10995  5 minutes ago. 3.08GB

main
docker images | grep cloud
docker.elastic.co/beats-ci/elastic-agent-cloud  8.12.0-SNAPSHOT  a6e4e5b9ff81  6 minutes ago. 3.08GB

…nstances (elastic#3576)" (elastic#3723)" This reverts commit 29386eb.

…stom-uid

elasticmachine · 2023-11-16T10:03:04Z

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

Start Time: 2023-11-20T08:12:34.979+0000
Duration: 49 min 55 sec

❕ Flaky test report

No test was executed to be analysed.

🤖 GitHub comments

Expand to view the GitHub comments

To re-run your PR in the CI, just comment with:

/test : Re-trigger the build.
/package : Generate the packages.
run integration tests : Run the Elastic Agent Integration tests.
run end-to-end tests : Generate the packages and run the E2E Tests.
run elasticsearch-ci/docs : Re-trigger the docs validation. (use unformatted text in the comment!)

michalpristas · 2023-11-16T10:14:21Z

still working on -complete image size

…stom-uid

elasticmachine · 2023-11-20T08:27:45Z

Pinging @elastic/elastic-agent-control-plane (Team:Elastic-Agent-Control-Plane)

pchila

Aside from some unnecessary change of ownership in the earlier stage, the change looks good.

Just out of curiosity: how hard would it be to always create the elastic-agent user and group, associate all the files to that and as last instruction add a USER instruction ? (ideally with the value being an argument instead of a template)

The user could be specified at runtime by using something like docker run --user elastic-agent ... if we want to run as non-root without having to decide at image build time

pchila · 2023-11-20T12:47:28Z

dev-tools/packaging/templates/docker/Dockerfile.elastic-agent.tmpl

+{{- if ne .user "root" }}
+RUN groupadd --gid 1000 {{ .BeatName }} && \
+    useradd -M --uid 1000 --gid 1000 --groups 0 {{ .user }} && \
+    true
+{{- end }}

-RUN mkdir -p {{ $beatHome }}/data {{ $beatHome }}/data/elastic-agent-{{ commit_short }}/logs && \
-    chown -R root:root {{ $beatHome }} && \
+COPY --chown={{ .user }}:{{ .user }} beat {{ $beatHome }}
+
+RUN true && \
+{{- if ne .user "root" }}
+    usermod -d {{ $beatHome}} {{ .user }} && \
+{{- end}}
+    # ECE needs to create config here under non-1000 user
+    chmod 0777 {{ $beatHome}} && \
+    mkdir -p {{ $beatHome }}/data {{ $beatHome }}/data/elastic-agent-{{ commit_short }}/logs && \
    find {{ $beatHome }} -type d -exec chmod 0755 {} \; && \
    find {{ $beatHome }} -type f -exec chmod 0644 {} \; && \
-    find {{ $beatHome }}/data -type d -exec chmod 0770 {} \; && \
-    find {{ $beatHome }}/data -type f -exec chmod 0660 {} \; && \
+    find {{ $beatHome }}/data -type d -exec chmod 0777 {} \; && \
+    find {{ $beatHome }}/data -type f -exec chmod 0666 {} \; && \


Why do we need to create the user and the correct ownership in the intermediate stage ? Wouldn't it be enough to copy with the correct ownership and user in the last stage (line ~127) ?

hmm that;s a nice idea, will check, not sure it will let us differentiate /d/f though

…stom-uid

michalpristas · 2023-11-23T13:29:58Z

dev-tools/packaging/templates/docker/Dockerfile.elastic-agent.tmpl

@@ -27,7 +38,7 @@ RUN mkdir -p {{ $beatHome }}/data {{ $beatHome }}/data/elastic-agent-{{ commit_s
    (chmod 0755 {{ $beatHome }}/data/elastic-agent-*/components/pf-elastic-collector || true) && \
    (chmod 0755 {{ $beatHome }}/data/elastic-agent-*/components/pf-elastic-symbolizer || true) && \
    (chmod 0755 {{ $beatHome }}/data/elastic-agent-*/components/pf-host-agent || true) && \
-    find {{ $beatHome }}/data/elastic-agent-{{ commit_short }}/components -name "*.yml*" -type f -exec chown root:root {} \; && \
+    find {{ $beatHome }}/data/elastic-agent-{{ commit_short }}/components -name "*.yml*" -type f -exec chown {{ .user }}:{{ .user }} {} \; && \


this is a no, modules permissions are checked to be owned by root

…stom-uid

…gent into image-custom-uid

cmacknz · 2023-11-23T15:07:42Z

Did we confirm this image can be deployed to Elastic Cloud successfully and that ECE stack can come up.

I would also like @blakerouse to review this before merge to see if there is anything that will cause problems for the non-root/unprivileged agent work.

michalpristas · 2023-11-23T15:18:00Z

i created cloud instance using our regular testing/cloud env. this was fine.
for the use case lastly reported i haven't found a way to deploy custom image (QA labs does not seem to work)

michalpristas · 2023-11-23T15:20:21Z

i'm seeing some weird behavior but i see it on main as well.
i need to test this on another machine to rule out my env. i don't trust my mac

michalpristas · 2023-11-24T12:51:56Z

seems like some env issues on my side, rebuilt on my linux machine and it seems to produce system metrics normally.

blakerouse

Looks good to me

…stom-uid

…gent into image-custom-uid

elastic-sonarqube · 2023-12-01T09:41:58Z

SonarQube Quality Gate

0 Bugs
0 Vulnerabilities
0 Security Hotspots
0 Code Smells

No Coverage information
No Duplication information

michalpristas added 5 commits November 10, 2023 08:40

Revert "Revert "[Fix] Agent incapable of running on Azure Container I…

bb214ff

…nstances (elastic#3576)" (elastic#3723)" This reverts commit 29386eb.

Saved 1GB

78ceea4

Merge branch 'main' of github.com:elastic/elastic-agent into image-cu…

cc02083

…stom-uid

permissions seems ok

7836742

Merge branch 'main' of github.com:elastic/elastic-agent into image-cu…

577a9b3

…stom-uid

michalpristas added Team:Elastic-Agent-Control-Plane Label for the Agent Control Plane team Team:Elastic-Agent Label for the Agent team backport-skip skip-changelog labels Nov 16, 2023

michalpristas self-assigned this Nov 16, 2023

michalpristas added 4 commits November 16, 2023 12:39

reduce complete image

924b14b

complete compact

2e64699

Merge branch 'main' of github.com:elastic/elastic-agent into image-cu…

88b1bc5

…stom-uid

Merge branch 'main' of github.com:elastic/elastic-agent into image-cu…

f4eb9ea

…stom-uid

michalpristas marked this pull request as ready for review November 20, 2023 08:27

michalpristas requested a review from a team as a code owner November 20, 2023 08:27

michalpristas requested review from faec and pchila November 20, 2023 08:27

pchila approved these changes Nov 20, 2023

View reviewed changes

michalpristas changed the title ~~Image custom uid~~ Elastic Agent ACI compliant image Nov 20, 2023

Merge branch 'main' of github.com:elastic/elastic-agent into image-cu…

aaeacba

…stom-uid

michalpristas commented Nov 23, 2023

View reviewed changes

pierrehilbert and others added 4 commits November 23, 2023 14:53

Merge branch 'main' into image-custom-uid

90566a2

simplify dockerfile

ede4e4d

Merge branch 'main' of github.com:elastic/elastic-agent into image-cu…

db60d7f

…stom-uid

Merge branch 'image-custom-uid' of github.com:michalpristas/elastic-a…

bc7e298

…gent into image-custom-uid

cmacknz requested a review from blakerouse November 23, 2023 15:06

blakerouse approved these changes Nov 27, 2023

View reviewed changes

michalpristas added 5 commits November 28, 2023 09:32

Merge branch 'main' into image-custom-uid

666cb98

Merge branch 'main' into image-custom-uid

9fddf26

Merge branch 'main' of github.com:elastic/elastic-agent into image-cu…

15bd418

…stom-uid

Merge branch 'image-custom-uid' of github.com:michalpristas/elastic-a…

55b0367

…gent into image-custom-uid

Merge branch 'main' into image-custom-uid

e453c5c

michalpristas merged commit 8d4e3da into elastic:main Dec 1, 2023
9 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Elastic Agent ACI compliant image #3778

Elastic Agent ACI compliant image #3778

michalpristas commented Nov 16, 2023 •

edited by cmacknz

Loading

elasticmachine commented Nov 16, 2023 •

edited

Loading

Build stats

michalpristas commented Nov 16, 2023

elasticmachine commented Nov 20, 2023

pchila left a comment •

edited

Loading

pchila Nov 20, 2023

michalpristas Nov 20, 2023 •

edited

Loading

michalpristas Nov 23, 2023

cmacknz commented Nov 23, 2023

michalpristas commented Nov 23, 2023

michalpristas commented Nov 23, 2023

michalpristas commented Nov 24, 2023

blakerouse left a comment

elastic-sonarqube bot commented Dec 1, 2023

Elastic Agent ACI compliant image #3778

Elastic Agent ACI compliant image #3778

Conversation

michalpristas commented Nov 16, 2023 • edited by cmacknz Loading

What does this PR do?

elasticmachine commented Nov 16, 2023 • edited Loading

💚 Build Succeeded

Build stats

❕ Flaky test report

🤖 GitHub comments

michalpristas commented Nov 16, 2023

elasticmachine commented Nov 20, 2023

pchila left a comment • edited Loading

Choose a reason for hiding this comment

pchila Nov 20, 2023

Choose a reason for hiding this comment

michalpristas Nov 20, 2023 • edited Loading

Choose a reason for hiding this comment

michalpristas Nov 23, 2023

Choose a reason for hiding this comment

cmacknz commented Nov 23, 2023

michalpristas commented Nov 23, 2023

michalpristas commented Nov 23, 2023

michalpristas commented Nov 24, 2023

blakerouse left a comment

Choose a reason for hiding this comment

elastic-sonarqube bot commented Dec 1, 2023

michalpristas commented Nov 16, 2023 •

edited by cmacknz

Loading

elasticmachine commented Nov 16, 2023 •

edited

Loading

pchila left a comment •

edited

Loading

michalpristas Nov 20, 2023 •

edited

Loading