Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use endpoint-specific service for discoverable endpoints #1903

Merged
merged 5 commits into from
Oct 9, 2024

Conversation

AObuchow
Copy link
Contributor

@AObuchow AObuchow commented Sep 20, 2024

What does this PR do?

When using the discoverable attribute on a devfile endpoint, a service is created for that specific endpoint that's associated with the route/ingress. However, the current behaviour of the Che Router always associates an endpoint with the workspace's common service, rather than any endpoint-specific service.

This PR fixes this behaviour, by ensuring the correct service is provided for each endpoint's route/ingress.

Screenshot/screencast of this PR

n/a

What issues does this PR fix or reference?

Fixes eclipse-che/che#23061

How to test this PR?

  1. Install Che using the changes from this PR by running:
  • On OpenShift: ./build/scripts/olm/test-catalog-from-sources.sh
  • On Minikube: ./build/scripts/minikube-tests/test-operator-from-sources.sh
  • Alternatively, I've built a Che-Operator image from this PR and pushed it to quay.io/aobuchow/che-operator:endpoint-service.
  1. Once Che is installed, create a workspace using a devfile that has an endpoint with the discoverable attribute. For convenience, you can use my reproducer devfile.
# Example endpoint with discoverable attribute
- exposure: public
  targetPort: 8080
  name: https-python
  protocol: http
  secure: true
  attributes:
    discoverable: true
  1. In your workspace, start up a web server that uses the discoverable endpoint. Using my reproducer devfile, this can be accomplished by running the pip-install-requirements then run-app devfile tasks from the CheCode UI. When the endpoint notification pops up, click "Open in New Tab".
  2. The web server should be available on the route provided. With my reproducer devfile, you'll see a page with Hello World!

For comparison, here's a comment displaying the behaviour without the changes from this PR applied.

PR Checklist

As the author of this Pull Request I made sure that:

Reviewers

Reviewers, please comment how you tested the PR when approving it.

@AObuchow
Copy link
Contributor Author

@tolusha please prioritize reviewing #1881 before this PR, as #1881 is of higher priority for DevSpaces 3.17. Thank you :)

@AObuchow
Copy link
Contributor Author

@dkwon17 this PR has now been rebased onto the main branch

@AObuchow
Copy link
Contributor Author

FWIW: I confirmed with @eye0fra that the che-operator image built from this PR fixes the issue described in eclipse-che/che#23061

@AObuchow
Copy link
Contributor Author

/retest-required

@@ -474,12 +476,38 @@ func getCommonService(objs *solvers.RoutingObjects, dwId string) *corev1.Service
return nil
}

func exposeAllEndpoints(cheCluster *chev2.CheCluster, routing *dwo.DevWorkspaceRouting, objs *solvers.RoutingObjects, ingressExpose func(*EndpointInfo), endpointStrategy EndpointStrategy) *corev1.ConfigMap {
func getServiceForEndpoint(objs *solvers.RoutingObjects, endpoint dwo.Endpoint) *corev1.Service {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
func getServiceForEndpoint(objs *solvers.RoutingObjects, endpoint dwo.Endpoint) *corev1.Service {
func getEndpointService(objs *solvers.RoutingObjects, endpoint dwo.Endpoint) *corev1.Service {

WDYT?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1 good call :)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

wsRouteConfig := gateway.CreateEmptyTraefikConfig()

commonService := getCommonService(objs, routing.Spec.DevWorkspaceId)
if commonService == nil {
return nil
return nil, fmt.Errorf("could not find the common service for workspace '%s'", routing.Spec.DevWorkspaceId)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we order functions in the file like so?
exposeAllEndpoints
determineEndpointService
getEndpointService

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I assume you mean just moving the functions around in the che_routing.go file right? If so, yes! Though I think it's worth also moving getCommonService

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, thank you

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done, though I might have gotten your order wrong (I was confused by determineEndpointService). Let me know if you want me to re-order things.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@dkwon17 did I correctly address this in my last commit fc373a1?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we update the function names?

getServiceForEndpoint and getEndpointService sound a bit too similar IMO, how about

getServiceForEndpoint to getEndpointService
getEndpointSerive to getService

?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

getServiceForEndpoint and getEndpointService sound a bit too similar IMO,

I agree, what do you think about the renames in df6fd02?

  • getServiceForEndpoint -> getEndpointService
  • getEndpointService -> determineEndpointService (I think this makes it a bit more clear that we're getting the appropriate service to use for an endpoint)

Otherwise, I'll just change getEndpointService -> getService as you initially suggested :)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

getServiceForEndpoint -> getEndpointService
getEndpointService -> determineEndpointService (I think this makes it a bit more clear that we're getting the appropriate service to use for an endpoint)

Ok, lets do that

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds good :) if you're +1 for this PR, I'll squash my fixup commit and it'll be good to merge

@AObuchow
Copy link
Contributor Author

/retest-required

@tolusha
Copy link
Contributor

tolusha commented Oct 2, 2024

@AObuchow
For some reason I wasn't able to start a workspace (IDE doesn't appear) using the reproducer on a minikube

@AObuchow
Copy link
Contributor Author

AObuchow commented Oct 3, 2024

@AObuchow For some reason I wasn't able to start a workspace (IDE doesn't appear) using the reproducer on a minikube

@tolusha Thank you for bringing this to my attention. I've been going crazy trying to figure out why this is happening (I was able to reproduce this issue as well on minikube).

So far, it seems like the health endpoint is returning a 502, as I see the following in the ingress-nginx-controller logs:

ingress-nginx-controller-768f948f8f-kkfbv 10.244.0.1 - - [03/Oct/2024:01:33:49 +0000] "GET /user1/python/3100/healthz HTTP/2.0" 502 11 "-" "Go-http-client/2.0" 6 0.003 [eclipse-che-che-gateway-8080] [] 10.244.0.22:8080 11 0.003 502 39b3764a10341ac4068b2b5006253895

Similarly, in the DWO logs:

devworkspace-controller-manager-69f7956695-smkk2 devworkspace-controller {"level":"info","ts":"2024-10-03T01:34:28Z","logger":"controllers.DevWorkspace","msg":"Main URL server not ready","Request.Namespace":"user1-che","Request.Name":"python","devworkspace_id":"workspace7ce3bf9851524dc1","status-code":502}

Oddly enough, this issue seems to not occur if you use the UDI. For example, creating a workspace with the following URL works for me: https://github.com/AObuchow/python-endpoint-devfile-test/tree/main?image=quay.io%2Fdevfile%2Funiversal-developer-image%3Aubi8-latest

So I'm led to believe that the health check doesn't work on certain non-udi images when Che is deployed on minikube? The image used in my reproducer devfile was registry.access.redhat.com/ubi9/python-39:1-192 (from the devfile registry) This issue also occurs using the upstream Che-Operator image (so I don't think it's related to my PR, though I may be mistaken). I'll have to investigate further and report a bug about this, as it's very odd issue.

At some points in my testing, I also had errors like the following in the ingress-nginx-controller logs:

"invalid ingress configuration" err="host \"user1-python-code-redirect-2.192.168.49.2.nip.io\" and path \"/\" is already defined in ingress user1-che/workspacee32ce48b05694318-py-13132-code-redirect-2" ingress="user1-che/workspacebda168270ec8441d-py-13132-code-redirect-2" 

But using the "Create New if Existing" option in the dashboard resolved it?... which is confusing as I compared the devworkspace, devworkspacetemplate, devworkspacerouting and ingresses created and everything seemed the same. However, for the workspace with the above error, only the code-redirect-1 ingress could be created (as the error describes it couldn't use the host & path for the code-redirect-2).

@AObuchow
Copy link
Contributor Author

AObuchow commented Oct 3, 2024

My above theory as to what was going wrong was incorrect. I've now gotten my minikube cluster into a state where devfiles using the UDI (that were working just moments ago) are now giving the incorrect ingress configuration error:

"invalid ingress configuration" err="host \"user1-python-code-redirect-2.192.168.49.2.nip.io\" and path \"/\" is already defined in ingress user1-che/workspacea705c32af9f348f4-tools-13132-code-redirect-2" ingress="user1-che/workspace6037a927113d4279-py-13132-code-redirect-2" 

This is occuring while using the upstream image of Che Operator.

@AObuchow
Copy link
Contributor Author

AObuchow commented Oct 3, 2024

After terminating the ingress-nginx-controller pod on my cluster and allowing the deployment to recreate the pod, the issue seems to be fixed. So my new theory is that somehow the ingress-nginx-controller pod is getting into a weird/broken state.

@AObuchow
Copy link
Contributor Author

AObuchow commented Oct 4, 2024

@tolusha I'm still confused as to what's going on, so I opened eclipse-che/che#23179.
For now, I've updated the reproducer devfile in the PR description to use the UDI as the tooling image, which seems to work correctly on Minikube.

When you have a moment, please test using the above devfile - thank you :)

@openshift-ci openshift-ci bot added lgtm and removed lgtm labels Oct 4, 2024
@AObuchow AObuchow requested a review from dkwon17 October 7, 2024 16:58
Copy link

openshift-ci bot commented Oct 8, 2024

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: AObuchow, dkwon17, tolusha

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@devstudio-release
Copy link

@devstudio-release
Copy link

@devstudio-release
Copy link

@devstudio-release
Copy link

@devstudio-release
Copy link

Build 3.18 :: sync-to-downstream_3.x/7802: SUCCESS

Build container: devspaces-operator-bundle synced; /DS_CI/get-sources-rhpkg-container-build_3.x/7809 triggered; /job/DS_CI/job/dsc_3.x triggered;

@devstudio-release
Copy link

Build 3.18 :: operator-bundle_3.x/3657: SUCCESS

Upstream sync done; /DS_CI/sync-to-downstream_3.x/7802 triggered

@devstudio-release
Copy link

Build 3.18 :: copyIIBsToQuay/2835: Console, Changes, Git Data

@devstudio-release
Copy link

Build 3.18 :: dsc_3.x/2017: Console, Changes, Git Data

@devstudio-release
Copy link

@devstudio-release
Copy link

@devstudio-release
Copy link

@devstudio-release
Copy link

@devstudio-release
Copy link

Build 3.18 :: sync-to-downstream_3.x/7804: SUCCESS

Build container: devspaces-operator-bundle synced; /DS_CI/get-sources-rhpkg-container-build_3.x/7811 triggered; /job/DS_CI/job/dsc_3.x triggered;

@devstudio-release
Copy link

Build 3.18 :: operator-bundle_3.x/3658: SUCCESS

Upstream sync done; /DS_CI/sync-to-downstream_3.x/7804 triggered

@devstudio-release
Copy link

Build 3.18 :: copyIIBsToQuay/2836: Console, Changes, Git Data

@devstudio-release
Copy link

Build 3.18 :: dsc_3.x/2018: Console, Changes, Git Data

@devstudio-release
Copy link

Build 3.18 :: dsc_3.x/2018: SUCCESS

3.18.0-CI

@devstudio-release
Copy link

@devstudio-release
Copy link

@devstudio-release
Copy link

@devstudio-release
Copy link

@devstudio-release
Copy link

Build 3.18 :: sync-to-downstream_3.x/7806: SUCCESS

Build container: devspaces-operator-bundle synced; /DS_CI/get-sources-rhpkg-container-build_3.x/7813 triggered; /job/DS_CI/job/dsc_3.x triggered;

@devstudio-release
Copy link

Build 3.18 :: operator-bundle_3.x/3659: SUCCESS

Upstream sync done; /DS_CI/sync-to-downstream_3.x/7806 triggered

@devstudio-release
Copy link

Build 3.18 :: copyIIBsToQuay/2837: Console, Changes, Git Data

@devstudio-release
Copy link

Build 3.18 :: dsc_3.x/2019: Console, Changes, Git Data

@devstudio-release
Copy link

Build 3.18 :: dsc_3.x/2019: SUCCESS

3.18.0-CI

@devstudio-release
Copy link

Build 3.18 :: copyIIBsToQuay/2837: SUCCESS

3.18
arches = x86_64, s390x, ppc64le;
  * LATEST DS OPERATOR BUNDLE = <a href=https://quay.io/repository/devspaces/devspaces-operator-bundle?tab=tags>registry-proxy.engineering.redhat.com/rh-osbs/devspaces-operator-bundle:3.18-10
  * LATEST DWO OPERATOR BUNDLE = <a href=https://quay.io/repository/devworkspace/devworkspace-operator-bundle?tab=tags>registry-proxy.engineering.redhat.com/rh-osbs/devworkspace-operator-bundle:???
+ s390x-rhel8 IIB(s) copied:
  + <a href=https://quay.io/devspaces/iib:3.18-v4.17-835069- /tmp/getIIBsForBundle.sh -t PROD_VER [OPTIONS]-s390x>quay.io/devspaces/iib:3.18-v4.17-835069- /tmp/getIIBsForBundle.sh -t PROD_VER [OPTIONS]-s390x
  + quay.io/devspaces/iib:3.18-v4.17-s390x
  + <a href=https://quay.io/devspaces/iib:3.18-v4.16-835067- /tmp/getIIBsForBundle.sh -t PROD_VER [OPTIONS]-s390x>quay.io/devspaces/iib:3.18-v4.16-835067- /tmp/getIIBsForBundle.sh -t PROD_VER [OPTIONS]-s390x
  + quay.io/devspaces/iib:3.18-v4.16-s390x
  + <a href=https://quay.io/devspaces/iib:3.18-v4.15-835066- /tmp/getIIBsForBundle.sh -t PROD_VER [OPTIONS]-s390x>quay.io/devspaces/iib:3.18-v4.15-835066- /tmp/getIIBsForBundle.sh -t PROD_VER [OPTIONS]-s390x
  + quay.io/devspaces/iib:3.18-v4.15-s390x
  + <a href=https://quay.io/devspaces/iib:3.18-v4.14-835065- /tmp/getIIBsForBundle.sh -t PROD_VER [OPTIONS]-s390x>quay.io/devspaces/iib:3.18-v4.14-835065- /tmp/getIIBsForBundle.sh -t PROD_VER [OPTIONS]-s390x
  + quay.io/devspaces/iib:3.18-v4.14-s390x
  + <a href=https://quay.io/devspaces/iib:3.18-v4.13-835064- /tmp/getIIBsForBundle.sh -t PROD_VER [OPTIONS]-s390x>quay.io/devspaces/iib:3.18-v4.13-835064- /tmp/getIIBsForBundle.sh -t PROD_VER [OPTIONS]-s390x
  + quay.io/devspaces/iib:3.18-v4.13-s390x
  + <a href=https://quay.io/devspaces/iib:3.18-v4.12-835063- /tmp/getIIBsForBundle.sh -t PROD_VER [OPTIONS]-s390x>quay.io/devspaces/iib:3.18-v4.12-835063- /tmp/getIIBsForBundle.sh -t PROD_VER [OPTIONS]-s390x
  + quay.io/devspaces/iib:3.18-v4.12-s390x
+ ppc64le-rhel8 IIB(s) copied:
  + <a href=https://quay.io/devspaces/iib:3.18-v4.17-835069- /tmp/getIIBsForBundle.sh -t PROD_VER [OPTIONS]-ppc64le>quay.io/devspaces/iib:3.18-v4.17-835069- /tmp/getIIBsForBundle.sh -t PROD_VER [OPTIONS]-ppc64le
  + quay.io/devspaces/iib:3.18-v4.17-ppc64le
  + <a href=https://quay.io/devspaces/iib:3.18-v4.16-835067- /tmp/getIIBsForBundle.sh -t PROD_VER [OPTIONS]-ppc64le>quay.io/devspaces/iib:3.18-v4.16-835067- /tmp/getIIBsForBundle.sh -t PROD_VER [OPTIONS]-ppc64le
  + quay.io/devspaces/iib:3.18-v4.16-ppc64le
  + <a href=https://quay.io/devspaces/iib:3.18-v4.15-835066- /tmp/getIIBsForBundle.sh -t PROD_VER [OPTIONS]-ppc64le>quay.io/devspaces/iib:3.18-v4.15-835066- /tmp/getIIBsForBundle.sh -t PROD_VER [OPTIONS]-ppc64le
  + quay.io/devspaces/iib:3.18-v4.15-ppc64le
  + <a href=https://quay.io/devspaces/iib:3.18-v4.14-835065- /tmp/getIIBsForBundle.sh -t PROD_VER [OPTIONS]-ppc64le>quay.io/devspaces/iib:3.18-v4.14-835065- /tmp/getIIBsForBundle.sh -t PROD_VER [OPTIONS]-ppc64le
  + quay.io/devspaces/iib:3.18-v4.14-ppc64le
  + <a href=https://quay.io/devspaces/iib:3.18-v4.13-835064- /tmp/getIIBsForBundle.sh -t PROD_VER [OPTIONS]-ppc64le>quay.io/devspaces/iib:3.18-v4.13-835064- /tmp/getIIBsForBundle.sh -t PROD_VER [OPTIONS]-ppc64le
  + quay.io/devspaces/iib:3.18-v4.13-ppc64le
  + <a href=https://quay.io/devspaces/iib:3.18-v4.12-835063- /tmp/getIIBsForBundle.sh -t PROD_VER [OPTIONS]-ppc64le>quay.io/devspaces/iib:3.18-v4.12-835063- /tmp/getIIBsForBundle.sh -t PROD_VER [OPTIONS]-ppc64le
  + quay.io/devspaces/iib:3.18-v4.12-ppc64le
+ x86_64-rhel8 IIB(s) copied:
  + <a href=https://quay.io/devspaces/iib:3.18-v4.17-835069- /tmp/getIIBsForBundle.sh -t PROD_VER [OPTIONS]-x86_64>quay.io/devspaces/iib:3.18-v4.17-835069- /tmp/getIIBsForBundle.sh -t PROD_VER [OPTIONS]-x86_64
  + quay.io/devspaces/iib:3.18-v4.17-x86_64
  + <a href=https://quay.io/devspaces/iib:3.18-v4.16-835067- /tmp/getIIBsForBundle.sh -t PROD_VER [OPTIONS]-x86_64>quay.io/devspaces/iib:3.18-v4.16-835067- /tmp/getIIBsForBundle.sh -t PROD_VER [OPTIONS]-x86_64
  + quay.io/devspaces/iib:3.18-v4.16-x86_64
  + <a href=https://quay.io/devspaces/iib:3.18-v4.15-835066- /tmp/getIIBsForBundle.sh -t PROD_VER [OPTIONS]-x86_64>quay.io/devspaces/iib:3.18-v4.15-835066- /tmp/getIIBsForBundle.sh -t PROD_VER [OPTIONS]-x86_64
  + quay.io/devspaces/iib:3.18-v4.15-x86_64
  + <a href=https://quay.io/devspaces/iib:3.18-v4.14-835065- /tmp/getIIBsForBundle.sh -t PROD_VER [OPTIONS]-x86_64>quay.io/devspaces/iib:3.18-v4.14-835065- /tmp/getIIBsForBundle.sh -t PROD_VER [OPTIONS]-x86_64
  + quay.io/devspaces/iib:3.18-v4.14-x86_64
  + <a href=https://quay.io/devspaces/iib:3.18-v4.13-835064- /tmp/getIIBsForBundle.sh -t PROD_VER [OPTIONS]-x86_64>quay.io/devspaces/iib:3.18-v4.13-835064- /tmp/getIIBsForBundle.sh -t PROD_VER [OPTIONS]-x86_64
  + quay.io/devspaces/iib:3.18-v4.13-x86_64
  + <a href=https://quay.io/devspaces/iib:3.18-v4.12-835063- /tmp/getIIBsForBundle.sh -t PROD_VER [OPTIONS]-x86_64>quay.io/devspaces/iib:3.18-v4.12-835063- /tmp/getIIBsForBundle.sh -t PROD_VER [OPTIONS]-x86_64
  + quay.io/devspaces/iib:3.18-v4.12-x86_64

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Support TLS edge termination to expose http endpoints.
5 participants