fix ExternalTrafficPolicy for different service types

[clsv]

Problem:
The ExternalTrafficPolicy for services of type LoadBalancer and NodePort was incorrectly set. For LoadBalancer services, the policy should be Local to ensure external connections reach the node where the service is running. However, it was incorrectly set to Cluster. Similarly, NodePort services had ExternalTrafficPolicy set to Local, when it should be Cluster for proper internal routing.

Cause:
The wrong ExternalTrafficPolicy was applied for NodePort and LoadBalancer services, causing inefficient routing for external traffic.

Solution:
The ExternalTrafficPolicy for NodePort services has been updated to Cluster, and for LoadBalancer services to Local, ensuring proper traffic flow for both service types.

[CLAassistant]

All committers have signed the CLA.

[hors]

Hi @clsv, maybe having these settings flexible is a better idea. Users will be able to change them for their needs. Like, users will be able to change it via CR https://github.com/percona/percona-server-mongodb-operator/blob/main/deploy/cr.yaml#L227 because now it is impossible.

[clsv]

Hi @hors, i encountered this issue yesterday in version 1.17.0, and I had to create a deployment with a kubectl patch for services of type NodePort. While I agree that having flexible settings is a good idea, I still believe it would be preferable to have more expected default behavior. When I request a LoadBalancer, I would expect that only the nodes running MongoDB would respond to external traffic.

[JNKPercona]

Test name	Status
arbiter	passed
balancer	passed
custom-replset-name	passed
custom-tls	passed
custom-users-roles	passed
custom-users-roles-sharded	passed
cross-site-sharded	passed
data-at-rest-encryption	passed
data-sharded	passed
demand-backup	passed
demand-backup-eks-credentials	passed
demand-backup-physical	failure
demand-backup-physical-sharded	failure
demand-backup-sharded	passed
expose-sharded	passed
ignore-labels-annotations	passed
init-deploy	passed
finalizer	passed
ldap	passed
ldap-tls	passed
limits	passed
liveness	passed
mongod-major-upgrade	passed
mongod-major-upgrade-sharded	passed
monitoring-2-0	failure
multi-cluster-service	failure
non-voting	passed
one-pod	passed
operator-self-healing-chaos	passed
pitr	passed
pitr-sharded	passed
pitr-physical	passed
pvc-resize	passed
recover-no-primary	passed
replset-overrides	passed
rs-shard-migration	passed
scaling	passed
scheduled-backup	passed
security-context	failure
self-healing-chaos	failure
service-per-pod	passed
serviceless-external-nodes	passed
smart-update	passed
split-horizon	passed
storage	passed
tls-issue-cert-manager	passed
upgrade	passed
upgrade-consistency	passed
upgrade-consistency-sharded-tls	passed
upgrade-sharded	passed
users	passed
version-service	passed
We run 52 out of 52

commit: 799f0ca
image: perconalab/percona-server-mongodb-operator:PR-1673-799f0ca3