Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cannot create SSLO HTTP service proxy_type as Transparent with existing self IP #82

Open
imohmmad opened this issue Feb 29, 2024 · 11 comments
Labels
Backlog Added to JIRA board bug Something isn't working

Comments

@imohmmad
Copy link

imohmmad commented Feb 29, 2024

COMPONENT NAME
bigip_sslo_service_http

Environment
ANSIBLE VERSION
ansible [core 2.16.2]
config file = /python-env/POD-ONBOARDING/ansible.cfg
configured module search path = ['/root/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
ansible python module location = /python-env/ansible-9.1.0/lib/python3.11/site-packages/ansible
ansible collection location = /root/.ansible/collections:/usr/share/ansible/collections
executable location = /python-env/ansible-9.1.0/bin/ansible
python version = 3.11.4 (main, Dec 7 2023, 15:43:41) [GCC 12.3.0] (/python-env/ansible-9.1.0/bin/python3)
jinja version = 3.1.3
libyaml = True

BIGIP VERSION
Sys::Version
Main Package
Product BIG-IP
Version 17.1.1.1
Build 0.68.2

CONFIGURATION

OS / ENVIRONMENT
Ubuntu 20.04.3

SUMMARY
unable to create SSLO http service with proxy_type transparent while utilizing existing self_ip.

STEPS TO REPRODUCE
Create the following SSLO http service
- name: Create Explicit HTTP service
bigip_sslo_service_http:
name: "HTTP-TRANSPARENT"
proxy_type: "transparent"
service_down_action: "ignore"
auto_manage: false
use_exist_selfip: true
snat: "none"
ip_family: "ipv4"
devices_to:
vlan: "/Common/TO_FE"
self_ip: 10.10.1.1
netmask: 255.255.255.0
devices_from:
vlan: "/Common/FROM_FE"
self_ip: 10.10.2.2
netmask: 255.255.255.0
devices:
- ip: "10.10.1.2"

EXPECTED RESULTS

should be able to create http service with proxy type as transparent

ACTUAL RESULTS

"msg": "CREATE operation error: a04ed767-cdec-4516-859e-8b1a8f001985 : [OrchestratorConfigProcessor] Deployment failed for Error: [BaseHAConfigProcessor (ServiceBaseHAConfigProcessor)] TypeError: Cannot read property 'lastIndexOf' of undefined"
}

@imohmmad imohmmad added the bug Something isn't working label Feb 29, 2024
@pgouband
Copy link

Hi,

Thanks for reporting. Added to the backlog and internal tracking ID for this request is: INFRAANO-1535.

@pgouband pgouband added the Backlog Added to JIRA board label Apr 12, 2024
@megamattzilla
Copy link

Hello!

Is there an ETA for this fix?

Thanks!

@megamattzilla
Copy link

megamattzilla commented Aug 5, 2024

I retested today on latest ansible version and bigip modules and I can still recreate the error:

Error:

TASK [Create Transparent HTTP service] **************************************************************************************************
fatal: [192.168.1.102]: FAILED! => {"changed": false, "msg": "CREATE operation error: bc5ffa65-faa6-4085-b2d4-e4ade90c3367 : [OrchestratorConfigProcessor] Deployment failed for Error: [BaseHAConfigProcessor (ServiceBaseHAConfigProcessor)] TypeError: Cannot read property 'lastIndexOf' of undefined"}

Playbook

- name: Create HTTP service
  hosts: "sslo17"
  connection: httpapi
  gather_facts: false
  any_errors_fatal: true


  tasks:

      - name: Test Module
        f5networks.f5_bigip.bigip_sslo_service_http:
          name: "proxy1a"
          state: "absent"

      - name: Create Transparent HTTP service
        f5networks.f5_bigip.bigip_sslo_service_http:
          name: "HTTP-TRANSPARENT"
          proxy_type: "transparent"
          service_down_action: "ignore"
          auto_manage: false
          use_exist_selfip: true
          snat: "none"
          ip_family: "ipv4"
          devices_to:
            vlan: "/Common/Internal1"
            self_ip: 10.10.1.1
            netmask: 255.255.255.0
          devices_from:
            vlan: "/Common/Internal2"
            self_ip: 10.10.2.2
            netmask: 255.255.255.0
          devices:
          - ip: "10.10.1.2"

Ansible version:

ansible-playbook --version                                     
ansible-playbook [core 2.17.2]
  config file = /Documents/GitHub/local-ansible/ansible.cfg
  configured module search path = ['/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /.pyenv/versions/3.10.14/envs/myenv/lib/python3.10/site-packages/ansible
  ansible collection location = /.ansible/collections:/usr/share/ansible/collections
  executable location = /.pyenv/versions/myenv/bin/ansible-playbook
  python version = 3.10.14 (main, Aug  5 2024, 09:03:11) [Clang 15.0.0 (clang-1500.3.9.4)] (/.pyenv/versions/3.10.14/envs/myenv/bin/python)
  jinja version = 3.1.4
  libyaml = True
ansible-galaxy collection list  | grep -i bigip           
f5networks.f5_bigip                      3.6.2

@prateekramani
Copy link

Hi @megamattzilla,

I was able to reproduce the issue, and it appears to originate from the BIG-IP side. I have been in communication with the BIG-IP team, and they have indicated that this fix will be included in the upcoming BIG-IP v17.5 release, which may take some time.
If it is a critical issue, please raise this issue with the BIG-IP team and if you find any problem please let us know.

Thanks.

@megamattzilla
Copy link

Could you provide the F5 BZ ID please? I'll add that to the list we are tracking.

Its worth mentioning we can configure the same type of HTTP transparent proxy service (with existing self-IPs) in the Big-IP GUI fine.

The Big-IP GUI is able to configure the appropriate service, but the F5 Rest API cannot?

@prateekramani
Copy link

prateekramani commented Aug 26, 2024

Here's the BZ ID - 1633173.
Thanks

@megamattzilla
Copy link

Hi @prateekramani,

Thanks for the info!

Are we sure the root cause of this issue is BZ ID 1633173?

Is it possible both issues have similar, vague error messages?

The reason I ask is there is not very many details in BZ 1633173, the few details it mentions is "POST Upgrade" which there is no upgrade involved here.

I'm also able to create the expected HTTP transparent service fine using the GUI with all the same values I'm trying with the ansible module:

2024-08-26_09-58-30

Interestingly enough, if I re-run the ansible module now (with same values I used to successfully created the service in the GUI) the ansible module errors out with a different message.

If the ansible module is doing everything correctly, I would expect it to say "no changes":

This is the ansible output when I run it after creating the service successfully via the GUI:

fatal: [192.168.1.102]: FAILED! => {
    "changed": false,
    "invocation": {
        "module_args": {
            "auth_offload": null,
            "auto_manage": false,
            "devices": [
                {
                    "ip": "10.10.1.2",
                    "port": null
                }
            ],
            "devices_from": {
                "interface": null,
                "netmask": "255.255.255.0",
                "self_ip": "10.10.2.2",
                "tag": null,
                "vlan": "/Common/Internal2"
            },
            "devices_to": {
                "interface": null,
                "netmask": "255.255.255.0",
                "self_ip": "10.10.1.1",
                "tag": null,
                "vlan": "/Common/Internal1"
            },
            "dump_json": false,
            "ip_family": "ipv4",
            "monitor": null,
            "name": "HTTP-TRANSPARENT",
            "port_remap": null,
            "proxy_type": "transparent",
            "rules": null,
            "service_down_action": "ignore",
            "snat": "none",
            "snat_list": null,
            "snat_pool": null,
            "state": "present",
            "timeout": 300,
            "use_exist_selfip": true,
            "vendor_info": null
        }
    },
    "msg": "Self-IPs are immutable. You must delete and recreate the service to change the self-IPs."
}

Why is the ansible module trying to change the self-IPs? Seems like there is a problem with the flag "use_exist_selfip": true.

@megamattzilla
Copy link

Repeating the same test- create the HTTP-TRANSPARENT service successfully via the GUI and then running the ansible module with same settings- the ansible module fails even when I add dump_json: true.

fatal: [192.168.1.102]: FAILED! => {
    "changed": false,
    "invocation": {
        "module_args": {
            "auth_offload": null,
            "auto_manage": false,
            "devices": [
                {
                    "ip": "10.10.1.2",
                    "port": null
                }
            ],
            "devices_from": {
                "interface": null,
                "netmask": "255.255.255.0",
                "self_ip": "10.10.2.2",
                "tag": null,
                "vlan": "/Common/Internal2"
            },
            "devices_to": {
                "interface": null,
                "netmask": "255.255.255.0",
                "self_ip": "10.10.1.1",
                "tag": null,
                "vlan": "/Common/Internal1"
            },
            "dump_json": true,
            "ip_family": "ipv4",
            "monitor": null,
            "name": "HTTP-TRANSPARENT",
            "port_remap": null,
            "proxy_type": "transparent",
            "rules": null,
            "service_down_action": "ignore",
            "snat": "none",
            "snat_list": null,
            "snat_pool": null,
            "state": "present",
            "timeout": 300,
            "use_exist_selfip": true,
            "vendor_info": null
        }
    },
    "msg": "Self-IPs are immutable. You must delete and recreate the service to change the self-IPs."
}

This should return "no changes".

@megamattzilla
Copy link

Hi @prateekramani ,

This is definitely looking like an ansible modules issue.

When the ansible module generates the POST request to /mgmt/shared/iapp/blocks the JSON item for "id": "f5-ssl-orchestrator-service" has something wrong with it.

I replaced the JSON item for "id": "f5-ssl-orchestrator-service" from a working configuration (creating the same service via the GUI) and now the JSON document ansible is sending properly creates the service (in postman).

The ansible playbook configuration here generates the following POST request:

POST /mgmt/shared/iapp/blocks/ HTTP/1.1
Accept-Encoding: identity
Content-Length: 4534
Host: 10.6.1.10:8443
User-Agent: Python-urllib/3.10
Content-Type: application/json
X-F5-Auth-Token: REDACTED
Connection: close

{"name": "sslo_obj_SERVICE_CREATE_ssloS_HTTP-TRANSPARENT", "inputProperties": [{"id": "f5-ssl-orchestrator-operation-context", "type": "JSON", "value": {"version": 11.1, "partition": "Common", "strictness": false, "operationType": "CREATE", "deploymentName": "ssloS_HTTP-TRANSPARENT", "deploymentType": "SERVICE"}}, {"id": "f5-ssl-orchestrator-network", "type": "JSON", "value": []}, {"id": "f5-ssl-orchestrator-service", "type": "JSON", "value": {"customService": {"name": "ssloS_HTTP-TRANSPARENT", "serviceType": "http-proxy", "serviceSpecific": {"name": "ssloS_HTTP-TRANSPARENT", "proxyType": "Transparent", "authOffload": false}, "connectionInformation": {"fromBigipNetwork": {"name": "toNetwork", "vlan": {"path": "/Common/Internal1-Stovall-62", "create": false, "modify": false, "selectedValue": "/Common/Internal1-Stovall-62", "networkVlanValue": ""}, "routeDomain": {"id": 0, "create": false}, "selfIpConfig": {"create": false, "modify": false, "autoValue": "198.19.96.7/25", "selectedValue": "", "selfIp": "10.10.1.1", "netmask": "255.255.255.0", "floating": false, "HAstaticIpMap": []}, "networkBlockId": ""}, "toBigipNetwork": {"name": "fromNetwork", "vlan": {"path": "/Common/Internal2-Stovall-63", "create": false, "modify": false, "selectedValue": "/Common/Internal2-Stovall-63", "networkVlanValue": ""}, "routeDomain": {"id": 0, "create": false}, "selfIpConfig": {"create": false, "modify": false, "autoValue": "198.19.96.245/25", "selectedValue": "", "selfIp": "10.10.2.2", "netmask": "255.255.255.0", "floating": false, "HAstaticIpMap": []}, "networkBlockId": ""}}, "snatConfiguration": {"clientSnat": "None", "snat": {"referredObj": "", "ipv4SnatAddresses": [], "ipv6SnatAddresses": []}}, "loadBalancing": {"devices": [{"ip": "10.10.1.2", "port": 80}], "monitor": {"fromSystem": "/Common/gateway_icmp"}}, "initialIpFamily": "ipv4", "ipFamily": "ipv4", "isAutoManage": false, "portRemap": false, "httpPortRemapValue": 80, "serviceDownAction": "ignore", "iRuleList": [], "managedNetwork": {"serviceType": "http-proxy", "ipFamily": "ipv4", "isAutoManage": false, "ipv4": {"serviceType": "http-proxy", "ipFamily": "ipv4", "serviceSubnet": "10.10.1.0", "serviceIndex": 0, "subnetMask": "255.255.255.0", "toServiceNetwork": "10.10.1.0", "toServiceMask": "255.255.255.0", "toServiceSelfIp": "10.10.1.1", "fromServiceNetwork": "10.10.2.0", "fromServiceMask": "255.255.255.0", "fromServiceSelfIp": "10.10.2.2"}, "operation": "RESERVEANDCOMMIT"}}, "fromVlanNetworkObj": {"create": false, "modify": false, "networkError": false}, "toVlanNetworkObj": {"create": false, "modify": false, "networkError": false}, "fromNetworkObj": {"name": "ssloN_HTTP-TRANSPARENT_in", "partition": "Common", "strictness": false, "vlan": {"create": false, "modify": false, "name": "ssloN_HTTP-TRANSPARENT_in", "path": "/Common/Internal1-Stovall-62", "networkError": false, "interface": [], "tag": 0, "networkInterface": "", "networkTag": 0}, "selfIpConfig": {"create": false, "modify": false, "selfIp": "10.10.1.1", "netmask": "255.255.255.0", "floating": false, "HAstaticIpMap": []}, "routeDomain": {"id": 0, "create": false}}, "toNetworkObj": {"name": "ssloN_HTTP-TRANSPARENT_out", "partition": "Common", "strictness": true, "vlan": {"create": false, "modify": false, "name": "ssloN_HTTP-TRANSPARENT_out", "path": "/Common/Internal2-Stovall-63", "networkError": false, "interface": [], "tag": 0, "networkInterface": "", "networkTag": 0}, "selfIpConfig": {"create": false, "modify": false, "selfIp": "10.10.2.2", "netmask": "255.255.255.0", "floating": false, "HAstaticIpMap": []}, "routeDomain": {"id": 0, "create": false}}, "vendorInfo": {"name": "Generic HTTP Service"}, "name": "ssloS_HTTP-TRANSPARENT", "partition": "Common", "description": "Type: HTTP", "strictness": false, "useTemplate": false, "serviceTemplate": "", "templateName": "HTTP Service", "previousVersion": 11.1, "version": 11.1}}, {"id": "f5-ssl-orchestrator-service-chain", "type": "JSON", "value": []}, {"id": "f5-ssl-orchestrator-policy", "type": "JSON", "value": []}], "configurationProcessorReference": {"link": "https://localhost/mgmt/shared/iapp/processors/f5-iappslx-ssl-orchestrator-gc"}, "configProcessorTimeoutSeconds": 120, "statsProcessorTimeoutSeconds": 60, "configProcessorAffinity": {"processorPolicy": "LOCAL", "affinityProcessorReference": {"link": "https://localhost/mgmt/shared/iapp/affinity/local"}}, "state": "BINDING", "presentationHtmlReference": {"link": "https://localhost/iapps/f5-iappslx-ssl-orchestrator/sgc/sgcIndex.html"}, "operation": "CREATE"}

I tested sending that same POST request in postman and found that I could replicate the issue- the deploy failed every time with Deployment failed for Error: [BaseHAConfigProcessor (ServiceBaseHAConfigProcessor)] TypeError: Cannot read property 'lastIndexOf' of undefined"

After a few hours of testing, I found the root issue is with the JSON item for "id": "f5-ssl-orchestrator-service".

If I replace the ansible authored JSON item for "id": "f5-ssl-orchestrator-service" with a JSON item from a working environment (obtained by running a tcpdump on loopback interface while creating same service via GUI) it successfully creates the service in postman.

This is the fixed JSON item for "id": "f5-ssl-orchestrator-service":

          {
            "id": "f5-ssl-orchestrator-service",
            "type": "JSON",
            "value": {
              "customService": {
                "connectionInformation": {
                  "fromBigipNetwork": {
                    "name": "toNetwork",
                    "routeDomain": {
                      "create": false,
                      "id": 0,
                      "path": ""
                    },
                    "selfIpConfig": {
                      "autoValue": "198.19.96.7/25",
                      "create": false,
                      "netmask": "255.255.255.0",
                      "selectedValue": "10.10.1.1/24",
                      "selfIp": "10.10.1.1/24"
                    },
                    "vlan": {
                      "create": false,
                      "networkVlanValue": "/Common/Internal1-Stovall-62",
                      "path": "/Common/Internal1-Stovall-62",
                      "selectedValue": ""
                    }
                  },
                  "toBigipNetwork": {
                    "name": "fromNetwork",
                    "routeDomain": {
                      "create": false,
                      "id": 0,
                      "path": ""
                    },
                    "selfIpConfig": {
                      "autoValue": "198.19.96.245/25",
                      "create": false,
                      "netmask": "255.255.255.0",
                      "selectedValue": "10.10.2.2/24",
                      "selfIp": "10.10.2.2/24"
                    },
                    "vlan": {
                      "create": false,
                      "networkVlanValue": "/Common/Internal2-Stovall-63",
                      "path": "/Common/Internal2-Stovall-63",
                      "selectedValue": ""
                    }
                  }
                },
                "controlChannels": [],
                "httpPortRemapValue": "80",
                "iRuleList": [],
                "ipFamily": "ipv4",
                "isAutoManage": false,
                "loadBalancing": {
                  "devices": [
                    {
                      "deviceDescription": "",
                      "ip": "10.10.1.2",
                      "port": 80
                    }
                  ],
                  "monitor": {
                    "fromSystem": "/Common/gateway_icmp"
                  }
                },
                "managedNetwork": {
                  "ipFamily": "ipv4",
                  "ipv4": {
                    "fromServiceMask": "255.255.255.128",
                    "fromServiceNetwork": "198.19.96.128",
                    "fromServiceSelfIp": "198.19.96.245",
                    "ipFamily": "ipv4",
                    "serviceIndex": 0,
                    "serviceSubnet": "198.19.96.0",
                    "serviceType": "http-proxy",
                    "subnetMask": "255.255.255.0",
                    "toServiceMask": "255.255.255.128",
                    "toServiceNetwork": "198.19.96.0",
                    "toServiceSelfIp": "198.19.96.7"
                  },
                  "isAutoManage": false,
                  "serviceType": "http-proxy"
                },
                "name": "ssloS_HTTP-TRANSPARENT",
                "portRemap": false,
                "serviceDownAction": "ignore",
                "serviceEntrySSLProfile": "",
                "serviceReturnSSLProfile": "",
                "serviceSpecific": {
                  "name": "ssloS_HTTP-TRANSPARENT",
                  "proxyType": "Transparent"
                },
                "serviceType": "http-proxy",
                "snatConfiguration": {
                  "clientSnat": "None",
                  "snat": {
                    "ipv4SnatAddresses": [
                      {
                        "ip": ""
                      }
                    ],
                    "ipv6SnatAddresses": [
                      {
                        "ip": ""
                      }
                    ],
                    "referredObj": ""
                  }
                }
              },
              "description": "Type: http-proxy",
              "fromVlanNetworkObj": {
                "create": false,
                "modify": false,
                "networkError": false
              },
              "initialIpFamily": "ipv4",
              "name": "ssloS_HTTP-TRANSPARENT",
              "obRestrictedAttribute": {
                "obId": "79740939-fe71-40a3-bd7f-d5209d13d67f"
              },
              "partition": "Common",
              "previousVersion": "11.0",
              "serviceTemplate": "",
              "strictness": true,
              "toVlanNetworkObj": {
                "create": false,
                "modify": false,
                "networkError": false
              },
              "useTemplate": false,
              "vendorInfo": {
                "name": "Generic HTTP Service"
              },
              "version": "11.0"
            }
          }

There is quite a few differences in the JSON keys between the working and non-working item, but one of them seems to be causing the 'lastIndexOf' of undefined error.

@prateekramani
Copy link

Hi @megamattzilla
I will try to reproduce the issue and let you know.

@pgouband pgouband reopened this Aug 28, 2024
@megamattzilla
Copy link

Files:
bad.json
good.json

POST payload ansible is sending fails with error 'lastIndexOf' of undefined:
curl -k -X POST -H "Content-Type: application/json" https://{{Big-IP MGMT IP}}/mgmt/shared/iapp/blocks -u admin -d @bad.json -v

POST payload of ansible + fixed JSON is 100% successful on 17.1.1.3:
curl -k -X POST -H "Content-Type: application/json" https://{{Big-IP MGMT IP}}/mgmt/shared/iapp/blocks -u admin -d @good.json -v
2024-09-20_10-04-50

The Big-IP BZ 1633173 might affect how ansible was previously constructing the JSON payload, but there appears to be alternative ways to structure the JSON that works on 17.1.1.x.

Could the ansible module construct the JSON payload in the way that works for 17.1.1.x?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Backlog Added to JIRA board bug Something isn't working
Projects
None yet
Development

No branches or pull requests

4 participants