Support for Autoscale, using VNF in L2 Networks.

[btzq]

ISSUE TYPE

• Enhancement Request

COMPONENT NAME

L2 Networks, Autoscaling

CLOUDSTACK VERSION

4.19.0

CONFIGURATION

OS / ENVIRONMENT

SUMMARY

Autoscaling was introduce few months ago and so far it works well.

However in more enterprise environments, Autoscale is usually required for web servers, which needs a layer of security applicance in the front to prevent the service from being exposed to vulnerabilities.

VNF Appliances have been introduced in Cloudstack which allows users to bring in their own Virtual Network Appliances (eg. PFsense, FortigateVM) to help with this. But these work effectively only in a L2 Environment, because if it was used in a normal VPC, the default gateway for all VMs would be the Virtual Router, hence bypassing the VNF altogether.

As a cloud provider, looking to service enterprise customers, id like to be able to use both autoscaling and VNF appliances together. Hence, am wondering if it is possible to enhance the virtual router to forward specified traffic (based on specific rules eg. CIDR, Private IPs) to a specified VNF hosted within the VPC. Then, the VNF would be configured to handle the 'filtering/firewalling' and other sort of functions that the VNF is responsible to handle. And once done, it will route the traffic back to the Virtual Router.

This method would not only solve the autoscaling issue, but would solve the issue in general where customers cant filter their traffic to the internet/virtual router if using a VPC. Also would solve the pain of using ACL Lists right now, where if a VPC has a large number of networks, ACL rules are too tedious to manage. This would be made much easier with a VNF.

Just ideas, may sound crazy, open for discussion.

STEPS TO REPRODUCE

EXPECTED RESULTS

To be able to use VNFs together with Autoscaling Function.

ACTUAL RESULTS

VNF is only available for L2 Networks. If VNF is used for VPC, every single VM would need to be configure to use the VNF as the default gateway, which is a tedious approach for cloud. (Feels like going back to on prem practices)

[weizhouapache]

Currently the autoscale feature has dependency on loadbalancing service. For now it only works with isolated networks or vpc with lb.
Do you want VNF to be working as the LB provider in the isolated network with VR( as source nat) ?

[btzq]

Hi @weizhouapache ,

Allowing VNF to be the LB Provider would be great, but come to think of it, i think it will pose a challenge because Cloudstack will need to recognise the different metrics that the VNF outputs for Scale Up, Scale Down etc.

If Autoscale relies on Virtual Router for now, then would this suggestion below be easier to implement instead?

In this diagram:

• No change to the way Autoscaling works is required.
• Virtual Router just needs to be allowed to route incoming traffic from the internet, to the VNF.
• VNF will then return the traffic back to Virtual Router.
• Any VNF can be used as its just routing of traffic.

I believe a UI is needed to specify the conditions in which the Virtual Router will route the traffic to the VNF. I think basic conditions will do, as the more advanced stuff will be done by the VNF.

[weizhouapache]

Hi @weizhouapache ,

Allowing VNF to be the LB Provider would be great, but come to think of it, i think it will pose a challenge because Cloudstack will need to recognise the different metrics that the VNF outputs for Scale Up, Scale Down etc.

If Autoscale relies on Virtual Router for now, then would this suggestion below be easier to implement instead?

In this diagram:

• No change to the way Autoscaling works is required.
• Virtual Router just needs to be allowed to route incoming traffic from the internet, to the VNF.
• VNF will then return the traffic back to Virtual Router.
• Any VNF can be used as its just routing of traffic.

I believe a UI is needed to specify the conditions in which the Virtual Router will route the traffic to the VNF. I think basic conditions will do, as the more advanced stuff will be done by the VNF.

currently Autoscaling supports 5 counters, see https://docs.cloudstack.apache.org/en/latest/adminguide/autoscale_with_virtual_router.html#id2
the cpu/memory usage are fetched from hypervisor
the network/lb metrics are fetched from virtual routers.
for now, cloudstack does not get metrics for Autoscaling from other sources.

it looks like you need a feature like traffic mirroring.
@btzq

[btzq]

@weizhouapache actually in my diagram above, im suggesting to not use VNF to provide Autoscaling Feature. Let the current logic work as is.

The end result we are trying to achieve is to implement some sort of security implementation from the internet, just before it gets distributed to the VMs in the VPC.

• Internet -> VNF -> Virtual Router -> Autoscale Grousp (Ideal Scenario)

But this is not possible because the default gateway must be the Virtual Router.

So in my diagram, i was suggesting another way to achieve that is by having the Virtual Router, route traffic to the VNF.
VNF will then return the traffic after processing. Then, the Router can continue to distribute the traffic to whichever services need it via the existing logic (Eg. Autoscaling)

The implementation would look like this.

• Internet -> Virtual Router -> VNF -> Virtual Router -> Autoscale Groups

[weizhouapache]

@weizhouapache actually in my diagram above, im suggesting to not use VNF to provide Autoscaling Feature. Let the current logic work as is.

The end result we are trying to achieve is to implement some sort of security implementation from the internet, just before it gets distributed to the VMs in the VPC.

• Internet -> VNF -> Virtual Router -> Autoscale Grousp (Ideal Scenario)

But this is not possible because the default gateway must be the Virtual Router.

So in my diagram, i was suggesting another way to achieve that is by having the Virtual Router, route traffic to the VNF. VNF will then return the traffic after processing. Then, the Router can continue to distribute the traffic to whichever services need it via the existing logic (Eg. Autoscaling)

The implementation would look like this.

• Internet -> Virtual Router -> VNF -> Virtual Router -> Autoscale Groups

the second option looks like
https://docs.aws.amazon.com/vpc/latest/mirroring/what-is-traffic-mirroring.html
not exactly the same
If you have full control on the virtual router and VNF, you can do PoC by some manual changes.
I have no idea if it is possible.

option 1 make more sense to me. the public Ip needs to be associated to the VNF.

[btzq]

@weizhouapache traffic mirroring is a great tool but it doesnt allow for taking action against malicious traffic.

Would Option 1 be more straightforward to implement? And in this option, i suppose assigning a public ip to the vnf should not be an issue.

But how would the traffic then get send back to the virtual router?

And how would the virtual router recognise the vnf as the new default gateway?

[btzq]

@weizhouapache , ive updated the diagram to reflect Option 1 better.
Is my understanding right?

[DaanHoogland]

this seems like a discussion that ccan lead to several implementation ideas/issues. I'm converting it to a discussion.

[btzq]

@weizhouapache what are your thoughts on this so far?

[weizhouapache]

@weizhouapache what are your thoughts on this so far?

@btzq
you latest diagram looks good to me
the VNF needs two nics

• public Ip which is reachable from the Internet.
• another IP which is in the same vlan as the public IP of virtual router so that the traffic can be forwarded to virtual router (then to the VMs). You may create a L2 network with vlan equals to the vlan of public Ip range.
  I think it is feasible. no idea how to automate the process

[btzq]

@weizhouapache i just realised an issue from the lastest diagram.

As a public cloud operator, we would usually charge a customer based on their usage of the Internet (eg. Egress/Ingress Charges). That means, needing to be able to monitor and control these metrics.

With the latest diagram, where we force the VNF to be the first point of contact to the internet, this means we will no longer be able to measure the traffic anymore unless its passed to the Virtual Router.

This also further opens up a loop hole (similar to if the user has a VNF+L2 Network) where they can completely bypass the router and just route traffic directly to the selected VMs.

The result is that the Cloud Operator will not be able to control the users usage of the internet anymore.

But, i dont have any ideas how to overcome this, with this architecture.

[alexandremattioli]

@btzq the VR's firewall is good enough for the majority of Web Traffic.

With that said, I really like your idea as that could fit many use cases, I hope someone picks up the development of that.