separation of reserved capacity and deployment contracts

[muhamadazmy]

Right now (still work in progress actually) a contract between a user and the node is sealed with a Deployment Contract:
In theory reservation goes like this

• A user starts by selecting the farm
• Create a contract with the farm
• The capacity planner chooses the best node based on the policy of the contract
• the contract is assigned a node
• the user then deploy finally on the node

While this sounds straight forward, some issues arise because of the following:

• Some workload information actually depends on some node information for example, the free ports and network ranges, and ips.
• If a user wants to deploy a complex setup (like K8S) he is forced to first create the vm contracts to make sure they are assigned nodes with enough capacity then create the network contracts to be able to force deploy them on the same nodes as the vm. This make the deployment plan confusing and not clear (also not generalized)
• Another issue if a user wants to delete a vm and then redeploy it (say a different image) on the same node there is no grantee that the capacity will not be picked up by someone else before he manages to re create the deployment hence he loses his spot.

To avoid all those issues and create a more "general" way to plan a complex deployment I suggest the following refactoring:

• Separate "Capacity" reservation from "Deployment" contracts.
• Capacity can be reserved (based on policy) separate from the deployment data (hence no hash)
• Same capacity reservation can be utilized by one or more contracts.
• Capacity can be expanded (if the node supports it) or shrinked (if not used by contracts) or completely released if not used anymore.

The scenario goes as follows:

• User start by requesting Capacity. A capacity object is a stripped down version of the deployment contracts it needs the following fields:
• Resources (CU, SU, MU, and HU)
• Policy {single, exclusive(group.id)}
• Features: list of features, where a feature can be something like (GPU, or AccessNode for example)

This way a user can ask for capacity on multiple nodes. and plan their span in the network. Once all Capacity is reserved the user then can proceed by creating contracts. Contracts now has less attributes. It should include mainly the following:

• Capacity ID. link to the capacity object created above.
• Hash
• Resources (CU, SU, MU and HU)
• NumPublicIps
• State
• etc...

Note: there is no policy since now contracts already know ahead what node ids they have (from their reserved capacity)

• Multiple contracts can use the same Capacity as long as the total contracts resources is less than or equal than reserved capacity.
• It's up to the node to make sure deployment resources does not go over the contract resources.
• Billing now takes into account ONLY the reserved capacity from the capacity object since it's locked for the user.
  • Contracts will still report NU.
  • IPs are still reserved and billed per contract because that's a deployment feature.
• Means also user can create/delete deployment without losing his spot on the node.
• If user need more capacity on a certain node he can try to expand this capacity which can fail or succeed based on how much resources available on the node.

Visually (from the dashboard or the user perspective in general) The user sees his reserved capacity as dedicated "almost rented nodes" where he can use to put his workloads. He pays for the full rented capacity.

User then can choose to rent more, expand, shrink, or delete certain "sub-nodes" when needed.

I will try to add sequence diagrams and object diagrams to make it cleare

[DylanVerstraete]

I see what kind of issues it solves and I do agree it can be a good idea. If we go this way, I think we can remove the concept of rent contracts, since you could create a capacity contract that spans the entire node's resources.

[muhamadazmy]

User should not be able to specify specific node. It's completely up to the capacity planner to select the node for this capacity reservation. The policy in reservation are guides to the planner that must take into account while finding the suitable node.
From the current use case what we want are:

• Any -> capacity planner then can choose any node that provide all the features request by the capacity reservation
• Exclusive(group) -> On creating a capacity reservation, the capacity accepts an optional group. If policy is exclusive with that group the capacity then can not share the node of any of other reservations that has the same group id.

Then there is a list of features that works as filters to the planner this list can be empty or hold one or more filters: here is an not complete list (other can be added later)

• PublicNode -> regarding of the policy this can apply
• GPU -> won't be implemented yet

[brandonpille]

So the concept rent contracts will completely be removed (renting a specific node)?

[muhamadazmy]

Renting imho should be removed and replaced with this. But anyway, renting was taking a specific node was just because that was how things was done (you also had to specify a specific node with normal contracts). But renting (in traditional sense) need to also to rely on capacity planner imho to select which node.

In an abstract sense a rent contract is just a capacity reservation that spans the entire node. right ?

[brandonpille]

Indeed. And what about the discount for renting a complete node? Will that still happen?

[DylanVerstraete]

Indeed. And what about the discount for renting a complete node? Will that still happen?

yes, this is still required

[DylanVerstraete]

How would this work across different nodes? I need to create capacity contract for each node?

[AhmedHanafy725]

so the capacity will be created against a farm ID, and once the chain found a node that satisfies the required resources/features, it will set the node ID on the capacity object, right?

[brandonpille]

We should already be able to know the node id when calling reserve_capacity (or whatever name we give the extrinsic). I assume we will be billing from the moment the user reserves capacity, right?

[muhamadazmy]

@AhmedHanafy725 yes, exactly

@brandonpille no, the reserve capacity will only take the farm id, it's up to the capacity planner with the policy requirement (you give this while calling the create_reservation) that sets the node id.

[brandonpille]

What I meant was: we will already know the node_id after reserve_capacity was called, before deployment.
And what about the billing? I assume that starts whenever you reserve capacity.

[DylanVerstraete]

I guess billing will be on capacity level and not on deployment contract. The only thing to account for is NRU consumed by deployment contracts, this still needs to be billed