Skip to content

Characterization

Jump to bottom
Paul Lorenz edited this page Jun 26, 2020 · 6 revisions

Overview

We need to characterize Ziti performance so that we can compare it against plain internet, against other technologies and against itself, so we can tell if we improving, maintaining or degrading performance over time.

Characterization scenarios will be done across three axis.

  • The model
    • This includes the numbers and interactions of services, identities and polices
  • The deployment
    • This includes the number and type of instances and in which regions they are deployed. It also includes if we are using tunnelers or native Ziti applications
  • The traffic
    • This includes the number of concurrent concurrent sessions, the amount of data sent and the number of iterations.

Models

Baseline

  • 1 service
  • 1 identity
  • 1 edge router
  • 1 of each policy

For models with multiple edge routers, do we need to set the runtime up so only one is active, for consistency in test results (and also keeping testing costs down?)

For each policy from A <-> B, ensure we have at least

  1. an A with 1 policy which has 1 B
  2. an A with 1 policy which has all Bs
  3. a B with 1 policy which has all As
  4. an A with all policies
  5. a B with all policies

Small

  • 10 services
  • 100 identities
  • 10 edge routers
  • 5 Service Policies
  • 5 Edge Router Policies
  • 2 Service Edge Router Policies

Medium

  • 100 services
  • 5,000 identities
  • 20 edge routers
  • 25 Service Policies
  • 25 Edge Router Policies
  • 10 Service Edge Router Policies

Large

  • 200 services
  • 100,000 identities
  • 100 edge routers
  • 250 Service Policies
  • 250 Edge Router Policies
  • 100 Service Edge Router Policies

Deployments

We should test with a variety of instance types, from t2 on up. Until we start testing, it will be hard to say what is needed. For high bandwidth applications you often need bigger instance types, even if the CPU and memory aren't required.

The controller should require smaller instances than the router, at least in terms of network use.

We shouldn't need to test deployment variations, such as tunneler vs SDK enabled application for all scenarios. We can pick one or two scenarios in order to find out if there are noticeable differences.

Traffic

There are some different traffic types we should test:

  1. IPerf, for sustained throughput testing. This can be done with various degrees of parallelism.
  2. Something like a web-service or HTTP server, for lots of concurrent, short lived connections, to get a feel for connection setup/teardown overhead.
Clone this wiki locally