Get Rid of YAML Objects

[Mierdin]

At first I thought it was "cool" to do things like define primitives like groups or testruns in YAML, since it's similar to how Kubernetes does things. It offers a bit of that "infra as code" flexibility, as such artifacts can be version controlled.

However, I'm starting to wonder if this is just dumb, added complexity. Especially for group objects, which don't have a lot of arguments, a full-blown YAML file seems overblown compared to simply entering such parameters directly to the ToDD client. As an example (this doesn't exist yet):

todd group add datacenter subnet:192.168.0.0/16

Testrun objects are a bit more complicated, but I wonder if a similar CLI command could be created for testruns, so that users could simply describe their test in one line.

todd execute --source="datacenter:ping:-c 10" --dest="8.8.8.8,4.2.2.2"

todd execute --source="datacenter:iperf:-c {{ target }}" --dest="headquarters:iperf:-s"

This makes testing more scriptable, and I think it's a bit more obvious for folks that don't want to/don't know how to deal with YAML files. My only concern is, especially for complicated tests, does this make the user experience better or worse, compared to the old style with YAML files? The workflow around YAML files is a bit cumbersome, to be fair, but at least it is able to clearly show what's going on during a test. I fear that having this on one line might make this more unclear.

My plan, if I am going to make this change, is to keep YAML objects around, but device some client commands to "bypass" things a bit. Over time I'll deprecate the old model, and the new commands will be the canonical way.

What I would like to discuss here is whether or not this change is worth it, and if so, any additional ideas on the path I've laid out here?

[Mierdin]

Reaching out to @vcabbage @bigmstone @jedelman8 @mrlesmithjr for input to this discussion

[vcabbage]

I can see the advantage of being able to get tests going without creating YAML files. I've wanted to do that a number of times myself.

I would even take it farther and optionally allow defining the source and destination matching in the execution too. I can see sometimes wanting to do a quick one-off test between temporary groups.

Do YAML definitions need to be removed? The logic to handle them is fairly simple since they're simply converted to JSON and sent to the API.

[Mierdin]

Honestly I don't know the answer regarding removal. Surely it's not "remove now" since I'd like to preserve that functionality while building easier functionality. This coincides with https://github.com/Mierdin/todd/issues/44 a bit, since I've always wanted to rename the command, so perhaps we just keep run the way it is, and use execute as a keyword that indicates more "immediate" functionality?

[bigmstone]

I'm of the opinion you should add the cli arguments and keep YAML definition around. For quick test and collecting stuff in the fly having the cli option would be nice but for tests that will be triggered by some sort of infrastructure management tooling on set time intervals it would be nice to have a repository describing the tests that should be run.

[namachieli]

Ill weigh in here with the opinion of: Do both.

Command line allows for very simple create of simple groups.

YAML allows you to expand to complex functionality in a controllable way.

You could add a feature set that allows you to define a series of tests to run, and define that 'runbook' in YAML.

I will be emulating that functionality currently by calling independent testruns from a local script on the server. It would be nice if I didnt need an extra script to loop through.