Remove use of eval

[chrisinabox]

Removes use of unsafe eval() and replaces with a rudimentary interpreter

[sveinse]

Somewhat messy commit history here, but the diff seems ok, so it'll be ok if we squash it.

Please explain the logic of what you've done here

• The RE_NAME regexp addition
• Please add a separate test function for StringFormat()
• The logic of EvaluateExpression() is a bit mirky and needs description
• The test function for test_evaluate_expression() should also include constants

[chrisinabox]

Somewhat messy commit history here, but the diff seems ok, so it'll be ok if we squash it.

Please explain the logic of what you've done here

• The RE_NAME regexp addition
• Please add a separate test function for StringFormat()
• The logic of EvaluateExpression() is a bit mirky and needs description
• The test function for test_evaluate_expression() should also include constants

Of course. I'll do the necessary updates as soon as I can.

The StringFormat() method is used a few times throughout node.py file to parse node object names. This is primarily used for pre-defined objects which can be found in the map.py. An example an unformatted string is this - Additional Server SDO %d Parameter[(idx)].

The historic behavior (pre-this-change) would be as follows:

1. Match the input string to the regex (if no match then just return the string). For the example this would give two groups Additional Server SDO %d Parameter and (idx)
2. Attempt to format the string using the eval method. In this case "Additional Server SDO %d Parameter" % eval("(idx)"). As the StringFormat(text, idx, sub) method expects three arguments the implementation assumes that string text only includes operations using the variables idx and sub.
3. Return the formatted string if no exception thrown

I changed the regex so that it now matches and groups ignoring the () brackets. For the example the groups now are Additional Server SDO %d Parameter and idx. I then treat that as a comma seperated list and replace any instance of the string "idx" and "sub" with the passed parameters:

args = fmt[1].split(',')
args = [a.replace('idx', str(idx)) for a in args]
args = [a.replace('sub', str(sub)) for a in args]

This should mean that the strings in args only contain numerical constants. These are then passed to the Evaluate Expression method described below.

The EvaluateExpression() is a very basic parser for simple arithmetic equations and currently only matches the equations found in strings in maps.py. It uses the ast library to parse input strings and then evaluates if the resulting tree matches the handled formats and computes the results. For simple arithmetic it is expected the results tree to contain a BinOp type with left and right parts both being Constants.

As the majority of strings passed to this method are just constants (i.e. "4"), EvaluateExpression() first attempts to do an eval() type operation using literal_eval() which is a relatively safe alternative to eval() -

If it's unable to match the expression then currently this method throws a SyntaxError although potentially another error type should be used.

Hopefully this provides some clarity on the implementation.

@sveinse One observation I had in the previous PR is that the new .json formatted ODs are directly using the unparsed strings that are contained in maps.py. You can see a simple example of this by generating a new OD. Potentially this should be changed?

 "index": "0x1600",  // 5632
      "name": "Receive PDO %d Mapping[(idx)]",
      "struct": "narray",
      "group": "built-in",
      "mandatory": false,
      "incr": 1,
      "nbmax": 512,
      "each": {
        "name": "PDO %d Mapping for an application object %d[(idx,sub)]",
        "type": "UNSIGNED32",  // 7
        "access": "rw",
        "pdo": false,
        "nbmin": 0,
        "nbmax": 64
      },

[chrisinabox]

@sveinse I've significantly refactored the evaluate expression functions due to finding some issues when building out the tests.

The approach now attempts to recursively parse the ast.tree that's generated from the input string. This is a little cleaner and helps handle the wide variety of potential inputs without requiring an extensive set of if statements.

I was considering updating all string formatting to new py3 style but as the current .json format includes old style formatting strings I did not want to do this until a decision is made on how to handle them.