Read dimensions other than variable and region from external repo

[korsbakken]

This PR fixes #414, as far as I can see. All tests pass, the error cited in #414 goes away, and the resulting DataStructureDefinition object has the expected dimensions.

If possible, it would be great if this PR could be merged as soon as possible, since it's a bug that can cause significant problems. Some possible items that might be considered in the future (none of which I have the capacity to work on in the next few months):

• Add a test. A proper test for this issue would probably be an integration test, which tries to read from a test repo that has all the valid dimensions and a non-empty codelist for each them, and checks that the resulting DataStructureDefinition object has all the expected dimensions and the expected codes in each codelist. As far as I can see, there is no such test currently?
• This PR still hardcodes the dimensions to be in {'region', 'variable', 'model', 'scenario'}. The question is whether other arbitrary dimensions should be allowed. But at the moment, I think that would cause issues multiple other places in the code, so probably goes far beyond the scope of this PR.

[danielhuppmann]

I'm surprised that line 139 doesn't have to be changed as well?

[korsbakken]

I'm surprised that line 139 doesn't have to be changed as well?

Yes, it does for crossing t's and dotting i's, thanks for flagging that. It doesn't really affect the final DataStructureDefinition object, but it causes the DataStructureConfig object to lack those dimensions in its repos attribute. It doesn't look like that issue propagates to anywhere in this case, but it might cause problems in other circumstances?

Also, I added the extra dimensions in the add_dimension field validator. Again, I don't have a lot of pydantic expertise. But I assume not including them there might mean that formatting errors under model and scenario in the yaml file might not be caught.