Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

various minor changes, and updated opendap mapper for Arome Arctic forecast data #533

Open
wants to merge 66 commits into
base: master
Choose a base branch
from

Conversation

mortenwh
Copy link
Contributor

Tests failed locally but, as agreed with @akorosov , we test in actions and see how it goes...

mortenwh and others added 30 commits October 23, 2020 16:09
… issue when fill values are not correctly picked up
…'CF-1.5' which is obviously wrong. This is now deleted. Same goes for the history. That should also be handled in the processing software rather then by the gdal netcdf driver.
Copy link
Member

@aperrin66 aperrin66 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for the update, I added some comments describing the changes necessary to pass the tests. Can you have a look at it?

@@ -123,25 +129,29 @@ def export(self, filename='', bands=None, rm_metadata=None, add_geolocation=True
# Rename variable names to get rid of the band numbers
self.rename_variables(filename)
# Rename attributes to get rid of "GDAL_" added by gdal
self.rename_attributes(filename)
self.correct_attributes(filename, history=self.vrt.dataset.GetMetadata()['history'])
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This makes a lot of unit tests fail. May I suggest the following:

Suggested change
self.correct_attributes(filename, history=self.vrt.dataset.GetMetadata()['history'])
self.correct_attributes(filename, history=self.vrt.dataset.GetMetadata().get('history'))

"""
_, filename = os.path.split(filename)
t = datetime.strptime(filename.split('_')[-1], '%Y%m%dT%HZ.nc')
return datetime.strftime(t, '%Y-%m-%dT%H:%MZ')
return datetime.strptime(filename.split('_')[-1], '%Y%m%dT%HZ.nc')
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All other mapper's get_date() method return a formatted string.
I think it would be best to keep consistency across all mappers, would you mind reverting this?

it as a datetime object.
"""
_, filename = os.path.split(filename)
return datetime.strptime(filename.split('_')[-1], '%Y%m%dT%HZ.nc')
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same comment as in mapper_opendap_arome.py: for consistency across mappers, I think it would be best to return a formatted string.

return metadictlist


def _get_band_from_subfile(self, fn, netcdf_dim={}, bands=[]):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are still calls to _get_band_from_subfile() in mapper_netcdf_cf_sentinel1.py and in the unit tests (test_mapper_netcdf_cf.py).
Would you mind updating these with your new methods?

@@ -526,9 +526,9 @@ def _create_extent_dict(extent_str):
raise ValueError('<extent_dict> must contains exactly 2 parameters '
'("-te" or "-lle") and ("-ts" or "-tr")')
key, extent_dict = Domain._add_to_dict(extent_dict, option)
if key is 'te' or key is 'lle':
if key == 'te' or key == 'lle':
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This change highlights a problem in this test:
the arguments to -te are given in the wrong order.
Could you change "-te -180 180 60 90 -ts 500 500" to "-te -180 60 180 90 -ts 500 500"?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants