MAINT: Switch from gdal to rasterio #167
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This commit addresses the problem of distributing smash with gdal. gdal doesn't provide binary wheels directly, so you need to install the gdal library on your system before you can install gdal bindings in Python. One way of working with gdal simply in Python is conda, but in order to facilitate the distribution of smash on different OS (linux, maxos and windows), it is simpler to use rasterio to read raster (rasterio provides a binary wheel directly which impacts gdal). This change is not neutral and leads to performance losses in terms of raster calculation time (rain, etp, descriptors, etc). Generally speaking, reading the input data is a relatively quick process compared with a calibration. However, if the calculation time for reading becomes too great a concern, it can be parallelized. 2 other minor changes have been made: - Date regular expression match only on the file name and not the entire path. This can cause problems, for example, if the path is something like this (``/home/2020930920/data/prcp/rain_202001010000.tif``) - Update tests to avoid reading rasters ``N`` times and instead copy arrays NOTE: A future commit will be made to integrate meson to manage the smash build system. A number of changes will be made to the way smash is installed.
nghi-truyen
approved these changes
Apr 20, 2024
Co-authored-by: Ngo Nghi Truyen Huynh <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This commit addresses the problem of distributing smash with gdal. gdal doesn't provide binary wheels directly, so you need to install the gdal library on your system before you can install gdal bindings in Python. One way of working with gdal simply in Python is conda, but in order to facilitate the distribution of smash on different OS (linux, maxos and windows), it is simpler to use rasterio to read raster (rasterio provides a binary wheel directly which include gdal). This change is not neutral and leads to performance losses in terms of raster calculation time (rain, etp, descriptors, etc). Generally speaking, reading the input data is a relatively quick process compared with a calibration. However, if the calculation time for reading becomes too great a concern, it can be parallelized.
2 other minor changes have been made:
/home/2020930920/data/prcp/rain_202001010000.tif)Ntimes and instead copy arraysNOTE: A future commit will be made to integrate meson to manage the smash build system. A number of changes will be made to the way smash is installed.