CCDR tools into RDL tools

[matamadio]

Objective

Take the existing CCDR tools and make them:

• Easier to use
• More efficient
• Flexible, that can be linked to different datasets provided by RDL

Workplan and Timeline

Mat and Jasper to split tasks and coordinate releases on new branch dev_push.
First we optimise and test a flood-only version, which is richer in terms of options compared to other hazards.
Once solid and tested, we try to split the components that can be used for multi-hazard from hazard-specific configuration.
Then we produce a launcher that can:

1. wrap everything in a visual interface and guide the user in a friendly way (limit manual data preparation, avoid script editing)
2. execute the program in batch (sequential for different parameters)
3. produce maps and charts once results are calculated

Tasks

• Auto-fetching of boundaries #32
• Auto-collection of Population and Built-up data layers #6 (1/3)
• Ensure that parallel processing works when running program from jupyter notebook
• Graphic User Interface (GUI) #34
• Add flood damage functions for all regions
• Re-structure code as common parts and hazard-specific interfaces
  • Modularization and defining specific config files #35
  • Efficiency: don't download datasets that have been previously downloaded
• Update documentation to reflect changes
• Chance to concatenate (batch) several analyses, e.g. several countries one after the other.
• Extended testing, find the cap limitations

Timeline

[matamadio]

Tasks update

GUI

• New GUI for selecting hazard, and then for running the specific hazard screening. Only flood is included for now, placeholder for cyclones and heat stress.
  • Move away from old hazard notebooks (those feature superseeded analysis code) and focus on the new approach (dynamic GUI): different versions of the same GUI are generated dynamically based on each hazard's hzd_config.py file
  • For now, the choice leads to hazard-specific GUI static notebooks - let's finalise floods first, then extend to other hazards using config files approach.
• Preview results as map and EAI charts after analysis is completed

FUNCTIONALITIES

• Country selector: dropdown menu with all country names as autocomplete field; resolve argument as ISO_A3 (based on either a csv with global [name, ISO_A3, WB_REGION]
• Connect GUI.py to run_analysis.py (run analysis button simply runs try.py at the moment). Rename try.py into manual_run.py, used just for testing purpose, might delete later.
  • Ensure parallel processing works as expected, and all stages of progress from run_analytics are printed in the interface
• Auto-fetching and processing of ADM, POP. BU.
  • AGR will need pre-clipped input data on S3 - original source data too big to fetch and process.
• Link to notebook to pre-process FATHOM v3 country dataset (merge tiles)
• Preview impact functions for each exp based on country georegion.
• Save output as multi-tab excel with auto-summary, and multi-layer geopackage.
• Option to edit the manually edit the damage functions

STRUCTURE AND REPO

• input_utils.py includes libs to fetch and pre-process input data. Only called if input_data not already present in the folder.
• GH repo: set protection rules for push to main (dev_push has none)
• dev_push branch: file renaming and restructuring folders.