From e9638fcf5bae0d10cc562cd0b904c117bf301ad4 Mon Sep 17 00:00:00 2001 From: "Shashank S. Harivyasi" <45761248+harivyasi@users.noreply.github.com> Date: Wed, 16 Aug 2023 06:35:38 +0000 Subject: [PATCH] readme and formatting changes --- README.md | 3 ++- developer_notes.md | 35 ++++++++++++--------------- generate_map.py | 60 +++++++++++++++++++++++++++------------------- 3 files changed, 53 insertions(+), 45 deletions(-) diff --git a/README.md b/README.md index 5268de7..9cd5c92 100644 --- a/README.md +++ b/README.md @@ -2,7 +2,7 @@ ## Introduction -A repository to generate static and dynamic maps of installed instances of the [Chemotion](https://chemotion.net/) ELN. The latest version of both these maps can be downloaded [here](https://github.com/harivyasi/ELNmap/releases/tag/latest). +A repository to generate static and dynamic maps of planned and installed instances of the [Chemotion](https://chemotion.net/) ELN. The latest version of both these maps can be downloaded [here](https://github.com/harivyasi/ELNmap/releases/tag/latest). There are minor decrepencies in the exact location of places marked on the map because these are generated algorithmically. @@ -16,6 +16,7 @@ See [Developer Notes](developer_notes.md). - Administrative boundaries: © EuroGeographics. - Files for the geographical data are retrieved from [EuroStat](https://ec.europa.eu/eurostat/web/gisco/geodata/reference-data/administrative-units-statistical-units/nuts). +- Geographical coordinates for locations outside Europe are as defined by [Natural Earth](https://www.naturalearthdata.com). - Font uses the SIL Open Font License (OFL), and is downloaded from servers of GitHub. ### For the dynamic map: diff --git a/developer_notes.md b/developer_notes.md index 43088f4..1bcb1e5 100644 --- a/developer_notes.md +++ b/developer_notes.md @@ -3,18 +3,18 @@ ## What is it? - The repository is useful for creating _static_ and _dynamic_ maps with pins. Each pin has a quanitity and tag associated with it. The tag also determines the color of the pin e.g. [here](https://map.chemotion.scc.kit.edu/) the tags are 'Planned usage', 'Test usage' etc and the quantity is visible by hovering over the pins. -- The repository consists primarly of a single python script called [`generate_map.py`](generate_map.py). +- The repository consists primarly of a single python script called [`generate_map.py`](generate_map.py). When given a single argument `germany`, it produces a static map of Germany with instances marked on it. Otherwise it produces a static map of European countries as well as a dynamic map of the world with instances marked on it. - The workflow is supported by data in the [`data`](data) folder. - The workflow is automated for GitHub actions using [`map_workflow`](.github/workflows/map_workflow.yml) file. ## How it works? -- The static map is generated using [`matplotlib`](https://matplotlib.org/) and its ability to draw maps from topological files. These topological files are downloaded from [Eurostat](https://ec.europa.eu/eurostat/web/gisco/). The generated file is saved as `map.svg`. -- The dynamic version relies on the [Leaflet](https://leafletjs.com) library. We use [template.html](data/template.html) and then use Python to include markers, layers, and layer control panel on the map as well as replace variables written as `$...VARNAME...$`. This file is saved as `map.html`. +- The static map is generated using the [`geopandas`](https://geopandas.org/) library and its ability to draw maps from topological (.geojson) files. These topological files are downloaded from [Eurostat](https://ec.europa.eu/eurostat/web/gisco/) and the [GitHub repository](https://github.com/nvkelso/natural-earth-vector) of [Natural Earth](https://www.naturalearthdata.com). The generated file is saved as `europe.svg` (and `germany.svg`). +- The dynamic version relies on the [Leaflet](https://leafletjs.com) JS library. We use [template.html](data/template.html) and employ Python to include markers, layers, and layer control panel on the map as well as replace variables written as `$...VARNAME...$`. This file is saved as `map.html`. - Prerequistes: - - Python 3.8 + - Python - packages listed in [requirements.txt](requirements.txt) - - font and topography files that can be downloaded using [download_prerequistes.sh](download_prerequistes.sh) + - font and topography files that are be downloaded using [download_prerequistes.sh](download_prerequistes.sh) ### Installing Prerequistes @@ -30,7 +30,7 @@ sh download_prerequistes.sh #### Using `conda` ```bash -conda create -n mapenv python=3.8 pip +conda create -n mapenv python=3 pip conda activate mapenv pip install -r requirements.txt sh download_prerequistes.sh @@ -47,20 +47,16 @@ Open folder in a supporting IDE e.g. VS Code. Everything should be setup in ca. This can be done by adding JSON entries to the [data/plotted_locations.json](data/plotted_locations.json) file. Each entry must have the following keys: ```json -{ - "common_name" : "Aachen", - "nuts_lvl3" : "Städteregion Aachen", - "stage" : "production", - "num_users" : 4, - "country_code": "DE" -}, + { + "common_name": "Zürich", + "id_name": "Zürich", + "stage": "Planned", + "num_users": 2, + "country_code": "CH" + }, ``` -where `common_name` is the name of the city/region (in language of your choice), `nuts_lvl3` is the standard name of the city/region according to the NUTS standard, `stage` is the key that corresponds to `stage` variable in the script, `num_users` is the number of users in that location -- which then appears on the map -- and `country_code` is the [two letter country code as defined by NUTS](https://en.wikipedia.org/wiki/ISO_3166-1_alpha-2). - -### Adding / Removing connections (connection lines) - -Connections are defined as a list of tuples called `connections`. You can add as many tuples to the list as you like. An empty list means no connections will be drawn on the maps. +where `common_name` is the name of the city/region (in language of your choice), `id_name` is the standard name of the city/region (in Europe) according to the NUTS standard or, for places outside Europe, the `name` as used by Natural Earth geojson (most likely the common English name), `stage` is the key that corresponds to `stage` variable in the script, `num_users` is the number of users in that location -- which then appears on the map -- and `country_code` is the [two letter country code as defined by ISO](https://en.wikipedia.org/wiki/ISO_3166-1_alpha-2). ### Changing appearence @@ -68,7 +64,6 @@ Connections are defined as a list of tuples called `connections`. You can add as - We then overlay an icon on the pins. This icon can be changed by changing `overlay_icon`. - The order of stages, as listed in the legend, can be changed by changing the order in the list `ordered_stages`. - The title of the legend, in the dynamic version, can be changed by changing `legend_header`. -- The location of the legend, in the static version, can be changed by changing `legend_location`. Use [one supported]("https://matplotlib.org/stable/api/_as_gen/matplotlib.pyplot.legend.html") by `matplotlib`. - Zooming and padding (set to sensible defaults) can be changed using these variables: - `static_map_patch_size` - `dynamic_map_start_zoom` @@ -77,5 +72,5 @@ Connections are defined as a list of tuples called `connections`. You can add as ### Things to remember -- On the static map, a (European) country appears as soon as it is included. For France, the map does not show areas outside mainland (i.e. island regions) for conciseness. The code for this can be changed under the comment `# modifications to map, if any`. Similar modifications can be made to other regions on the map using similar code. This might be useful when plotting other countries with far-away islands e.g. Spain & Portugal. A helpful guide to NUTS codes is available [here](https://en.wikipedia.org/wiki/First-level_NUTS_of_the_European_Union). +- On the static map, a (European) country appears as soon as it is included. For countries with (faraway) island territorries, the map does not show areas outside mainland for conciseness e.g. France & Spain. The code for this can be changed under the comment `# modifications to eur map, if any`. A helpful guide to NUTS codes is available [here](https://en.wikipedia.org/wiki/First-level_NUTS_of_the_European_Union). - The exact location of a pin is set to be the centroid of the polygon that determines its borders (as defined by NUTS level 3). In rare cases this might [actually be outside the polygon](https://support.esri.com/en/technical-article/000017716). The code will have to be adapted for the same. PR is welcome, though most algorithms that fix can be slow. diff --git a/generate_map.py b/generate_map.py index aa1a687..4405657 100644 --- a/generate_map.py +++ b/generate_map.py @@ -1,6 +1,5 @@ from pathlib import Path from matplotlib.colors import rgb2hex -from matplotlib import colormaps from matplotlib.patches import Circle import matplotlib.pyplot as plt import pandas as pd @@ -8,7 +7,8 @@ import sys # no need to change unless changes in apprearance are required. Please know what you are doing. -only_germany = len(sys.argv) > 1 and sys.argv[1] == "germany" # static map only for Germany, not other european countries +# static map only for Germany, not other european countries +only_germany = len(sys.argv) > 1 and sys.argv[1] == "germany" color = {"Production": "green", # color codes for different types of instances "Mixed": "blue", # color supported in dynamic map are listed here https://github.com/pointhi/leaflet-color-markers "Test": "orange", @@ -22,12 +22,12 @@ overlay_icon = "https://raw.githubusercontent.com/harivyasi/ELNmap/main/data/favicon.ico" legend_header = "Chemotion" if only_germany: - legend_location = (0.2,0.7) + legend_location = (0.2, 0.7) color_based_on = "NUTS_NAME" map_filename = "germany.svg" else: - legend_location = (1.0,0.4) - color_based_on="CNTR_CODE" + legend_location = (1.0, 0.4) + color_based_on = "CNTR_CODE" map_filename = "europe.svg" # access font and JSON data @@ -36,8 +36,8 @@ # NUTS level 1 is 'states' or 'group of states' or something similar # NUTS level 3 is (usually) city-level boundary geojsons = {"eur_country": data_dir / "NUTS_RG_01M_2021_4326_LEVL_1.geojson", - "eur_location": data_dir / "NUTS_RG_01M_2021_4326_LEVL_3.geojson", - "int_location": data_dir / "ne_10m_populated_places_simple.geojson"} + "eur_location": data_dir / "NUTS_RG_01M_2021_4326_LEVL_3.geojson", + "int_location": data_dir / "ne_10m_populated_places_simple.geojson"} # read all locations to be plotted locations = pd.read_json(data_dir / "plotted_locations.json") @@ -51,11 +51,13 @@ eur_country_list = ["DE"] eur_country = eur_country[eur_country.CNTR_CODE.isin(eur_country_list)] else: - eur_country = eur_country[eur_country.CNTR_CODE.isin(locations["country_code"].unique().tolist())] + eur_country = eur_country[eur_country.CNTR_CODE.isin( + locations["country_code"].unique().tolist())] eur_country_list = eur_country.CNTR_CODE.unique().tolist() # modifications to eur map, if any -eur_country_drop_parts = {"FR":["FRY", "FRM"], "ES":["ES7"]} # keep only mainland parts for conciseness +# keep only mainland parts for conciseness +eur_country_drop_parts = {"FR": ["FRY", "FRM"], "ES": ["ES7"]} for v in eur_country_drop_parts.values(): eur_country.drop(eur_country[eur_country.FID.isin(v)].index, inplace=True) @@ -74,15 +76,19 @@ for idx, row in locations.iterrows(): # try and get location from the european city list geometry = eur_location[eur_location.NUTS_NAME == row.id_name].geometry - if geometry.empty: # if not then check international cities list + if geometry.empty: # if not then check international cities list geometry = int_location[int_location.name == row.id_name].geometry - if geometry.empty: # if still not found then raise error - raise IndexError("Could not place the following location on map: "+row.common_name) + if geometry.empty: # if still not found then raise error + raise IndexError( + "Could not place the following location on map: "+row.common_name) else: - locations.loc[locations.id_name == row.id_name, 'latitude'] = geometry.x.values[0] - locations.loc[locations.id_name == row.id_name, 'longitude'] = geometry.y.values[0] + locations.loc[locations.id_name == row.id_name, + 'latitude'] = geometry.x.values[0] + locations.loc[locations.id_name == row.id_name, + 'longitude'] = geometry.y.values[0] -locations = gpd.GeoDataFrame(locations, geometry=gpd.points_from_xy(locations.latitude, locations.longitude), crs=crs) +locations = gpd.GeoDataFrame(locations, geometry=gpd.points_from_xy( + locations.latitude, locations.longitude), crs=crs) ####################### # Plot the static map # @@ -94,27 +100,32 @@ # change projection for static map eur_country = eur_country.to_crs("EPSG:3857") -eur_locations = locations[locations.country_code.isin(eur_country.CNTR_CODE)].to_crs("EPSG:3857") +eur_locations = locations[locations.country_code.isin( + eur_country.CNTR_CODE)].to_crs("EPSG:3857") # plot countries map -fig, ax = plt.subplots(1, figsize=(10,10), tight_layout=True) -ax = eur_country.plot(ax=ax, column=color_based_on, cmap='tab20', edgecolor='w') +fig, ax = plt.subplots(1, figsize=(10, 10), tight_layout=True) +ax = eur_country.plot(ax=ax, column=color_based_on, + cmap='tab20', edgecolor='w') # plot the patches -ax = eur_locations.plot(column="stage", cmap = 'Dark2', ax=ax, markersize=static_map_patch_size, zorder=2, legend=True, legend_kwds={'prop':{'size': 25}, 'frameon': False, 'framealpha': 0.2, 'handletextpad': 0.1, 'markerscale': 2, 'bbox_to_anchor': legend_location}) +ax = eur_locations.plot(column="stage", cmap='Dark2', ax=ax, markersize=static_map_patch_size, zorder=2, legend=True, legend_kwds={ + 'prop': {'size': 25}, 'frameon': False, 'framealpha': 0.2, 'handletextpad': 0.1, 'markerscale': 2, 'bbox_to_anchor': legend_location}) # plot the number of users for x, y, num_users in zip(eur_locations.geometry.x, eur_locations.geometry.y, eur_locations.num_users): - ax.annotate(num_users, xy=(x, y), horizontalalignment='center', verticalalignment='center', color='white', font=font_file, size=static_number_fontsize) + ax.annotate(num_users, xy=(x, y), horizontalalignment='center', + verticalalignment='center', color='white', font=font_file, size=static_number_fontsize) plt.axis('off') plt.savefig(map_filename) if only_germany: - exit() # exit after producing the static map + exit() # exit after producing the static map ######################## # Plot the dynamic map # -######################## +######################## -map_limits = {"lon": {}, "lat": {}} # opposite of what can happen so that the values are always overwritten +# opposite of what can happen so that the values are always overwritten +map_limits = {"lon": {}, "lat": {}} map_limits["lat"]["max"] = locations.latitude.max() + dynamic_map_padding map_limits["lat"]["min"] = locations.latitude.min() - dynamic_map_padding @@ -166,7 +177,8 @@ locations.loc[idx, "inHTML"] += str(idx) for stage in ordered_stages: - layers[stage] += ",".join(locations[locations.stage == stage].inHTML.tolist())+"]).addTo(map)" + layers[stage] += ",".join(locations[locations.stage == + stage].inHTML.tolist())+"]).addTo(map)" for stage in ordered_stages: marker_text += layers[stage]