Skip to content

Commit

Permalink
update
Browse files Browse the repository at this point in the history
  • Loading branch information
spatialthoughts committed Sep 4, 2024
1 parent 29e161e commit b30cc55
Show file tree
Hide file tree
Showing 46 changed files with 358 additions and 4 deletions.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
208 changes: 206 additions & 2 deletions docs/qgis-cloud-native-geospatial.html
Original file line number Diff line number Diff line change
Expand Up @@ -319,6 +319,12 @@ <h4 class="author">Ujaval Gandhi</h4>
id="toc-hosting-cloud-native-geospatial-data">4.2 Hosting Cloud Native
Geospatial Data</a></li>
</ul></li>
<li><a href="#supplement" id="toc-supplement">Supplement</a>
<ul>
<li><a href="#analyzing-cloud-hosted-satellite-imagery"
id="toc-analyzing-cloud-hosted-satellite-imagery">Analyzing Cloud-Hosted
Satellite Imagery</a></li>
</ul></li>
<li><a href="#data-credits" id="toc-data-credits">Data Credits</a></li>
<li><a href="#license" id="toc-license">License</a></li>
</ul>
Expand All @@ -331,7 +337,7 @@ <h4 class="author">Ujaval Gandhi</h4>
<div style="page-break-after: always;"></div>
<div id="introduction" class="section level1">
<h1>Introduction</h1>
<p>This workshop will introduce participants to the modern approach to
<p>This workshop introduces participants to the modern approach to
working with large datasets in QGIS. Modern data formats - such as
Cloud-Optimized GeoTIFFs (COG), Cloud-Optimized Point Clouds (COPC), and
FlatGeoBuf (FGB) allow datasets to be streamed from cloud storage
Expand Down Expand Up @@ -774,7 +780,7 @@ <h2>3.2 Analyze Landcover Change</h2>
<p><img src="images/qgis_cloud_native_geospatial/landcover14.png" width="75%" style="display: block; margin: auto;" /></p>
<ol start="15" style="list-style-type: decimal">
<li>The <code>waterlost_export</code> is a local raster with pixels
showing regions that experienced loss in surface water. Let’s conver
showing regions that experienced loss in surface water. Let’s convert
this layer to polygons. From the Processing Toolbox, search and locate
the <strong>GDAL → Raster Conversion → Polygonize (raster to
vector)</strong> algorithm.</li>
Expand Down Expand Up @@ -812,13 +818,211 @@ <h2>4.1 Creating Cloud Native Geospatial Data</h2>
<h2>4.2 Hosting Cloud Native Geospatial Data</h2>
</div>
</div>
<div id="supplement" class="section level1">
<h1>Supplement</h1>
<div id="analyzing-cloud-hosted-satellite-imagery"
class="section level2">
<h2>Analyzing Cloud-Hosted Satellite Imagery</h2>
<p>In this section, we will learn how to query a STAC catalog of
Sentinel-2 L2A data, load and create a RGB composite and compute a
spectral index using a cloud-native approach.</p>
<ol style="list-style-type: decimal">
<li>Start by loading a basemap. From the <em>Browser</em> panel, scroll
down and locate <strong>XYZ Tiles → OpenStreetMap</strong> tile layer.
Drag and drop it to the main canvas. Zoom to your region of interest. We
will use the <em>STAC API Browser</em> plugin to query for satellite
images over this area. Open the plugin from <strong>Plugins → STAC API
Browser Plugin → Open STAC API Browser</strong>.</li>
</ol>
<p><img src="images/qgis_cloud_native_geospatial/satellite1.png" width="75%" style="display: block; margin: auto;" /></p>
<ol start="2" style="list-style-type: decimal">
<li>Select <code>Earth Search</code> from the <em>Connections</em>
dropdown. Click <em>Edit</em> to view the connection details.</li>
</ol>
<p><img src="images/qgis_cloud_native_geospatial/satellite2.png" width="75%" style="display: block; margin: auto;" /></p>
<ol start="3" style="list-style-type: decimal">
<li>Make sure the URL is
<code>https://earth-search.aws.element84.com/v1</code> and click
<em>OK</em>.</li>
</ol>
<p><img src="images/qgis_cloud_native_geospatial/satellite3.png" width="75%" style="display: block; margin: auto;" /></p>
<ol start="4" style="list-style-type: decimal">
<li>In the <em>STAC API Browser</em> window, click the <em>Fetch
collections</em> button. Once the list updates, select the
<code>Sentinel-2 Level-2A</code> collection. If you do not see this
dataset listed, please go to the previous step and verify the URL is
correct.</li>
</ol>
<p><img src="images/qgis_cloud_native_geospatial/satellite4.png" width="75%" style="display: block; margin: auto;" /></p>
<ol start="5" style="list-style-type: decimal">
<li>Check the <em>Filter by date</em> option and enter a start and end
date. Sentinel-2 Level-2A dataset is available from 2017 onward and a
new image is collected every 5 days. Check the <em>Extent</em> option
and click <em>Map Canvas Extent</em> button to fill the coordinates of
the region visible on the map. Click <em>Search</em>.</li>
</ol>
<p><img src="images/qgis_cloud_native_geospatial/satellite5.png" width="75%" style="display: block; margin: auto;" /></p>
<ol start="6" style="list-style-type: decimal">
<li>In the <em>Results</em> tab, you will see the list of matching
images. From the preview of the image, locate a reasonably cloud-free
image and click <em>View assets</em>.</li>
</ol>
<p><img src="images/qgis_cloud_native_geospatial/satellite6.png" width="75%" style="display: block; margin: auto;" /></p>
<ol start="7" style="list-style-type: decimal">
<li>For this exercise, we will load data for 4 image bands. Check the
<em>Select to add as a layer</em> box for <code>Blue (band 2)</code>,
<code>Green (band 3)</code>, <code>Red (band 4)</code> and
<code>SWIR 1 (band 11)</code>. Click <em>Add select assets as layers
(4)</em>.</li>
</ol>
<p><img src="images/qgis_cloud_native_geospatial/satellite7.png" width="75%" style="display: block; margin: auto;" /></p>
<ol start="8" style="list-style-type: decimal">
<li>The selected bands will be loaded as new layers in the main QGIS
canvas. Each band is a Cloud-Optimized GeoTIFF (COG) that is hosted on
AWS and being streamed to your QGIS. We can combine multiple bands into
a single image and visualize it. These are called <em>Color
Composites</em>. Let’s stack the Red, Green and Blue bands to create an
natural color composite. Go to <strong>Processing → Toolbox</strong>.
From the <em>Processing Toolbox</em>, search and locate the <strong>GDAL
→ Raster miscellaneous → Build virtual raster</strong> algorithm.
Double-click to open it.</li>
</ol>
<p><img src="images/qgis_cloud_native_geospatial/satellite8.png" width="75%" style="display: block; margin: auto;" /></p>
<ol start="9" style="list-style-type: decimal">
<li>This algorithm creates a new layer in the <a
href="https://gdal.org/en/latest/drivers/raster/vrt.html">Virtual
Raster</a> format which allows you to combine multiple raster layers in
to a single file without consuming extra disk space. Select
<code>Blue (band 2)</code>, <code>Green (band 3)</code>,
<code>Red (band 4)</code> as <em>Input layers</em> and check <em>Place
each input file into a separate band</em>. Click the <code>...</code>
button next to <em>Virtual</em> and save the output file as
<code>rgb.vrt</code>. Next click <em>Run</em>.</li>
</ol>
<p><img src="images/qgis_cloud_native_geospatial/satellite9.png" width="75%" style="display: block; margin: auto;" /></p>
<ol start="10" style="list-style-type: decimal">
<li>A new layer <code>rgb</code> will be added to the <em>Layers</em>
panel. This layer contains references to the 3 different images. Note
that the order of the bands in alphabetical, so the mapping in the
virtual raster is as follows: <em>Band 1 → Blue (band 2)</em>, <em>Band
2 → Green (band 3)</em> and <em>Band 3 → Red (band 4)</em>. Open the
<em>Layer Styling Panel</em> and place <code>Band 3</code>,
<code>Band 2</code> and <code>Band 1</code> as <em>Red</em>,
<em>Green</em> and <em>Blue</em> bands. Change the <em>Min</em> and
<em>Max</em> values to <code>0</code> and <code>3000</code>
respectively. You will see the image now appear in natural colors.</li>
</ol>
<p><img src="images/qgis_cloud_native_geospatial/satellite10.png" width="75%" style="display: block; margin: auto;" /></p>
<ol start="11" style="list-style-type: decimal">
<li>We can also do some analysis with the satellite image bands. Let’s
calculate the <strong>Modified Normalized Difference Water Index
(MNDWI)</strong>. This is a widely used spectral index for identifying
and extracting water pixels from satellite images. This index is
calculated by taking the normalized ratio of the Green (band 3) and the
SWIR-1 (band 11) bands. From the <em>Processing Toolbox</em>, search and
locate the <strong>GDAL → Raster analysis → Raster calculator
(virtual)</strong> algorithm. Double-click to open it.</li>
</ol>
<p><img src="images/qgis_cloud_native_geospatial/satellite11.png" width="75%" style="display: block; margin: auto;" /></p>
<ol start="12" style="list-style-type: decimal">
<li>In the <em>Raster calculator (virtual)</em> dialog, click the
<code>..</code> button for <em>Input layers</em>.</li>
</ol>
<p><img src="images/qgis_cloud_native_geospatial/satellite12.png" width="75%" style="display: block; margin: auto;" /></p>
<ol start="13" style="list-style-type: decimal">
<li>Select the <code>Green (band 3)</code> and the
<code>SWIR 1 (band 11)</code> layers and click <em>OK</em>.</li>
</ol>
<p><img src="images/qgis_cloud_native_geospatial/satellite13.png" width="75%" style="display: block; margin: auto;" /></p>
<ol start="14" style="list-style-type: decimal">
<li>Next click the <em>Expression</em> button.</li>
</ol>
<p><img src="images/qgis_cloud_native_geospatial/satellite14.png" width="75%" style="display: block; margin: auto;" /></p>
<ol start="15" style="list-style-type: decimal">
<li>Remember the formula for MNDWI is (Green - SWIR1)/(Green + SWIR1).
You can click on the layer names to enter it in the <em>Raster
Calculator Expression</em> and construct the expression as shown below.
Once done, click <em>OK</em>.</li>
</ol>
<pre><code>(&quot;Green (band 3) - 10m@1&quot; - &quot;SWIR 1 (band 11) - 10m@1&quot;)
/ (&quot;Green (band 3) - 10m@1&quot; + &quot;SWIR 1 (band 11) - 10m@1&quot;)</code></pre>
<p><img src="images/qgis_cloud_native_geospatial/satellite15.png" width="75%" style="display: block; margin: auto;" /></p>
<ol start="16" style="list-style-type: decimal">
<li>Enter the <em>Output layer name</em> as <code>MNDWI</code> and click
<em>Run</em>.</li>
</ol>
<p><img src="images/qgis_cloud_native_geospatial/satellite16.png" width="75%" style="display: block; margin: auto;" /></p>
<ol start="17" style="list-style-type: decimal">
<li>A new layer <code>MNDWI</code> will be added to the <em>Layers</em>
panel. Click the open the <em>Layer Styling Panel</em> button. Change
the renderer to <code>Singleband psuedocolor</code>. Set the
<em>Min</em> and <em>Max</em> values to <code>0</code> and
<code>0.8</code> respectively. Choose the <code>Blues</code> as the
<em>Color ramp</em>. You will see a visualization where all detected
water pixels are highlighted in the blue color.</li>
</ol>
<p><img src="images/qgis_cloud_native_geospatial/satellite17.png" width="75%" style="display: block; margin: auto;" /></p>
<ol start="18" style="list-style-type: decimal">
<li>The MNDWI image contains values in the range -1 to +1 with pixel
values 0 and above typically indicating presence of water. We can
extract the water pixels to a new layer by applying a threshold of 0.
From the <em>Processing Toolbox</em>, search and locate the <strong>GDAL
→ Raster analysis → Raster calculator (virtual)</strong> algorithm.
Double-click to open it.</li>
</ol>
<p><img src="images/qgis_cloud_native_geospatial/satellite18.png" width="75%" style="display: block; margin: auto;" /></p>
<ol start="19" style="list-style-type: decimal">
<li>Select the <code>MNDWI</code> layer as the <em>Input layers</em> and
enter the expression shown below. Set the <em>Output layer name</em> to
<code>Water</code> and click <em>Run</em>.</li>
</ol>
<pre><code>&quot;MNDWI@1&quot; &gt; 0</code></pre>
<p><img src="images/qgis_cloud_native_geospatial/satellite19.png" width="75%" style="display: block; margin: auto;" /></p>
<ol start="20" style="list-style-type: decimal">
<li>A new layer named <code>Water</code> will be added to the
<em>Layers</em> panel. Click the open the <em>Layer Styling Panel</em>
button. Set the <em>Min</em> and <em>Max</em> values to <code>0</code>
and <code>1</code> respectively. You will see all the extracted water
pixels in white. Remember this image is still a virtual image being
computed on-the-fly by streaming in the required data directly from the
cloud-hosted COGs. You can Zoom and Pan the map and the water pixels
will be extracted dynamically. We can save the results as a local
GeoTIFF. Right-click on the <code>Water</code> layer and select
<strong>Export → Save As…</strong>.</li>
</ol>
<p><img src="images/qgis_cloud_native_geospatial/satellite20.png" width="75%" style="display: block; margin: auto;" /></p>
<ol start="21" style="list-style-type: decimal">
<li>Click the <code>..</code> button next to <em>File name</em> and save
the file as <code>extracted_water.tif</code>. Expand the <em>Extent</em>
section and click the <em>Map Canvas Extent</em> button. Set both both
<em>Horizontal</em> and <em>Vertical</em> values for <em>Resolution</em>
to be <code>10</code> meters. Check the <em>Create options</em> box and
select <code>High Compression</code> as the <em>Profile</em>. Click
<em>OK</em> to start the export process. At this stage, QGIS will fetch
the required pixels from the cloud dataset at the required resolution,
perform the computation to calculate MNDWI and extract the water pixels.
Depending on how big is your region, this process may take a few
minutes.</li>
</ol>
<p><img src="images/qgis_cloud_native_geospatial/satellite21.png" width="75%" style="display: block; margin: auto;" /></p>
<ol start="22" style="list-style-type: decimal">
<li>Once complete, a new layer <code>extracted_water</code> will be
added to the <em>Layers</em> panel. You were able visualize a satellite
image and perform a complex calculation to extract water from the image
without first having to download the entire scene. Saving time,
bandwidth and computation resources.</li>
</ol>
<p><img src="images/qgis_cloud_native_geospatial/satellite22.png" width="75%" style="display: block; margin: auto;" /></p>
</div>
</div>
<div id="data-credits" class="section level1">
<h1>Data Credits</h1>
<ul>
<li>Global Edge-matched Subnational Boundaries: Humanitarian Edge
Matched, FieldMaps, OCHA, geoBoundaries, U.S. Department of State,
OpenStreetMap. Downloaded from <a href="https://fieldmaps.io/data"
class="uri">https://fieldmaps.io/data</a></li>
<li>Sentinel-2 Level-2A: Contains Copernicus Sentinel data.</li>
</ul>
</div>
<div id="license" class="section level1">
Expand Down
Binary file modified images/qgis_cloud_native_geospatial/satellite1.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified images/qgis_cloud_native_geospatial/satellite10.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified images/qgis_cloud_native_geospatial/satellite11.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified images/qgis_cloud_native_geospatial/satellite12.png
Binary file modified images/qgis_cloud_native_geospatial/satellite13.png
Binary file modified images/qgis_cloud_native_geospatial/satellite14.png
Binary file modified images/qgis_cloud_native_geospatial/satellite15.png
Binary file modified images/qgis_cloud_native_geospatial/satellite16.png
Binary file modified images/qgis_cloud_native_geospatial/satellite17.png
Binary file modified images/qgis_cloud_native_geospatial/satellite18.png
Binary file modified images/qgis_cloud_native_geospatial/satellite19.png
Binary file modified images/qgis_cloud_native_geospatial/satellite2.png
Binary file modified images/qgis_cloud_native_geospatial/satellite20.png
Binary file modified images/qgis_cloud_native_geospatial/satellite21.png
Binary file modified images/qgis_cloud_native_geospatial/satellite22.png
Binary file modified images/qgis_cloud_native_geospatial/satellite3.png
Binary file modified images/qgis_cloud_native_geospatial/satellite4.png
Binary file modified images/qgis_cloud_native_geospatial/satellite5.png
Binary file modified images/qgis_cloud_native_geospatial/satellite6.png
Binary file modified images/qgis_cloud_native_geospatial/satellite7.png
Binary file modified images/qgis_cloud_native_geospatial/satellite8.png
Binary file modified images/qgis_cloud_native_geospatial/satellite9.png
Loading

0 comments on commit b30cc55

Please sign in to comment.