Future access to the Satellite Receiving Station

[ohnonot]

Hello,
found on this page:

The funding provided by NERC for the Satellite Receiving Station will end after March 2019.
For access to our website, images, data and archive after this date please contact us.
If we do not hear from you then the receiving station may close permanently.

Please drop them a line!
Either via their comment system or directly via email: [email protected].

I wrote this:

Hello,
thank you for providing this service!
I am only a hobbyist who recently started using your cloud maps.
However, there's probably a few more out there who would like to continue using this service.
Here's a script that generates a global cloudmap from your data:
github.com/jmozmoz/cloudmap
And here's a repository that offers that global map to anyone who might need it:
github.com/apollo-ng/cloudmap
The latter wouldn't even show up as traffic on your site, which is unfortunate.
The generated map is particularly useful for Xplanet, a powerful opensource application, developed since the late nineties and available for all major operating systems.

PS: and thanks to jmozmoz for providing this script!

[dirteat]

Damned, we certainly will need a new source of cloud maps.

It is quite amazing that our taxes are used to finance the satellites producing these maps, but we cannot actually get the bloody maps afterwards, they do not seem to be public domain...

[jmozmoz]

There are some possible sources for images:
Overview of services: https://dbaron.org/sat/comp/

https://www.aviationweather.gov/satellite (all images seem to have a grid overlayed)

https://a.atmos.washington.edu/data/weather.html#satellite (also with grid)
https://atmos.uw.edu/cgi-bin/latest.cgi?ir-moll

https://www.ssec.wisc.edu/data/geo/#/animation?satellite=fy2g

https://www.ssec.wisc.edu/data/geo/#/animation?satellite=goes-17-fusion&end_datetime=latest&n_images=1&coverage=fd&channel=12&image_quality=gif&anim_method=javascript

The latter also has coast lines overlayed. But it seems to be the complete collection of images. Unfortunately, it does not give any license/usage information.

[jmozmoz]

The dundee website seems really be dead since end of August ("Inactive since 21 Aug 2019")

For a first implementation to use a new image source see apollo-ng/cloudmap#1

[jmozmoz]

Another source:
https://en.allmetsat.com/index.html

[jmozmoz]

And yet another source:
https://www.nrlmry.navy.mil/sat-bin/global.cgi

[dirteat]

Things move. Have you seen those ones? Very nice, full disk very high resolution, a lot of channel, free download:

https://www.star.nesdis.noaa.gov/GOES/fulldisk.php?sat=G16
https://www.star.nesdis.noaa.gov/GOES/fulldisk.php?sat=G17

Still we would need some equivalent over Europe and Japan, something that does not suck in terms of resolution.

[dirteat]

For hardcore enthusiasts, but still GEOS:

https://pietern.github.io/goestools/guides/minimal_receiver.html#minimal-receiver

[dirteat]

This rocks too:
https://worldview.earthdata.nasa.gov/

[jmozmoz]

I guess, I will wait how this works out:
http://www.dundeesat.co.uk/
https://www.theregister.co.uk/2020/01/02/dundee_satellite_station/

[krkeegan]

I have used this project for ages. @jmozmoz thank you for all of the amazing work!

I have spent the last few weeks trying to track down various reliable sources of full disk images. There are a lot of sporadic locations that maybe one could piece together, but it will always be prone to breaking when one location goes offline or changes, as we have seen.

In the process I found RealEarth from the University of Wisconsin Space Science and Engineering Center http://re.ssec.wisc.edu/. RealEarth has a web api for requesting and downloading the images we need http://re.ssec.wisc.edu/doc/api.php. In addition, they also provide a tiling service that provides the images directly without requiring any API http://re.ssec.wisc.edu/doc/dataaccess.php#tiles.

Without any account, they allow 500MB per day of downloading, which seems to be metered by IP address. Additionally, with a free account they allow up to 1GB per day.

They do offer the full disk images, albeit in a projected form. And one could download each and perform the appropriate high latitude cleanup and blending.

But they also offer a full IR band composite image of the world updated every hour http://re.ssec.wisc.edu/products/globalir. The stitching is pretty good and could probably be massaged relatively easily with a mask if desired. They offer various zoom levels. At zoom level 5 you have to download 512 images at 256x256 resolution each for a total resolution of 8192x8192. This seems to be about the max by my eye. The total download size is about 23MB. This means you could do the download once per hour with an account and stay well under the 1GB daily limit.

I made a mockup of the collage here https://krkeegan.github.io/realearth-example.html. It may take a minute for everything to download, and you make have to zoom out in your browser.

I think I am going to make a simply python script with PIL to download and stitch these images once per hour. It is going to be very basic. I am happy to contribute help to anyone else if they want to start a repo, but my python is all self taught so I don't have the know-how to make a proper package and such.

[dirteat]

Thanks for the link krkeegan!
For clouds.boff.in, it is not in my style to accept stolen things. No code available, no licence available, data taken from "some" public domain, and the guy asked for some money!? WTF...

[krkeegan]

If anyone is interested in running their own script to generate cloud maps, here is a simple python script I put together. The output with the default settings will produce a 4096*2048 image identical to that produced by this project.

The script is here https://gist.github.com/krkeegan/64e96290eb6569790d230085016501da

Happy coding!

[mikehardy]

@krkeegan your script in that gist is ace! Thank you so much, just integrated it and I have a beautiful desktop background again :-)

[lmcmicu]

Thanks @krkeegan. FYI it is working with python 3.7.3.

[marneu]

@krkeegan great on 3.8.5, very much appreciated. xplanetFX is back on my monitors, thank you

[matteason]

Hi all - apologies for resurrecting an old issue. I've been looking for cloud maps for a project and was sad to see that you lost your data source. I've built a new project which generates and hosts daily cloud maps from NASA (NOAA/VIIRS) data which might be of interest to some of you: https://github.com/matteason/daily-cloud-maps

[mikehardy]

@matteason you are the hero we needed! 😄

[jmozmoz]

See https://github.com/matteason/live-cloud-maps/

[jmozmoz]

Please try the version from branch https://github.com/jmozmoz/cloudmap/tree/live_cloud_maps

It uses the cloud map provided by https://clouds.matteason.co.uk/ but checks before downloading the full image if it is newer than the latest downloaded.

Here is the installation package for pip:

CreateCloudMap-0.9.13+21.gfe1600d.tar.gz

[krkeegan]

Nice, the cloud images seem "denser." I suspect this is a difference in the wavelength used for cloud detection. Not something in the purview of this project, but something I will followup with matteason about.

[matteason]

Nice, the cloud images seem "denser." I suspect this is a difference in the wavelength used for cloud detection. Not something in the purview of this project, but something I will followup with matteason about.

Yeah, I've found that purely using greyscale IR results in a lot of missing clouds - cold clouds are closer to white and warm earth/sea are closer to black, but in between, the same grey value could be warm cloud or cool earth so it's hard to process into something that looks accurate.

I use three images from EUMETSAT:

IR: https://view.eumetsat.int/geoserver/ows?service=WMS&request=GetMap&version=1.3.0&layers=mumi:worldcloudmap_ir108&styles=&format=image/png&crs=EPSG:4326&bbox=-90,-180,90,180&width=2048&height=1024

Dust: https://view.eumetsat.int/geoserver/ows?service=WMS&request=GetMap&version=1.3.0&layers=mumi:wideareacoverage_rgb_dust&styles=&format=image/png&crs=EPSG:4326&bbox=-90,-180,90,180&width=2048&height=1024

Natural colour: https://view.eumetsat.int/geoserver/ows?service=WMS&request=GetMap&version=1.3.0&layers=mumi:wideareacoverage_rgb_natural&styles=&format=image/png&crs=EPSG:4326&bbox=-90,-180,90,180&width=2048&height=1024

All the code is commented in https://github.com/matteason/live-cloud-maps/blob/main/index.js#L179 but basically the IR provides the base layer and I layer the clouds from the visible light image on top of that, which provides a lot more detail in daylight areas - for example most of the clouds in the upper-left quadrant of this image would be lost if only using the IR image

IR:

Visible:

Mine:

Xeric Design's, with data from SSEC RealEarth, UW-Madison - I believe this is purely IR-based

Worth saying I'm not an expert on any of this stuff by any means, my background is in design so I've just been aiming to make something that looks decent without necessarily being completely meteorologically accurate, so any feedback on how it could be improved would be gratefully received