Future access to the Satellite Receiving Station

[ohnonot]

Hello,
found on this page:

The funding provided by NERC for the Satellite Receiving Station will end after March 2019.
For access to our website, images, data and archive after this date please contact us.
If we do not hear from you then the receiving station may close permanently.

Please drop them a line!
Either via their comment system or directly via email: [email protected].

I wrote this:

Hello,
thank you for providing this service!
I am only a hobbyist who recently started using your cloud maps.
However, there's probably a few more out there who would like to continue using this service.
Here's a script that generates a global cloudmap from your data:
github.com/jmozmoz/cloudmap
And here's a repository that offers that global map to anyone who might need it:
github.com/apollo-ng/cloudmap
The latter wouldn't even show up as traffic on your site, which is unfortunate.
The generated map is particularly useful for Xplanet, a powerful opensource application, developed since the late nineties and available for all major operating systems.

PS: And thanks to apollo-ng for providing this cloudmap!

[chron0]

Ahoy, yeah, we've already seen it a couple of days back and I already wrote to them something similar. Thank you for taking the time to notify us regardless and even more for making a voice towards dundee. It would really be a damn shame to lose it, since as of now it seems to be still the only source for the data without having to run a hefty subscription fee and invest in receiving hardware.

[chron0]

Hello,

Thank you for contacting us regarding the future of our facility and for your comments.

I am afraid there is no guarantee that the service will continue after March 2019, but we are making every effort to find a way of funding operations after that and remain hopeful. If we can find a way forward it is very likely that it will have to include subscription fee based access to our data in future. Our intention would be to keep any fees as low as possible, particularly for individuals and educational use.

We are noting user feedback we receive and will provide an update on our plans as soon as possible by email and/or website notification.

Best regards,

Neil.

---

Neil Lonie
Satellite Receiving Station
University of Dundee

Tel. +44 1382 384409, Fax. +44 1382 386726
Email: [email protected]
Web: http://www.sat.dundee.ac.uk/

Scottish Registered Charity no: SC015096

---

[steviehs]

Hmm... reading the above and seeing, that the latest commit is now 9 days old: is this way of getting a cloudmap also closed now? I really hesitate to go for a payed service for my desktop background :-)

[chron0]

damn... I'll check if that is just a bot/download hickup or if we're finally cut off for good :/

[chron0]

yeap, looks like its ded. RIP.

[steviehs]

Thx, I tell you, 640x350 clouds on a 2560x1440 desktop look poor :-)

[chron0]

I know exactly what you mean - that's precisely why I created this 5 years ago in the first place. Feels like a lot is moving backwards now...

[chron0]

how does it look like in xplanet now? 640x350 is really a joke of a resolution, naturally 1222x639 will look better with twice the resolution. what do you use to crop imagemagick?

[ohnonot]

Since clouds are sort of blurry anyhow, it should be possible to upscale a lo-res image in a way that it at least looks good enough.

Are people here trying to use the original software directly?

There's also suggestions for alternative sources.

[jmozmoz]

The image at UW uses the Mollweide projection. I wrote a small script which converts it so xplanet can use it. Here is the code. I just hesitating to implement this in CreateCloudMap because I do not know what are the conditions/license requirements for using those images and directly scrapping their web site.

#!/usr/bin/env python
# coding: utf-8

# In[1]:


import numpy as np
import PIL
import matplotlib.pyplot as plt
from mpl_toolkits.basemap import Basemap
from pylab import rcParams


# In[2]:


get_ipython().run_line_magic('matplotlib', 'inline')
rcParams['figure.figsize'] = 7.5,5


# In[3]:


pil_im = PIL.Image.open('images/test/201909042100.gif', 'r')
pil_im_hsv = pil_im.convert('HSV')
pil_im_gray = pil_im.convert('L')


# Display the satellite image with the coastline grid and as gray-scale

# In[4]:


plt.imshow(pil_im)
plt.show()
plt.imshow(np.array(pil_im_gray))
plt.show()


# Check if the coastline can be identified by its color in the H-channel of HSV representation

# In[6]:


n, bins, patches = plt.hist(np.array(pil_im_hsv)[:, :, 0])
plt.show()

n, bins, patches = plt.hist(np.array(pil_im_hsv)[:, :, 1])
plt.show()

n, bins, patches = plt.hist(np.array(pil_im_hsv)[:, :, 2])
plt.show()


# In[53]:


# create mask for with borders

mask = pil_im_hsv.split()[0].point(lambda h: 0 if h < 50 else 255)
plt.imshow(mask, cmap=plt.cm.gray)


# In[54]:


# round away grid
# picture_count += 1
# plt.subplot(num_pictures, 1, picture_count)
from PIL import ImageFilter
im2 = pil_im_gray.filter(ImageFilter.MedianFilter(15))
plt.imshow(im2)


# In[59]:


im4 = PIL.Image.composite(
    im2.convert('RGB'), 
    pil_im.convert('RGB'),
    mask)

plt.imshow(im4)


# In[60]:


#pil_im = PIL.Image.open('201909042100.gif', 'r').convert('L')

w, h = im4.size
im4 = im4.crop((40, 57, w-40, h-58))
plt.imshow(im4);#, cmap=plt.cm.gray)


# In[61]:


def pc():
    from pyresample import geometry
    """Create area defintion for world map in Plate Carree projection"""
    proj_dict = {'proj': 'eqc'}
    area_extent = (-20037508.34, -10018754.17, 20037508.34, 10018754.17)
    x_size = 2048
    y_size = 1024
    pc = geometry.AreaDefinition('pc', 'Plate Carree world map', 'pc',
                                 proj_dict, x_size, y_size, area_extent)
    return pc


# In[62]:


pc()


# In[63]:


from pyresample import image, geometry
import pyresample

data = np.array(im4)

x_size = data.shape[1]
y_size = data.shape[0]


# In[64]:


area_extent = (
    -20037508.34*0.90, 
    -10018754.17*0.875, 
    20037508.34*0.90, 
    10018754.17*0.875
)

proj_dict = {'lon_0': '0.0', 
             'x_0': '0.0', 
             'y_0': '0.0',
             'proj': 'moll', 
             'ellps': 'WGS84', 
             'units': 'm'}
#Proj4js.defs["ESRI:54009"] = "+proj=moll +lon_0=0 +x_0=0 +y_0=0 +ellps=WGS84 +datum=WGS84 +units=m +no_defs";

rcParams['figure.figsize'] = 15,8
area = geometry.AreaDefinition('moll', 'moll', 'moll',
                               proj_dict, x_size,
                               y_size, area_extent)
from pyresample import plot
# plot.show_quicklook(area, np.array(data), num_meridians=10, num_parallels=10, coast_res='l')


# In[68]:


dataIC = image.ImageContainerQuick(data, area)
dataResampled = dataIC.resample(pc())


# In[83]:


dataResampledImage = dataResampled.image_data
outheight = 1024
polar_height = int(50.0 / 1024.0 * outheight)
north_pole_indices = range(0, polar_height)
north_pole_copy_indices = range(2 * polar_height, polar_height, -1)
dataResampledImage[north_pole_indices, :] =    dataResampledImage[north_pole_copy_indices, :]
south_pole_indices = range(outheight - polar_height,
                           outheight)
south_pole_copy_indices = range(outheight - polar_height,
                                outheight - 2 * polar_height,
                                -1)
dataResampledImage[south_pole_indices, :] =     dataResampledImage[south_pole_copy_indices, :]


# In[84]:


from pyresample import plot
rcParams['figure.figsize'] = 15,10
bmap = plot.area_def2basemap(pc(), resolution='c')
bmap.drawcoastlines(linewidth=0.5, color='red')
# bmap.drawmeridians(np.arange(-180, 180, 45), linewidth=0.2, color='red')
# bmap.drawparallels(np.arange(-90, 90, 10), linewidth=0.2, color='red')
bmap.imshow(dataResampled.image_data, origin='upper', vmin=0, vmax=255,
            cmap=plt.cm.Greys_r);  # @UndefinedVariable


# In[11]:


fig = plt.figure(num=None, figsize=(12, 8) )
m = Basemap(projection='moll',lon_0=0,resolution='c')    
m.drawcoastlines()
m.fillcontinents(color='tan',lake_color='lightblue')
# draw parallels and meridians.
m.drawparallels(np.arange(-90.,91.,30.),labels=[True,True,False,False],dashes=[2,2])
m.drawmeridians(np.arange(-180.,181.,60.),labels=[False,False,False,False],dashes=[2,2])
m.drawmapboundary(fill_color='lightblue')
plt.title("Mollweide Projection");


# In[12]:


w,h = fig.canvas.get_width_height()
w,h


# In[13]:


fig.canvas.draw ( )
buf = np.frombuffer  ( fig.canvas.tostring_argb(), dtype=np.uint8 )
buf.shape = ( h, w, 4 )

# canvas.tostring_argb give pixmap in ARGB mode. Roll the ALPHA channel to have it in RGBA mode
buf = np.roll ( buf, 3, axis = 2 )


# In[14]:


plt.imshow(buf[130:450, 100:800])
plt.axis('off')


# In[ ]:

[jmozmoz]

The above script gets rid of the gird by converting the image into HSV color scape and removing the colored grid.

Another possibilty would be to use the colored IR images from https://www.aviationweather.gov/satellite/intl?region=a&type=ircol&date=

Then you could use same method to get rid of the pure white grid and at the end convert the colored satellite image into BW.

[chron0]

@jmozmoz: nice - I have no reservations about or issues with "copyright" in this particular segment. Afaik US > DE/EU there, anyways because the data should be legally bound to be public as in public domain. And this is exactly my view on how meteorological data should be available to everyone - since it's all financed by tax payers money anyways. This is specially needed in a post factum world, where making sense of "truth" is becoming harder instead of easier. RAW, untainted source data should be one of the corner stones, cloud data is no exception there. I would continue to run the cloudmap service and pull the image from anywhere we can with as high as resultion that we can get. It's a shame to lose high res we used to have before but this would still be better than no data at all imho.

[chron0]

If you would update or create a separate branch in createcloudmap I could just re-enable the cron to pull and store the image here again, leaving the mechanic for everyone using this repo as a mirror just as it is and also save our source from too much individual traffic from single users

[chron0]

++ thanks for sharing - I'll see if I can archive the olkd repo and start a new one from scratch so that we have a new archive and offload traffic from boff.in

[chron0]

ah so this is not what you use but this actually your site? it still says dundee tho?

[chron0]

where do you pull the source images?

[chron0]

https://clouds.boff.in/ seems to be down anyways

[chron0]

https://www.gofundme.com/f/dundee-satellite-receiving-station

[krkeegan]

This is a script I wrote a while back that pulls the image tiles from UW and stitches them together. For anyone still looking for a solution, you can get a free API key from the University that will enable you to generate 8192x4096 images hourly.

https://gist.github.com/krkeegan/64e96290eb6569790d230085016501da

[matteason]

Hi all - I've been looking for live-ish cloud maps like the ones this project generated and haven't had much luck, so I've created my own: https://github.com/matteason/daily-cloud-maps

It generates cloud maps from NASA (NOAA/VIIRS) data via their official APIs so I'm hopeful it'll continue working for a while. Hopefully it's useful to anyone still looking for a solution