[Paper] GenCast Diffusion model for weather forecasting

[jacobbieker]

Arxiv/Blog/Paper Link

https://arxiv.org/abs/2312.15796

Detailed Description

A diffusion-based approach to weather forecasting, that is quite stable autoregressively. Probably fits in more with diffusion_weather but here to be with the other weather papers. They also say the model is based on GraphCast, but with a different graph connectivity, and a sparse transformer instead of the Processor GNN in GraphCast.

Context

Cool way of doing ensemble predictions, scalable, only at a 1 degree resolution though, 12 hour timesteps as well, so less than Graphcast and the like for some reason?

[jacobbieker]

The encoder and decoder are the same as GraphCast, but the latent grid is a 5-refined mesh, not a multi-mesh, with 10242 nodes and 61440 edges.

[jacobbieker]

I think this adds more support to modularizing graph_weather, what is being done in #76, so that it is easier to experiment with/replicate this kind of result

[jacobbieker]

They train on the 12 hour timestep to be in different data assimilation windows, as ERA5 only has 2 a day.

[jacobbieker]

Overall, really impressive results I think. Interesting combination of graph and diffusion model. A lot slower to run and train because of how diffusion models work, and still requires NWP analysis field for initialization. Compared to a few seconds for a 0.25 degree forecast with GraphCast, 1min per forecast with GenCast at 1 degree is a lot slower. But the better results, and it still be a lot faster than traditional methods make it quite interesting.

[jacobbieker]

Would be really keen to implement this here.

[aavashsubedi]

Seems like an interesting project. Is this open as a GSOC project? And if so, what would the scope/length be? (I don't see GraphCast in the repo, so I imagine that would also need to be ported over from DM ?)

[jacobbieker]

Yes, this could work as a GSoC project. It would be a large project (350h). GraphCast wouldn't need to be ported over, we already have the encoder/decoder graph networks implemented, although they definitely can be improved! It would be more the diffusion model that would need to be added. And some changes to make the code more modular, so that we can easily swap out Encoders/Processor/Decoders easily.

[jacobbieker]

They have released an updated paper now, increasing the resolution to 0.25 degrees, and more comparison against ENS, which they claim to beat in 97 percent of the time.

[gbruno16]

This is very cool! Also, it seems that you don't need to restart training from scratch to go from 1 degree to 0.25: just a few small modifications and fine-tuning are enough, similar to graph weather!