Transformers

Multi-Modal Learning for Geospatial Vegetation Forecasting

We benchmark a wide range of EarthNet models on the new GreenEarthNet dataset, plus introducing a new transformer-based SOTA: Contextformer.

Vitus Benson, Claire Robin, Christian Requena-Mesa, Lazaro Alonso, Nuno Carvalhais, José Cortés, Zhihan Gao, Nora Linscheid, Mélanie Weynants, Markus Reichstein

Earthformer: Exploring Space-Time Transformers for Earth System Forecasting

We propose Earthformer with novel generic building blocks “Cuboid Attention” to explore the design of space-time attention for Earth system forecasting problems, and achieve SOTA performance on two synthetic datasets and two real-world benchmarks.

Zhihan Gao, Xingjian Shi, Hao Wang, Yi Zhu, Yuyang Wang, Mu Li, Dit-Yan Yeung

Earthformer: Exploring Space-Time Transformers for Earth System Forecasting