Adaptation of the Geospatial Foundation Model Prithvi for Crop Classification using the ZueriCrop Dataset.
Crop classification, Geospatial foundation models, Prithvi, Fine-tuning, Remote sensing, Earth observation, Deep learning, ZueriCrop dataset, Hierarchical classification, Time series, Stratified fold-splitting, Multi-label data with counts
The main objective of this work is to adapt the geospatial foundation model Prithvi to the ZueriCrop dataset and evaluate its limitations, capabilities, and challenges during fine-tuning. Rather than developing a fully automated system for practical applications such as validating direct payments to farmers, we aim to gain insights that can support future developments in this field. Specific objectives include fine-tuning the Prithvi model and investigating the impact of higher temporal resolution as well as a new label hierarchy based on crop seasonality.
Crop classification using aerial and satellite images is a key technology for agricultural management. Current approaches utilize machine learning and deep learning to analyze these images, but they often require extensive, annotated datasets and face limitations in transferring to other regions. Geospatial foundation models, based on large pre-trained datasets, offer the potential to overcome these challenges. So far, the Prithvi model, a geospatial foundation model, has primarily been used for analyzing satellite data from the USA. Adapting this model to European data, particularly the ZueriCrop dataset from Switzerland, has not yet been explored.
The fine-tuned Prithvi model, implemented in our Messis architecture, achieved an F1-score of 34.8% across 48 different crop types. This represents a doubling of performance compared to randomly initialized weights. Our experiments show that the model can be successfully applied to crop classification despite a significantly imbalanced dataset and adaptation to European satellite data from Switzerland. Furthermore, introducing a new label hierarchy based on seasonality led to additional performance improvements.
Institute for Data Science (I4DS), FHNW
Hochschule für Technik
Bahnhofstrasse 6
5210 Windisch, Schweiz
https://www.fhnw.ch/en/about-fhnw/schools/school-of-engineering/institutes/institute-for-data-science
Yvo Keller
Florin Barbisch
Prof. Dr. Martin Melchior
E-Mail: martin.melchior@fhnw.ch
Roman Studer
E-Mail: roman.studer@fhnw.ch
