The Summer holidays are coming to an end and as we return I thought I would provide some interesting reading material. I had the pleasure of meeting Dr. Ben Robson a number of years ago before joining the Trimble eCognition team and it is nice to see the relationship come full circle. When I met up with him in Oslo earlier this year he mentioned that he was working on a new publication that would examine the use of deep learning and object-based image analysis (OBIA) and that sparked my curiosity.
I am happy to announce that Ben and his colleagues have now had their research “Automated detection of rock glaciers using deep learning and object-based image analysis” published in Remote Sensing of Environment.
Geologists have a saying – rocks remember.
In this paper, the authors discuss automating the detection of rock glaciers as high resolution images have become more widespread in their availability and allow us to access greater expanses of area. But as Ben points out, “due to the spectral similarity between rock glaciers and the surrounding material, the creation of such inventories is typically conducted based on manual interpretation, which is both time consuming and subjective”. In fact, many existing inventories of rock glaciers are based on high-resolution aerial images in combination with Digital Elevation Models (DEMs) and these are two geographical constraints since such data is often limited in extent and availability. Therefore, he and his colleagues propose a new method of identification using the Trimble eCognition Developer software that allows for the combination of deep learning (convolutional neural networks or CNNs) and OBIA technology based on satellite data.
In this study, they are looking to extract both the location and extent of the rock glaciers. For this purpose, study areas in Northern Chile and the Central Himalaya were chosen. Two different data sets were selected for the analysis: Sentinel-2 imagery with 10m resolution and Pléiades imagery with 2m resolution. The optical data was then combined with DEMs derived from SAR coherence data based on interferometric Sentinel-1 data and tri-stereo Pléades data respectively.
The first step of the image analysis focused on the development of a CNN model that would allow eCognition to learn the defining characteristics of a rock glacier. For the Sentinel-2 analysis the red, green, blue, NIR and SWIR bands, NDVI, MNDWI and SAVI as well as the Sentinel-1 coherence DEM and curvature data were input into the model to train the classes rock glacier, debris-covered glacier, clean ice, stable terrain and shadows. In total, 29,000 CNN sample patches were generated across the various target classes. After several rounds of experimentation, the authors determined the optimal parameters for their CNN model ultimately deciding for an architecture based on a higher number of convolutions with a smaller kernel size as this resulted in a “cleaner heatmap”. After generating the predictive heatmap, a Gaussian filter was applied to smooth the initial results.
A similar approach was taken with the Pléiades data, only adjusting to account for the lack of the SWIR band and the lack of indices that are SWIR-dependent such as the MNDWI. A canny edge filter was applied to the slope layer as well. Other than that, the rest of the analysis remained the same.
After generating the heatmaps, the deep learning results could be seamlessly input into the OBIA analysis. The authors developed a multi-level segmentation and classification approach based on fusing the multispectral, morphology, geometric and contextual data with the deep learning output and objects were then classified as rock glaciers using a mixture of fixed and fuzzy criteria – this is why I love eCognition, data fusion + methodology fusion!
The final results across both sites identified 108 of the 120 rock glaciers in the validation data set, yet the total area of the 108 glaciers was greater than that of the validation data corresponding to an overestimation of 28%. The user’s and producer’s accuracies reported by the authors are 65.9% and 71.4% respectively meaning that a high number of rock glaciers were identified but the classification results contain false positives.
A comparison of the different sensor types indicates that the higher resolution imagery minimally affected the producer’s accuracy (87.4% with Sentinel-2, 88.4% with Pléiades) but significantly improved the user’s accuracy (62.9% with Sentinel-2, 72.0% for Pléiades).
Generally speaking the results indicate “that even although the individual rock glacier polygons were in most cases overestimated, a large proportion of the total rock glaciers were successfully identiﬁed”. The authors go on to discuss the advantages of their methodology, first of all they state that with the CNN-OBIA approach they are not limited to only flowing rock glaciers as it is independent of SAR interferometry data. In addition, it is not dependent on VHR data and can be based on the freely available Sentinel data.
Nevertheless, they also admit there are certain limitations such as the availability of reliable inventory data needed for model training. In addition, the CNN model does result in miss-classifications due to the textural similarity between rock glaciers and debris flows and roch avalanche deposits. The addition of the OBIA refinement to the rule set helped mitigate such false positives to an extent but not completely.
I am always fascinated to read about the different uses of the Trimble eCognition software. It has been great to follow Ben’s work over the past few years and see how he and the team have leveraged the technology. Such projects exemplify the advantages of the eCognition software:
- The combination of different input data sets (optical, elevation, vector)
- The combination of different classification approaches (deep learning, knowledge-based and fuzzy logic)
- Fully automated workflows that include all of the above
I look forward to getting Ben in to present his research in an upcoming eCognition webinar, so stay tuned!