One more great Trimble eCognition story coming at you before we all go our separate ways on Summer vacation!
This piece from Karantanellis et al., 2021, entitled “Evaluation of Machine Learning Algorithms for Object-Based Mapping of Landslide Zones Using UAV Data” provides a wonderful breakdown of the use of two different classification techniques available within eCognition Developer – machine learning (ML) vs. knowledge-based.
The authors chose a representative study site with two rotational landslides in northwestern Greece within an open pit mine area – “the vegetation cover is mainly grass with occasional isolated small bushes; in addition, olive trees are growing in the vicinity”.
The total area of the UAV scene generated by the researchers was around 38 ha. The imagery (RGB) was acquired with a Phantom 4 Pro V2.0 from DJI and the final datasets (optical imagery and DSM) had a spatial resolution of 0.05 m.
After the data collections and processing, the Object-Based Image Analysis (OBIA) phase of the project began; this part of the “workflow includes four distinct phases: (a) segmentation, (b) training samples’ creation, (c) feature selection and tuning, and (d) classification, using Cognition Network programming language (CNL) within the eCognition Developer 10 software”.
Once the segmentation phase was completed with the Multi-resolution segmentation algorithm, the authors explored two different classification approaches – “First, a knowledge-based fuzzy ruleset has been developed based on spectral, spatial, and contextual features. Second, three ML classifiers, i.e., KNN, DT, and RF, have been tested for landslide mapping and zoning using training data from the created reference segments”. The image objects were split into training (60%) and validation (40%) groups to allow the authors to apply a cross-validation analysis approach. The groups were generated based on stratified random sampling. The training objects would subsequently be used as input for the supervised classification models and the validation objects would be used in the quality assessment of both the machine learning and knowledge-based approaches.
The knowledge-based classification method is still a fully automated rule-based approach in eCognition – “The selection of features and tuning of thresholds was determined based on expert knowledge”. A hierarchical classification was applied and “the initial characterization on the coarse segmentation level included the expert-based classification into landslide and non-affected objects, followed by the differentiation of landslide objects into scarp and depletion zones on the fine level”.
For the classification, the authors selected a number of features to describe the target classes. A mix of spectral and shape features were used in combination with information from the surface layers: “three spectral bands from the orthophoto and the brightness value were used as the spectral metrics. The length/width ratio, the asymmetry, and the object’s size were used as the spatial metrics. Slope, aspect, and curvature were used as the morphological metrics”. In addition, 7 Grey-Level Co-occurrence Matrix (GLCM) layers were calculated based on the DSM to account for texture.
The machine learning method examined three different supervised classifiers: K Nearest Neighbors (KNN), Random Forest (RF) and Decision Tree (DT) or Classification and Regression Trees (CART). The models were applied to three different segmentation results based on Scale Parameter. Scale parameters of 25, 100 and 200 were chosen and the subsequent supervised classification applied to each of the image object levels.
To assess the quality of the study, the different classification results were compared with a reference dataset and workflow was tested on a neighboring landslide area to examine the transferability and performance of the developed methods. The authors chose to measure the accuracy of the results based on a “per polygon evaluation method” whereby objects that were correctly classified are true positives (TP), objects that were incorrectly classified are false positives (FP) and false negatives (FN) are referenced areas that were not detected.
In general, the best input data for the analysis was the combination of optical imagery with the DSM and its derivatives (slope, aspect & hillshade). In total, 54 classifications were tested by the authors, of those 13 results in an agreement >75% and 4 with an agreement >80%.
The highest classification performance was recorded with the RF classifier, yielding a recall of 0.83 and a precision of 0.86. The overall model agreement (F1) was 0.85 – “As a result, 85% of the total scene was correctly identified as being either a scarp, depletion, or non-affected zone”.
The knowledge-based classification results were comparable with the DT performance. And, the KNN classifier resulted in the poorest classification.
(a) Orthophoto with the reference zones overlaid; (b) KNN classification result; (c) DT classification result; and (d) RF classification result for the “Prosilio” landslide site (green: non-affected, brown: depletion, red: scarp). Applied configuration: dataset: RGB+DSM, SP: 100, shape/color: 0.6.
In their concluding observations, the authors note the importance of data richness and data fusion. From my perspective, as a software producer, this is a conversation I am often having with users, i.e. “Can I do this or that with just imagery?”. Although imagery is a powerful data source, it is not the perfect source for all feature extraction tasks. As the authors state, “the DSM and its derivatives proved critical for accurate and precise mapping…” and “spectral information solely from the RGB setup was not adequate to classify the segmented scene with supervised procedures in an effective manner”.
I also enjoyed the inclusion of a knowledge-based classification approach since many of the papers I read restrict the analysis to a two phase analysis: 1) segmentation followed by 2) the application of a machine learning algorithm. Although the purely knowledge-based approach did not yield the highest quality assessment values, it still performed well and proved it’s value within a greater workflow, something we refer to as method fusion – the ability to combine supervised, unsupervised, deep learning and knowledge-based with one another in a single rule set.
So in the end, it is not a this vs. that debate, but a question of how can I combine this with that? The great thing about eCognition is that it fully supports the concept of method fusion and that, for me at least, is the beauty of it. Sometimes we can have the best of both worlds!
**Note, the original table with classification accuracy assessment results included in the publication forgot to include the values for the knowledge-based approach. The authors provided me with the data for this blog post and will update the publication as soon as possible.