Our applications of Computer Vision strategies to Agricolture (Olive trees plantations)
Abstract — Recent years have seen an increase of computer vision techniques applied to agriculture, especially to fruit detection and yield forecasting. However, in the case of olive orchards little work has been done. Olives are one relevant agricultural production of Southern Europe and North Africa, but olive trees present specific challenges that make the problem of yield estimation with computer vision techniques more challenging with respect to typical horticultural productions.
Despite this difficulty, the economic advantage of yield estimation for this type of cultivations makes the problem worth of consideration, and recent advancements, in particular in the field of segmentation with Deep Neural Networks, show promising results.
The aim of this work is twofold: first, we want to assess the capabilities of state-of-the-art techniques, based on RGB sensors, with respect to the problem of detecting olive flowers and fruits (called drupes). Secondly, we devise Encoder-Decoder Convolutional Neural Network models to perform image segmentation and improve precision and recall performance to ease the yield estimation task. To compare the approaches and validate the proposed approach we also gather datasets of different olive tree images and videos, collected in different illumination conditions and containing different cultivars (varieties).
Description
Olives are a relevant agricultural product in North Africa, Southern Europe and Middle East, especially for olive oil production. The automation of existing olive oil production pipelines or the application of precision farming protocols to olive orchards is of great economic interest and poses challenging open questions, mainly related to monitoring and forecasting of production.
In this context, computer vision techniques, eventually using autonomous robotic platforms, could be used to measure the inflorescence (and thus the pollination) in early spring and drupe development during summer, in order to allow for a more precise treatments planning and to forecast production. Current state-of-the-art vision system for fruit detection and yield estimation, and eventually harvesting, have been tested recently with success on cultivation big, round and generally not green fruits. In addition, many computer vision based systems for agricultural applications are tested under controlled light conditions (i.e. artificial illumination and absence of shadows) or rely on expensive Infra-Red sensors.
However, for olive flowers and fruits, many of the above considerations do not hold. In fact, olive flowers and fruits are small, tightly grouped and have the same color and very similar texture to tree leaves. In addition, olive trees plantations are usually located on difficult terrain, with high slopes, such as cliffs an hillsides, thus making the use of illumination control strategies infeasible. For these reasons, the development of robust and cost-effective computer vision based solutions for olive production is of great interest. In this project, we aim to address this topic, exploring different techniques and algorithms.
Our main contributions summarizes in the following points:
-
We collect two datasets of olive flowers and fruits RGB images taken under natural illumination conditions.
-
We train a Conditional Random Field (CRF) and a Support Vector Machine (SVM) on the basis of handcrafted feature, namely HOG, LBP and RGB, to discriminate between fruit/flowers and non-fruits/flowers pixels in each image
- We adapt and train a state-of-the-art segmentation Convolutional Neural Network (CNN) for pixel-wise segmentation of RGB images
Network Architecture
CNN based approaches are variants of the popular SegNet architecture. The original network architecture consists of a sequence of four encoding and decoding layers followed by a pixel-wise classification layer.
We use the original SegNet architecture trained with our olive flowers and drupes dataset (OliveSegNet, OSN), and we set also reduced variants of it:
-
Small-OliveSegNet (Small-OSN), where the number of channels in each convolutional layer is reduced from the original value of 64 to 16.
-
Tiny-OliveSegNet (Tiny-OSN), that presents a further reduction of the number of channels in each convolutional layer, to a total of 8 channels.
-
Ripped-OliveSegNet (Ripped-OSN), which has convolutions of 64 channels and where the last coder and the first decoder layers are removed.
Results
state-of-the-art techniques based on handcrafted features, improving performances by up to 25.09% precision and 14.74% recall.
Flowers images
The flowers of the olive trees are small, grouped and light-yellow colored. It is usually quite difficult to distinguish them from the foliage and, in general, the background. Our preliminary results on the flower images dataset are encouraging, especially those of the Segmentation CNN architectures we analysed.
Also from a qualitative point of view it is observed that the output of OSN is less noisy and provides more accurate segmentation with respect to CRF and, more definitely, to SVM.
|
|
F1 Score
|
Precision [%]
|
Recall [%]
|
Accuracy [%]
|
|
CRF
|
0.0859
|
73.97
|
4.56
|
93.83
|
|
SVM
|
0.3974
|
20.89
|
58.16
|
83.33
|
|
OSN
|
0.3824
|
35.53
|
50.86
|
91.88
|
|
Small-OSN
|
0.3815
|
29.37
|
69.71
|
88.28
|
|
Tiny-OSN
|
0.4058
|
31.53
|
70.98
|
88.87
|
|
Ripped-OSN
|
0.4004
|
32.63
|
72.95
|
86.89
|
 |
 |
 |
 |
 |
|
Test image
|
Ground truth
|
SVM result
|
CRF result
|
OSN result
|
FRUITS IMAGES
The fruits of the olive trees have texture, color and size very similar to the leaves. For this reason it is very challenging to detect them from a RGB image. However, our preliminary results on the fruits images dataset are encouraging, especially those of the Segmentation CNN architectures we analysed.
Also from a qualitative point of view it is observed that the output of OSN is less noisy and provides more accurate segmentation with respect to CRF and, more definitely, to SVM.
| F1 Score | Precision [%] | Recall [%] | Accuracy [%] | |
| CRF | 0.3070 | 20.86 | 58.18 | 83.31 |
| SVM | 0.2744 | 17.93 | 58.45 | 72.28 |
| OSN | 0.3962 | 43.02 | 47.59 | 90.81 |
| Small-OSN | 0.4012 | 34.80 | 59.11 | 88.99 |
| Tiny-OSN | 0.3595 | 30.22 | 58.42 | 86.01 |
| Ripped-OSN | 0.3634 | 32.35 | 62.92 | 83.53 |
 |
 |
 |
 |
 |
| Test image | Ground truth | SVM result | CRF result | OSN result |
Download
We will release the code used forour experiments, as well as the collected datasets.